External PR Review
External PR review applies Colony’s review executor to a GitHub pull request that is not part of Colony’s issue-to-PR pipeline. It creates a standalone review-external task; it does not create a pipeline issue or move the PR through Colony’s issue states.
Eligibility
Section titled “Eligibility”External review is off by default and is resolved from repository or global review configuration. Colony always skips:
- its own agent branches
- PRs authored by the configured Colony bot account
- PRs carrying any
colony:pipeline label
Configuration can also exclude drafts and bot-type authors (both excluded by default), allow only named authors, or require all labels in label_filter.
When to use each mode
Section titled “When to use each mode”| Mode | Current trigger behavior |
|---|---|
off | Does not enqueue external reviews. |
on_request | Enqueues when a human comments /colony:review on an eligible PR. |
on_ready | Enqueues for an eligible non-draft PR when opened or marked ready; pushes re-review a PR only after it has had an external-review task. |
auto | Uses the same automatic event triggers as on_ready and enables the auto_approve configuration option. |
/colony:review is accepted in any non-off mode, subject to the same filters. Active-task deduplication prevents concurrent duplicate reviews for the same PR.
auto_approve is not an automatic merge action. The external executor submits an ordinary GitHub review, and repository branch protection and merge policy determine what that review permits.
What the review produces
Section titled “What the review produces”The Worker creates a review worktree for the PR branch and default branch, then runs the deterministic commands configured under review.checks.
- a check failure introduced by the PR produces
REQUEST_CHANGESwith check output - failures already present on the base branch produce an informational
COMMENT - a runner environment failure produces an informational
COMMENTand no LLM review - a merge conflict prevents the review from proceeding
If deterministic checks pass, the LLM review receives the PR title, PR body, diff, passed-check names, .colony/conventions.md, and prior Colony review content. It does not look up a linked issue as part of this path.
A parsed approving verdict produces an APPROVE review. A changes verdict produces REQUEST_CHANGES. The body contains a summary, comments, and required or suggested action items. “Confidence flags” are not part of the documented output schema. If Colony cannot parse a verdict after fallback attempts, it posts an informational comment asking for manual review.
Before a successful new LLM verdict is posted, the prior Colony review is dismissed as superseded. In on_request mode, request another review after pushing changes. In on_ready and auto, a synchronize event re-reviews only when Colony has reviewed that PR before.
The current automatic trigger and review-publication path is implemented for GitHub pull requests. Do not infer identical Azure DevOps behavior from Colony’s general code-host abstraction.
See Configuration Schema for review.external_prs and Slash Commands for the command syntax.