Set up External PR Review

External PR review applies Colony’s configured deterministic checks and reviewer pass to eligible GitHub pull requests that were not created by Colony’s pipeline. The feature is off by default.

Before you start

The repository must already be connected and its webhooks must reach the Mayor. Colony also needs GitHub permissions to publish pull-request reviews; in Cloud, install colony-operations for that capability. Configure useful repository review checks before expanding automatic coverage.

Choose a mode:

Mode	Initial trigger
`off`	No external review
`on_request`	A human posts `/colony:review`
`on_ready`	An eligible PR opens ready for review or changes from draft to ready
`auto`	The same automatic initial trigger as `on_ready`

Start with on_request. See External PR Review for the behavioral differences and review outcomes.

Open Settings → Agents (/settings/agents).
In Review → External PR Review, set Mode to on_request.
Optionally restrict eligible GitHub usernames with Author filter.
Optionally set Label filter. A PR must carry every listed label.
Keep bot-authored and draft PR exclusions enabled until you have a reason to broaden the policy.
Save the configuration.

Add the policy to the effective review configuration:

review:
  external_prs:
    mode: on_request
    author_filter: ['*']
    label_filter: []
    exclude_bots: true
    exclude_drafts: true

Restart or redeploy the Colony services that consume configuration and webhooks. With Docker Compose, docker compose up -d applies the current service configuration.

exclude_bots uses the GitHub author’s account type, not a username suffix. Colony always excludes its own bot, agent branches, and PRs carrying a colony: label.

Run the first review

Open an eligible, non-draft PR and post this as a PR comment:

/colony:review

In on_request, repeat the command after new commits when you want another review. In on_ready and auto, a push retriggers review only after Colony has already reviewed that PR; a push alone does not opt a previously unseen PR into review.

If nothing happens, check webhook delivery, the installed app’s permissions, filters, draft/bot exclusions, and Mayor health. A command from a bot account is intentionally ignored.

Interpret the result

Deterministic failures can produce a review without an LLM pass:

merge conflicts prevent a normal review;
a review-environment failure produces an informational comment;
checks newly broken by the PR request changes;
failures already present on the base branch are reported as informational.

When deterministic checks pass, the reviewer considers the PR title, body, diff, repository conventions, check results, and prior reviews. It publishes a summary, findings/action items, and a GitHub verdict of approval, requested changes, or comment. That verdict does not merge the PR. Branch protection and repository merge policy still decide whether it is a required approval.