Skip to content

External PR review

Colony can review human-authored PRs using the same engine it applies internally — deterministic checks followed by an LLM code review. For a conceptual overview of modes, trigger behavior, and what the review produces, see External PR Review.

Configure this feature from Standards → Settings › Review → External PR Review (/standards/settings, with no repository selected). The feature is off by default; you opt in by choosing a mode.

ModeTriggerBest for
offNo automatic reviewsDefault — feature disabled
on_requestA human comments /colony:review on the PRGetting started; explicit opt-in per PR
on_readyPR is opened or transitions from draft to ready-for-reviewTeams that want automatic review without auto-approval
autoSame triggers as on_ready; adds the option to post formal approvalsFully automated review pipelines
  1. Open Standards → Settings in the Colony Cloud dashboard, with no repository selected.
  2. Open the Review section in the section nav and find the External PR Review subsection.
  3. Set the Mode dropdown to your preferred mode.
  4. Configure filters and exclusions as needed (see mode-specific sections below).
  5. Click Save at the bottom of the page.

Changes take effect on the next Worker heartbeat — no Worker restart is required.

Set Mode to When requested (/colony:review comment) (on_request). Colony reviews a PR only when a human explicitly requests it by posting a slash command.

  1. Open any non-draft, non-pipeline PR in your repository.
  2. Post /colony:review as a comment on the PR.
  3. Colony adds a reaction to acknowledge the request.
  4. Colony runs deterministic checks and the LLM review, then posts the results as a GitHub PR review.

By default, the /colony:review command is accepted from accounts with write access to the repository; anyone else gets a reply saying it was refused. If your organization sets commands.authorization to any, any human account can use it. Bot comments are ignored.

The author and label filters and the bot and draft exclusions also apply here. On a PR they exclude, /colony:review does nothing and Colony doesn’t reply.

After you address feedback and push new commits, post /colony:review again to request a fresh review. Colony runs against the updated head commit, dismisses the previous review, and posts a new one.

Set Mode to When a PR is opened or marked ready (on_ready). Colony automatically reviews a PR when it is opened as non-draft, or when it transitions from draft to ready-for-review.

Use the Author filter field to limit automatic reviews to a specific allowlist of GitHub usernames. Enter usernames as comma-separated values (e.g., alice, bob, carol). Leave the field blank to allow all authors.

Use the Label filter field to require specific labels on the PR before Colony reviews it. Enter label names as comma-separated values (e.g., review-please). All listed labels must be present — multiple labels use AND semantics. Leave the field blank to review without label requirements.

Two checkboxes control which PR types are automatically skipped:

  • Exclude bot-authored PRs — checked by default. Colony skips PRs authored by accounts GitHub reports as bots. Uncheck to include bot PRs.
  • Exclude draft PRs — checked by default. Colony skips PRs in draft state. Uncheck to review drafts.

When a contributor pushes new commits to a PR that Colony has already reviewed, Colony automatically re-reviews the updated PR. This only fires if Colony has previously reviewed that PR — new pushes to PRs that have never been reviewed do not trigger an unsolicited initial review.

Set Mode to Automatically, with auto-approve (auto). Triggers are identical to on_ready — Colony reviews every eligible PR when it is opened or transitions to ready-for-review.

The Auto-approve checkbox only works in this mode. Check it to have Colony post a formal Approve review when the LLM verdict is APPROVE.

Regardless of which mode you choose, Colony always skips the following:

  • Colony’s own pipeline PRs — branches matching colony/issue-N and other agent branch patterns are never reviewed.
  • Colony’s bot username — PRs opened by Colony’s configured bot account are skipped.
  • PRs carrying any colony: label — pipeline-managed PRs are excluded.

These exclusions cannot be overridden from the form or any configuration field.

The table below maps each underlying config field to its Colony Cloud form control and default value.

FieldDefaultCloud form fieldDescription
modeoffModeoff, on_request, on_ready, or auto
author_filter(all authors)Author filterComma-separated GitHub usernames eligible for review
label_filter(no requirement)Label filterAll listed labels must be present (AND semantics)
exclude_botsenabledExclude bot-authored PRsSkip PRs authored by accounts GitHub reports as bots
exclude_draftsenabledExclude draft PRsSkip PRs in draft state
auto_approvedisabledAuto-approveIn auto mode, post a formal Approve on an APPROVE verdict

Per-repository overrides are not configurable from Colony Cloud — contact support if you need different external review modes per repository within the same organization.