GitHub App Setup Guide

GitHub App Setup Guide

This guide walks through setting up a GitHub App for Colony authentication. Using a GitHub App (instead of a Personal Access Token) provides fine-grained permissions, higher rate limits, and bot-attributed activity.

Prerequisites

• Admin access to your GitHub organization (or repository owner access for personal repos)
• Colony installed and running with a PAT-based config (to verify the pipeline works first)

Step 1: Create the GitHub App

1. Go to your Organization Settings > Developer settings > GitHub Apps > New GitHub App

   • For personal repos: Settings > Developer settings > GitHub Apps > New GitHub App

2. Fill in the app details:

   • GitHub App name: e.g., colony-bot (must be globally unique)
   • Homepage URL: your org’s URL or https://github.com/<org>/<repo>
   • Webhook: uncheck “Active” (Colony polls; it does not use webhooks)

3. Set Repository permissions:

   Permission	Access	Purpose
   Contents	Read & write	Push branches, read repo files
   Issues	Read & write	Manage issue labels, post comments
   Pull requests	Read & write	Open PRs, post reviews, manage PR lifecycle
   Metadata	Read-only	Required (automatically granted)

4. Under Where can this GitHub App be installed? select “Only on this account”.

5. Click Create GitHub App.

6. Note the App ID displayed on the app’s settings page.

Step 2: Generate a Private Key

1. On the app’s settings page, scroll to Private keys.
2. Click Generate a private key.
3. A .pem file will be downloaded automatically.
4. Move the PEM file to a secure location:

   mkdir -p ~/.colony/keys
   mv ~/Downloads/<app-name>.*.private-key.pem ~/.colony/keys/github-app.pem
   chmod 600 ~/.colony/keys/github-app.pem

Step 3: Install the App on Target Repositories

1. On the app’s settings page, click Install App in the left sidebar.
2. Select your organization.
3. Choose Only select repositories and pick the repos Colony will manage.
4. Click Install.
5. Note the Installation ID from the URL: https://github.com/settings/installations/<INSTALLATION_ID>.

Step 4: Update Colony Configuration

Edit colony.config.yaml to use the App credentials:

github:
  owner: your-org
  repo: your-repo
  # Remove or comment out token_env when using App auth:
  # token_env: GITHUB_TOKEN
  app:
    app_id: 123456                              # From Step 1
    private_key_path: ~/.colony/keys/github-app.pem  # From Step 2
    installation_id: 78901234                   # From Step 3
  bot_username: "colony-bot[bot]"               # See Step 5

You can keep token_env alongside app — Colony will prefer the PAT when both are present, falling back to App auth if the PAT environment variable is not set.

Step 5: Set the Bot Username

The bot_username tells Colony which comments are its own (to avoid processing its own messages as human input). The format is:

<app-slug>[bot]

Where <app-slug> is the lowercased, hyphenated version of your app name. For example:

• App name “Colony Bot” → bot_username: colony-bot[bot]
• App name “My Colony App” → bot_username: my-colony-app[bot]

You can verify the exact slug by checking a comment posted by the app — the commenter login will show the [bot] suffix.

Step 6: Verify

Start Colony and check the startup logs:

npx colony start

Look for:

GitHub App auth active    owner=your-org repo=your-repo

If you see PAT auth active instead, the App credentials were not loaded — double-check the config and PEM path.

To test end-to-end, create a test issue. Colony should post a comment with the bot avatar and <app-slug>[bot] as the author.

Troubleshooting

Symptom	Cause	Fix
Authentication failed: token is invalid or expired	Bad PEM file or wrong app_id	Re-download the PEM; verify app_id matches your app
Authorization failed: token lacks required permissions	Missing repo permissions	Check the app’s permission settings (Step 1.3)
Repository not found or not accessible	App not installed on the repo	Install the app on the target repo (Step 3)
Failed to read private key	Wrong path or permissions	Verify the path in config; check chmod 600
Comments show wrong author	bot_username mismatch	Match the exact <slug>[bot] format (Step 5)

Dual App Setup (Autonomous Merging)

Colony can split its GitHub identity across two Apps: App A (colony-coder) authors code, and App B (colony-ops) reviews and approves it. This mirrors how healthy human teams operate on GitHub — the person who writes the code cannot approve their own PR.

With a single App, GitHub rejects the Reviewer’s APPROVED review because the Reviewer and Developer share the same identity (“cannot approve your own pull request”). The Reviewer falls back to posting a plain comment, and Sprint Master’s auto-merge never fires. A human must always press Approve.

With two Apps, the Reviewer posts a formal APPROVED review via App B’s identity, and Sprint Master’s checkAutoMerge can merge the PR without waiting for a human — closing the autonomous loop.

This setup is optional. Single-App deployments continue to work with no changes. Only configure a second App if you want auto_merge_on_approval: true to work without a human approving each PR.

App A — colony-coder

This is the App created in Steps 1–5 above. If you followed the single-App guide, you already have App A.

Required permissions for App A:

Permission	Access	Purpose
Contents	Read & write	Push branches, read repo files, create worktrees
Issues	Read & write	Manage issue labels, post comments
Pull requests	Read & write	Open PRs, post analyzer/developer comments
Metadata	Read-only	Required (automatically granted)

Used by: Analyzer, Developer

App B — colony-ops

Create a second GitHub App following the same steps (Steps 1–5), but with a different name (e.g., colony-ops). This App must be a different GitHub identity than App A — GitHub checks the identity of the reviewer, not just the token.

Required permissions for App B:

Permission	Access	Purpose
Contents	Read-only	Read repo files (no push — App B never writes code)
Issues	Read & write	Transition labels, post sprint-master/reviewer/merger comments
Pull requests	Read & write	Post formal PR reviews, merge PRs
Metadata	Read-only	Required (automatically granted)

Used by: Sprint Master, Planner, Reviewer, Merger (API calls)

Merger edge case: Merger performs git rebase and force-push (requiring contents: write) using App A’s git credentials, but all GitHub API calls (post comment, transition labels, merge PR) use App B’s identity. The git worktree is configured with App A credentials. No changes to Merger’s git setup are needed.

Step B1: Create App B

Follow the same Steps 1–5 from this guide for the second App, using the permissions table above. Note:

• A different App name (e.g., colony-ops) — must be globally unique
• Install on the same repositories as App A
• Save the PEM file at a different path (e.g., ~/.colony/keys/github-ops-app.pem)
• Note the App B app_id, installation_id, and bot_username

Step B2: Update Colony Configuration

Add an ops_app block alongside the existing app block:

github:
  owner: your-org
  repo: your-repo
  app:
    app_id: 111111                                   # App A (colony-coder)
    private_key_path: ~/.colony/keys/github-app.pem
    installation_id: 11111111
  ops_app:
    app_id: 222222                                   # App B (colony-ops)
    private_key_path: ~/.colony/keys/github-ops-app.pem
    installation_id: 22222222
  bot_username: 'colony-coder[bot]'                  # App A's bot username

review:
  auto_merge_on_approval: true

Alternatively, use a PAT for App B instead of a full GitHub App:

github:
  ops_token_env: COLONY_OPS_TOKEN   # env var holding App B's installation token

Per-repo override

For multi-repo deployments, you can set ops_app per repo in the repos array:

repos:
  - owner: your-org
    repo: your-repo
    app:
      app_id: 111111
      private_key_path: ~/.colony/keys/github-app.pem
      installation_id: 11111111
    ops_app:
      app_id: 222222
      private_key_path: ~/.colony/keys/github-ops-app.pem
      installation_id: 22222222

Git Push Credentials

Git operations (clone, fetch, push) use system-level authentication — SSH keys or a credential helper. These should be configured with App A’s identity since only App A has contents: write. App B cannot push code.

If you use SSH for git operations, the SSH key should correspond to App A. App B credentials are only used for GitHub API calls (reviews, comments, label transitions, merges).

Backward Compatibility

If only app is configured (no ops_app), Colony behaves exactly as before:

• coderGithub and opsGithub both point to the same App A client
• The Reviewer will fall back to posting a plain comment when GitHub rejects the self-review
• auto_merge_on_approval still works for human-approved PRs

No config changes are needed for existing single-App deployments.

Verification

After configuring both Apps:

1. Enable auto_merge_on_approval: true in your config

2. Run npx colony doctor — it should not warn about missing ops identity

3. Create a test issue and let it flow through the pipeline

4. When the Reviewer approves, check the Reviewer logs for:

   Posted formal PR review via ops identity   repo=your-org/your-repo issue=N event=APPROVE

   If you see Cannot APPROVE own PR, posting comment instead instead, the Reviewer is still using a single identity — verify the ops_app config and that App B is a different GitHub account than App A.

5. If auto_merge_on_approval: true, the Sprint Master should merge the PR automatically after the formal approval without waiting for a human.

Troubleshooting Dual App Setup

Symptom	Cause	Fix
Cannot APPROVE own PR in Reviewer logs	ops_app is not configured, or App B is the same identity as App A	Add ops_app config; ensure App B is a separate GitHub App
npx colony doctor warns about ops_app	auto_merge_on_approval: true but no ops_app configured	Add ops_app config or disable auto_merge_on_approval
Sprint Master does not auto-merge after Reviewer approves	App B’s formal review was not posted	Check Reviewer logs for “Posted formal PR review via ops identity”
Failed to read private key for ops_app	Wrong path in ops_app.private_key_path	Verify path; check chmod 600 on the PEM file
App B installation not found	App B not installed on the target repo	Install App B on the repo (same step as App A installation)

---

Multi-Repo Setup

For managing multiple repositories with the same App, use the repos array in config:

github:
  owner: your-org
  repo: primary-repo
  app:
    app_id: 123456
    private_key_path: ~/.colony/keys/github-app.pem
    installation_id: 78901234
  bot_username: "colony-bot[bot]"

repos:
  - owner: your-org
    repo: primary-repo
    app:
      app_id: 123456
      private_key_path: ~/.colony/keys/github-app.pem
      installation_id: 78901234
    bot_username: "colony-bot[bot]"
    workspace:
      repo_dir: /path/to/primary-repo
      base_dir: ~/.colony/workspaces/{owner}/{repo}

  - owner: your-org
    repo: secondary-repo
    app:
      app_id: 123456
      private_key_path: ~/.colony/keys/github-app.pem
      installation_id: 78901234  # Same installation if both repos are in one install
    bot_username: "colony-bot[bot]"
    workspace:
      repo_dir: /path/to/secondary-repo
      base_dir: ~/.colony/workspaces/{owner}/{repo}