First Issue Tutorial

First Issue Tutorial

Sections 1–3 below (creating an issue, adding the label, watching the pipeline) are also covered in the Getting Started Guide. This tutorial adds deeper coverage: reading the analyzer plan, reviewing the PR, slash commands, and common gotchas.

Walk through Colony processing a GitHub issue end-to-end — from creating the issue to merging the PR.

Prerequisite: A running Colony instance. If you haven’t set one up yet, follow the Getting Started Guide.

---

1. Creating a Well-Scoped First Issue

Colony works best on issues that are small, self-contained, and clearly described. For your first issue, pick something that:

• Has a single responsibility. One feature, one bug fix, or one refactor — not a combination.
• Has clear acceptance criteria. Describe what “done” looks like so the analyzer can write a concrete plan.
• Doesn’t require secrets or external services. Colony’s developer agent runs in an isolated worktree. It can read and write code, run build/test/lint commands, and interact with the local filesystem — but it cannot access external APIs, databases, or secret stores during implementation.
• Touches a small number of files. Fewer than 5–10 files is ideal for a first run. Large cross-cutting changes are better handled after you’re comfortable with the pipeline.

Size guidance: A good first issue changes 1–3 files and under 200 lines total. Larger changes are more likely to hit the turn limit or require re-implementation cycles before reaching human-review-ready.

Example issue

Title: Add a greeting utility function

Body:

Add a `greet(name: string): string` function to `src/utils.ts` that returns
`Hello, {name}!`.

Acceptance criteria:
- Function is exported from `src/utils.ts`
- Returns the greeting string with the provided name
- Unit test in `src/__tests__/utils.test.ts` covers the happy path

What to avoid for a first issue

• Vague descriptions (“improve the API” or “refactor the codebase”)
• Issues requiring environment variables or API keys at runtime
• Issues that depend on unreleased changes from other branches
• Large epics — save those for after you understand the pipeline

---

2. Adding the colony:enqueue Label

How Colony discovers your issue depends on the intake_mode setting in your config.

intake_mode: tagged (default)

Only issues explicitly labeled colony:enqueue enter the pipeline. This is the recommended mode for evaluation — it gives you full control over which issues Colony processes.

To add the label in the GitHub UI:

1. Open your issue on GitHub.
2. In the right sidebar, click Labels.
3. Search for colony:enqueue and select it.
4. The label is applied immediately.

Note: The colony:enqueue label is created automatically when you run npx colony init on your repo. If you don’t see it, re-run npx colony init -r owner/repo.

intake_mode: all

Every new issue is automatically picked up by Colony — no label needed. Use this mode only after you trust the pipeline and want Colony to process all incoming issues.

You can set the intake mode in your config:

agents:
  sprint_master:
    intake_mode: tagged # or "all"

After labeling, wait 30-60 seconds for Sprint Master to pick up the issue. Sprint Master polls on a configurable interval (default 30s), so your issue may take up to two poll cycles to appear. Run colony status to verify agents are running. If nothing happens after 2 minutes, check .colony/logs/sprint-master.log for errors.

---

3. Watching the Pipeline

After you add the colony:enqueue label (or create the issue with intake_mode: all), Colony processes it through a series of states. Each state is visible as a colony:<state> label on the issue.

The happy-path progression for a typical first issue:

colony:enqueue → colony:new → colony:analyzing → colony:ready-for-dev → colony:in-development → colony:in-review → colony:human-review-ready

Here’s what happens at each transition.

colony:enqueue → colony:new

What happens: The Sprint Master picks up the issue on its next poll cycle (default: every 30 seconds). It creates a pipeline_issues row in Postgres and transitions the issue to new.

Where to look	What you’ll see
GitHub	The colony:enqueue label is replaced by colony:new
Dashboard (http://localhost:9106)	The issue appears in the pipeline view with state new
Logs	Sprint Master logs: intake: picked up issue #N

colony:new → colony:analyzing

What happens: The Sprint Master enqueues an analyze task in the work_tasks queue. A worker claims the task and runs the analyzer, which reads the issue body, explores the codebase, and writes an implementation plan.

Where to look	What you’ll see
GitHub	Label changes to colony:analyzing
Dashboard	Issue state updates to analyzing; worker activity shows the claimed task
Logs	Worker logs: claimed task analyze for issue #N

colony:analyzing → colony:ready-for-dev

What happens: The analyzer finishes and posts a comment on the issue with its implementation plan. The issue transitions to ready-for-dev, which enqueues a develop task.

Where to look	What you’ll see
GitHub	Label changes to colony:ready-for-dev. A new comment from Colony appears with the analysis plan (see Section 4)
Dashboard	Issue state updates to ready-for-dev
Logs	Analyzer logs: analysis complete for issue #N

colony:ready-for-dev → colony:in-development

What happens: A worker claims the develop task. The developer agent creates a git worktree branched from origin/main, implements the changes using Claude Code, and opens a pull request.

Where to look	What you’ll see
GitHub	Label changes to colony:in-development. A new branch colony/issue-N appears. Eventually, a PR is opened
Dashboard	Issue state updates to in-development; worker activity shows active development
Logs	Developer logs: creating worktree for issue #N, then Claude Code session output

Timing: Development is the longest phase. A simple issue typically takes 2–5 minutes; more complex issues can take 10–30 minutes depending on the Claude model and issue complexity.

colony:in-development → colony:in-review

What happens: The developer opens a PR and transitions the issue to in-review, which enqueues a review task. A worker claims it and the reviewer runs deterministic checks (build, test, lint) followed by an LLM-powered code review.

Where to look	What you’ll see
GitHub	Label changes to colony:in-review. Review comments appear on the PR
Dashboard	Issue state updates to in-review
Logs	Reviewer logs: running checks for PR #N, then review verdict

colony:in-review → colony:human-review-ready

What happens: All automated checks passed and the reviewer’s LLM review is complete. The issue is now waiting for your approval.

Where to look	What you’ll see
GitHub	Label changes to colony:human-review-ready. The PR has Colony’s review comments and is ready for your review
Dashboard	Issue state updates to human-review-ready
Logs	Reviewer logs: review complete for PR #N, verdict: approve

Review cycle: If the reviewer finds issues, the PR goes through a colony:changes-requested → colony:in-development → colony:in-review cycle before reaching human-review-ready. This is normal — when colony:changes-requested is applied (by the automated reviewer or by you manually), the developer agent is re-enqueued automatically. It reads all review comments on the PR before re-implementing, so each cycle incorporates the previous feedback. Multiple cycles are expected for more complex issues.

Checking progress

CLI:

# Docker:
docker exec colony-sprint-master colony status

# Native:
npx colony status

Logs:

# Docker:
docker compose logs -f worker

# Native:
npx colony logs

Native log files: In native mode, each agent also writes to .colony/logs/<agent>.log in the Colony directory (e.g. .colony/logs/sprint-master.log, .colony/logs/worker.log). Use npx colony logs [agent] to tail them in the terminal, or open the files directly.

---

4. Reading the Analyzer Plan Comment

When the analyzer finishes, it posts a comment on the issue with its implementation plan. This comment typically contains:

• Complexity assessment — small, medium, or large. This determines the Claude model and turn budget used during development.
• Affected files — which files the analyzer expects to be created or modified.
• Implementation approach — a step-by-step plan describing how the issue will be implemented.
• Test strategy — how the implementation will be verified.

Requesting changes before development starts

If the analyzer’s plan doesn’t match your expectations, you can intervene before development begins:

1. Edit the issue body to clarify requirements or add constraints. The developer reads the issue body during implementation and will incorporate your changes.
2. Comment on the issue with specific guidance (e.g., “Use the existing formatName utility instead of creating a new one”).
3. Remove the colony:ready-for-dev label and add colony:analyzing to re-trigger analysis with updated context.

Acting before development starts saves time — once the developer begins, it’s working from the analyzer’s plan plus the issue body.

---

5. Reviewing the PR Colony Opens

Branch naming

Colony creates branches named colony/issue-N where N is the GitHub issue number. For example, issue #42 gets branch colony/issue-42.

Commit structure

The developer agent typically creates one or more commits on the branch. Commits reflect the implementation steps — you may see separate commits for the main implementation, tests, and formatting fixes.

What to look for

• Does the code match the acceptance criteria? Compare the diff against what you specified in the issue body.
• Are the tests meaningful? Colony writes tests based on the acceptance criteria. Check that they test actual behavior, not just that the code compiles.
• Are there any unnecessary changes? The developer should only modify files relevant to the issue. Flag any unrelated changes.

Requesting changes

If the PR needs work, you have two options:

Option A: Add the colony:changes-requested label

Remove the current state label and add colony:changes-requested. This sends the issue back to the developer for another implementation pass. The developer reads the review comments and iterates.

Option B: Comment /colony:retry on the issue

This retries the current pipeline stage. If the issue is in human-review-ready, it restarts development from the current state.

Option C: Comment /colony:reimplement on the issue

This closes the current PR and starts a fresh implementation from scratch. Use this when the current approach is fundamentally wrong and iterating on it would be more expensive than starting over.

---

6. Merging or Requesting Changes

Manual merge (default)

With the default auto_merge_on_approval: false, Colony stops at human-review-ready and waits for you to:

1. Review the PR on GitHub.
2. Approve and merge it using GitHub’s merge button.
3. Colony detects the merge and transitions the issue to done, closing it automatically.

Automatic merge

With auto_merge_on_approval: true in your config, Colony’s merger agent handles merging after the reviewer approves. The flow becomes:

colony:in-review → colony:merge-pending → colony:done

The merger checks that the PR is approved, the head branch is up to date with the base branch, and CI checks pass before merging. If the head branch is out of date, it attempts a rebase.

Note: Automatic merge requires a dual GitHub App setup (a separate ops app that can approve PRs). See the GitHub App Setup Guide for details.

Slash commands

You can control issue processing by commenting slash commands on the issue. Slash commands work as GitHub issue comments and can be posted at any time during processing — Colony’s Sprint Master checks for new comments on each poll cycle (every 30 seconds by default). These commands are recognized:

Command	Description
/colony:retry	Retry the current pipeline stage with current settings
/colony:retry large	Retry with a large complexity budget (more turns, higher model tier)
/colony:skip-planning	Skip the planning phase and implement directly
/colony:decompose	Send the issue to the planner for decomposition into sub-issues
/colony:cancel	Cancel processing and close the issue
/colony:reimplement	Close the current PR and start a fresh implementation

---

7. Common First-Issue Gotchas

Worker clone failure

Symptom: Issue transitions to colony:in-development but no PR appears. Worker logs show git errors.

Cause: Worker containers clone the target repo at startup. If the clone fails (network issue, auth problem, disk space), no worktree can be created.

Fix: Check worker logs for the specific error:

# Docker:
docker compose logs worker | grep -i "clone\|fatal\|error"

# Native:
npx colony logs worker

Common causes: incorrect repo_dir path (native mode), insufficient disk space, or the GitHub token/App lacking Contents read-write permission.

Missing GitHub permissions

Symptom: Colony can read issues but can’t add labels, post comments, or open PRs. Logs show 403 or 404 errors on write operations.

Cause: The GitHub token or App is missing required permissions.

Fix: Ensure your token or GitHub App has these permissions:

Permission	Access	Purpose
Contents	Read & write	Push branches, read repo files
Issues	Read & write	Manage labels, post comments
Pull requests	Read & write	Open PRs, post review comments

For PATs, verify the repo scope is enabled. For GitHub Apps, check the app’s permission settings and ensure it’s installed on the target repo.

Webhook not connected

Symptom: State transitions are delayed by up to 30 seconds after an event occurs.

Cause: Without webhooks, Colony relies on polling. The Sprint Master polls every 30 seconds by default.

Impact: This is expected behavior — webhooks are optional. Polling works for all operations. If you need faster response times, set up webhooks following the Setup Checklist.

review.checks configured for nonexistent scripts

Symptom: The developer adds phantom scripts to package.json to pass self-validation, then the reviewer flags them as unrelated changes. The issue loops between in-development and changes-requested.

Cause: The review.checks config references build/test/lint commands that don’t exist in the target repo (e.g., npm test when there’s no test script in package.json).

Fix: Only configure checks that already exist in your target repo:

review:
  checks:
    build: 'npm run build' # only if package.json has a "build" script
    test: 'npm test' # only if package.json has a "test" script
    lint: 'npm run lint' # only if package.json has a "lint" script

Remove any check whose command would fail if you ran it manually in the repo.

Issue too large or ambiguous

Symptom: The issue gets stuck in colony:in-development for a long time, or the developer blocks after exceeding the turn limit. Colony posts a comment with available retry commands.

Cause: The issue scope is too broad for a single development pass, or the requirements are ambiguous enough that the developer can’t determine what to build.

Fix:

• Too large: Comment /colony:decompose to send the issue to the planner, which will break it into smaller sub-issues.
• Too ambiguous: Edit the issue body to add specific acceptance criteria, then comment /colony:retry to restart.
• Stuck in a loop: Comment /colony:cancel to stop processing, revise the issue, and create a fresh issue with clearer scope.

For your first issue, start small — a single function, a simple bug fix, or a minor config change. You can tackle larger issues once you’re familiar with how Colony operates.