Docker Deployment Guide
Docker Deployment Guide
Section titled “Docker Deployment Guide”Run the full Colony pipeline with docker compose up. The topology consists of singleton services (sprint-master, monitor) plus per-repo worker containers that process all task types (analyze, develop, review, merge, plan). The webhook-receiver and central MCP services are opt-in Compose profiles (see Webhook Setup and Central MCP Server).
Prerequisites
Section titled “Prerequisites”- Docker Engine 20+ and Docker Compose v2
- A GitHub token (PAT or GitHub App)
- An Anthropic API key
- A
colony.config.yamltailored for your repo(s)
Quick Start
Section titled “Quick Start”# 0. Set deployment mode in colony.config.yaml (if not already set by colony quickstart):# deployment:# mode: docker-compose
# 1. Copy and fill in environment variablescp .env.example .env# Edit .env — set GITHUB_TOKEN and ANTHROPIC_API_KEY (or gateway auth token — see configuration.md#claudegatewayconfig)# Leave DATABASE_URL unset — Docker Compose provides it via the bundled Postgres
# 2. Prepare your colony.config.yaml# Workers clone repos at startup — workspace.repo_dir is overridden at runtime.# Only github.app.private_key_path needs to use the Docker volume path:# github.app.private_key_path → /colony/keys/github-app.pem
# 3. Build the imagedocker compose build
# 4. Start all agentsdocker compose up -d
# 5. View logsdocker compose logs -fEnvironment Variables
Section titled “Environment Variables”| Variable | Required | Description |
|---|---|---|
GITHUB_TOKEN | Yes* | GitHub personal access token. Not needed if using App auth. |
ANTHROPIC_API_KEY | Yes* | Anthropic API key for Claude Code CLI. Not required when using provider: gateway — supply the gateway auth token instead (see ClaudeGatewayConfig). |
DATABASE_URL | No* | Postgres connection string. When deployment.mode is docker-compose, docker-compose.yml provides this automatically via the bundled Postgres — leave it unset in .env. Only set manually when using an external database (native or apple-container mode). |
COLONY_CONFIG | No | Base64-encoded colony.config.yaml. If set, the entrypoint writes it to /colony/colony.config.yaml. Useful for cloud deployments where volume mounts are inconvenient. |
COLONY_AGENT | No | Which agent to run. Set per service in docker-compose.yml. Values: sprint-master, monitor, mcp, webhook-receiver, cli, <name>-worker-<N> (e.g., colony-worker-1). |
COLONY_REPO | No | Pins a worker container to a single repository (owner/repo). In single-repo container deployments, the worker auto-derives this from the config when the variable is unset — no manual configuration needed. In multi-repo deployments, set this per worker service — if omitted, the worker starts but logs a warning and claims no tasks, causing a silent task queue stall. |
COLONY_MULTI_REPO | No | Set to true to convert the multi-repo startup error in resolveConfig() into a warning. Self-hosted multi-repo orchestration is not supported — this escape hatch exists for Colony Cloud deployments. See runcolony.com. |
WEBHOOK_SECRET | No | GitHub webhook secret for signature validation. Required when running the webhook-receiver service. |
MCP_AUTH_TOKEN | No | Bearer token required by the central MCP HTTP server. Required when running the mcp Compose profile. |
MCP_PORT | No | Host/container port for the central MCP HTTP server. Defaults to 9300. |
MCP_PATH | No | MCP HTTP endpoint path. Defaults to /mcp. |
NODE_OPTIONS | No | Node.js runtime flags (e.g., --max-old-space-size=4096). |
Volume Layout
Section titled “Volume Layout”/colony/colony.config.yaml — config file (mount read-only)/colony/keys/github-app.pem — GitHub App A PEM (mount read-only, if using App auth)/colony/keys/github-ops-app.pem — GitHub App B PEM (mount read-only, if using dual App setup)/colony/workspaces/ — worktree base_dir (read-write, named volume mounted into every agent container via the shared `x-colony-common` anchor; only sprint-master and worker actually clone/write into it)/var/lib/colony/repos/{owner}/{repo}/ — clone path for workers and sprint-master (created automatically at startup; override the root with `COLONY_REPOS_DIR`)Target Repository
Section titled “Target Repository”Workers clone their target repos automatically at startup to /var/lib/colony/repos/{owner}/{repo} (override the root with COLONY_REPOS_DIR). Sprint-master does the same in its own container, cloning each configured repo to the same path at startup — its post-merge cleanup (stale-branch detection, rebasing active branches) needs full git history. No host-mounted repo directory is needed for either. The workspace.repo_dir and workspace.base_dir values in your config are overridden at runtime by the clone-setup process.
Clones live outside /colony on purpose: the image has /colony/package.json, so a clone under it with no package.json of its own would make npm run build run Colony’s scripts.
Upgrading: the old /colony/repos path was not a named volume, so recreated containers simply re-clone at the new path. Old clones are not migrated or deleted. If you mount a persistent volume for clones, either re-mount it at /var/lib/colony/repos or set COLONY_REPOS_DIR to a mount point outside the Colony install directory. Never point COLONY_REPOS_DIR at a path under /colony: npm would walk up to Colony’s package.json and undo the isolation.
Monitor and webhook-receiver do not clone the repository at all — they communicate exclusively via the GitHub API and Postgres. No service in this topology requires a repo bind mount.
# colony.config.yaml — workspace.repo_dir is ignored by workers in container moderepos: - owner: my-org repo: my-app workers: pool_size: 2 # Safe — each worker container gets its own isolated clone health_port_start: 9200 workspace: repo_dir: ~/git/my-app # Used by native mode only; workers override this base_dir: ~/.colony/workspaces # Used by native mode only; workers override thisWorkers clone via HTTPS using the system-wide git credential helper configured in the Dockerfile. After cloning, workers run mise install (if available) and the repo’s workspace.setup_command. If unset, the setup command is auto-detected from the repo’s manifest files (e.g. Gemfile → bundle install, go.mod → go mod download, requirements.txt/pyproject.toml → pip, Cargo.toml → cargo build; a bare package.json falls back to the detected JS package manager — npm, yarn, pnpm, or bun — not always npm install).
Credential helper token cache: The helper caches the GitHub App installation token per installation (keyed by app_id + installation_id) in $XDG_RUNTIME_DIR or the container’s temp directory, under colony-git-credential-<uid>/ (directory 0700, files 0600, never under /colony). A cached token is reused until it is within 10 minutes of expiry, when a new one is minted. When git reports a rejected credential (erase), the entry is deleted. The cache is not persistent: it resets when the container restarts.
Container restart behavior: Restarting a worker container produces a fresh clone with no stale worktree artifacts. This means pool_size > 1 works safely — each container gets its own .git/ state with no shared-git concurrency issues.
Disk usage: Each worker container needs enough ephemeral storage for a full repo clone plus worktrees created during task execution.
GitHub App Auth
Section titled “GitHub App Auth”If using GitHub App authentication, mount the PEM file and update the config path:
github: app: app_id: 123456 private_key_path: /colony/keys/github-app.pem installation_id: 78901234# docker-compose.yml — uncomment the keys volumevolumes: - ~/.colony/keys/github-app.pem:/colony/keys/github-app.pem:roHost → container path mapping: Place the PEM file at
~/.colony/keys/github-app.pemon the host (matching the native install convention). The volume mount maps it to/colony/keys/github-app.peminside the container, which is the path to set inprivate_key_pathfor Docker deployments.
Dual App Setup (Autonomous Merging)
Section titled “Dual App Setup (Autonomous Merging)”If using a second GitHub App (colony-ops) for autonomous merging, mount a second PEM file and add the ops_app block to your config:
github: app: app_id: 111111 private_key_path: /colony/keys/github-app.pem installation_id: 11111111 ops_app: app_id: 222222 private_key_path: /colony/keys/github-ops-app.pem installation_id: 22222222
review: auto_merge_on_approval: trueservices: sprint-master: volumes: - ~/.colony/keys/github-app.pem:/colony/keys/github-app.pem:ro - ~/.colony/keys/github-ops-app.pem:/colony/keys/github-ops-app.pem:ro worker: volumes: # No repo mount needed — workers clone repos at startup - ~/.colony/keys/github-app.pem:/colony/keys/github-app.pem:ro - ~/.colony/keys/github-ops-app.pem:/colony/keys/github-ops-app.pem:ro webhook-receiver: volumes: - ~/.colony/keys/github-app.pem:/colony/keys/github-app.pem:ro monitor: volumes: - ~/.colony/keys/github-app.pem:/colony/keys/github-app.pem:roPlace App B’s PEM at ~/.colony/keys/github-ops-app.pem on the host:
mv ~/Downloads/<ops-app-name>.*.private-key.pem ~/.colony/keys/github-ops-app.pemchmod 600 ~/.colony/keys/github-ops-app.pemSee docs/github-app-setup.md for the full dual App walkthrough.
Health Checks
Section titled “Health Checks”Each service exposes an HTTP health endpoint. Default ports:
| Service | Port | Notes |
|---|---|---|
| sprint-master | 9100 | Singleton |
| monitor | 9106 | Singleton (also serves dashboard) |
| webhook-receiver | 9800 | Singleton |
| worker | 9200+ | Per-repo; sequential from repos[].workers.health_port_start |
The docker-compose.yml health checks use these defaults. If you override health ports in your config YAML, update the health check URLs and port mappings in docker-compose.yml to match.
Port 9106 (monitor/dashboard) has no authentication by default. Anyone who can reach this port can retry, cancel, or transition pipeline issues. In Docker Compose, bind it to localhost with
127.0.0.1:9106:9106rather than9106:9106if the host is shared or internet-facing. To enable HTTP Basic Auth, see the dashboard documentation foragents.monitor.auth.password_env.
# Check health manuallycurl http://localhost:9100/healthWebhook Setup
Section titled “Webhook Setup”The webhook-receiver service provides instant GitHub event dispatch to agents, eliminating up to 30s polling delays.
1. Generate a webhook secret
Section titled “1. Generate a webhook secret”openssl rand -hex 32Add the result to your .env file:
WEBHOOK_SECRET=<generated-secret>2. Configure the webhook in your colony config
Section titled “2. Configure the webhook in your colony config”webhook: enabled: true secret_env: WEBHOOK_SECRET port: 98003. Register the webhook on GitHub
Section titled “3. Register the webhook on GitHub”In your GitHub repository (or organization) settings, add a webhook:
- Payload URL:
http://<your-host>:9800/webhook - Content type:
application/json - Secret: the same value you set in
WEBHOOK_SECRET - Events: select Let me select individual events and enable:
IssuesIssue comments(required for slash commands)Pull requestsPull request reviewsPull request review commentsCheck runsCheck suitesStatusesWorkflow runs
4. Start the service
Section titled “4. Start the service”The webhook-receiver runs behind an opt-in Compose profile and is not started by a plain docker compose up -d. Start it with:
docker compose --profile webhook up -d(Also ensure WEBHOOK_SECRET is set in your .env.) Then verify it is healthy:
curl http://localhost:9800/healthDocker networking note
Section titled “Docker networking note”In Docker, services communicate using Docker service names, not localhost. If you set explicit dispatch URLs in your config, use service names (e.g., http://sprint-master:9100/wake, http://worker:9200/health). When using the defaults derived from health_port values, services dispatch to the host network port, which works when all services share the default bridge network.
Without webhooks
Section titled “Without webhooks”All agents continue to poll GitHub on their configured interval. Webhooks are additive — polling remains the fallback for environments where inbound HTTP is not feasible (local development, firewalled deployments), except that slash commands have no polling fallback.
Central MCP Server
Section titled “Central MCP Server”Run one shared MCP HTTP server on the Colony VM for pilot users:
docker compose --profile mcp up -d mcpSet MCP_AUTH_TOKEN in .env before starting the service. Users connect MCP clients that support Streamable HTTP to:
http://<vm-host>:9300/mcpand send:
Authorization: Bearer <MCP_AUTH_TOKEN>The MCP endpoint can create issues and control pipeline state, so expose it only on a trusted network, VPN, or behind a TLS reverse proxy.
Host CLI vs. container agents
Section titled “Host CLI vs. container agents”In Docker Compose mode, Colony splits across two execution contexts:
- The
colonyCLI runs on the host (your machine, outside containers). Commands likecolony init,colony check,colony quickstart, andcolony config migraterun as host processes and readcolony.config.yamlfrom the current directory. - Agents run inside containers (sprint-master, worker, monitor, webhook-receiver). They read credentials from
.envat container startup.
Which commands run where
Section titled “Which commands run where”| Command | Where it runs | Notes |
|---|---|---|
colony init | Host | Creates GitHub labels and branch protection. Reads GITHUB_TOKEN from .env beside the config (or the shell). |
colony check | Host | Validates config, credentials, and connectivity. Pass --docker for container-aware checks. |
colony quickstart | Host | Interactive setup wizard. |
colony config migrate | Host | Converts old config format. |
colony start | Redirected | Shows: docker compose up -d |
colony stop | Redirected | Shows: docker compose down |
colony status | Redirected (no host DATABASE_URL) | Prints the redirect hint by default; with a reachable DATABASE_URL or --database-url it shows pipeline state directly. See below. |
colony logs | Redirected | See logs below. |
Checking pipeline state (where your issue is)
Section titled “Checking pipeline state (where your issue is)”By default (no DATABASE_URL on the host, as recommended above), colony status on the host only prints a redirect hint — it does not show where your issue is in the pipeline. If a database URL resolves (via --database-url, DATABASE_URL, or database.url), it queries Postgres and shows pipeline state directly. The recommended route is to exec into the sprint-master container:
docker exec colony-sprint-master colony statusTo check container health separately:
docker compose psViewing agent logs
Section titled “Viewing agent logs”To follow worker output (the agent processing your issue):
docker compose logs -f workerTo follow all services:
docker compose logs -fCredential split
Section titled “Credential split”Containers receive .env via Docker Compose. Host-run CLI commands (colony init, colony check) read the same file automatically: first the .env beside colony.config.yaml, then the one in the current directory. Variables already exported in your shell take precedence, and unedited placeholder values from .env.example (such as ghp_your_token_here) are ignored.
If a command reports GITHUB_TOKEN in <path>/.env is still the placeholder value from .env.example, replace it with a real token in the named file (or export GITHUB_TOKEN in your shell).
Using the CLI
Section titled “Using the CLI”Run ad-hoc CLI commands against the running deployment:
# colony statusdocker compose run --rm -e COLONY_AGENT=cli sprint-master status
# Or use docker exec on a running containerdocker exec colony-sprint-master colony status
# Migrate old config to worker pool formatdocker exec colony-sprint-master colony config migrate --config /colony/colony.config.yamlBuilding
Section titled “Building”The Dockerfile has three stages: build (compiles TypeScript), test (FROM build, runs npm test), and production (the default final stage, built from build’s output — never from test).
Default build (no tests)
Section titled “Default build (no tests)”docker build .An untargeted build resolves to the last stage, production, which does not depend on test — tests are not run. This is also what CI uses (docker build --target production .) and what docker compose build produces.
docker build --target build .Stops one stage earlier, at build. Also skips tests, and additionally skips the production-stage system/runtime setup (mise, language runtimes, non-root user, etc.) — mainly useful for debugging the compile step in isolation.
Running tests during the build
Section titled “Running tests during the build”docker build --target test .The only way to run npm test as part of a docker build. Not used by CI (which runs tests on the host and builds --target production directly); available for operators who want an in-image test gate.
Rebuilding after code changes
Section titled “Rebuilding after code changes”docker compose build --no-cachedocker compose up -dViewing Logs
Section titled “Viewing Logs”# All servicesdocker compose logs -f
# Single servicedocker compose logs -f worker
# Last 100 linesdocker compose logs --tail=100 sprint-masterStopping
Section titled “Stopping”# Stop all agentsdocker compose down
# Stop and remove volumes (deletes worktrees)docker compose down -vTroubleshooting
Section titled “Troubleshooting”Agent won’t start — “No config found”
Mount your colony.config.yaml at /colony/colony.config.yaml or set the COLONY_CONFIG env var.
Developer agent fails to create worktrees
Workers clone repos automatically at startup. If worktree creation fails, check network connectivity and GitHub App auth (the clone may have failed). For native deployments, ensure the target repo is cloned and accessible at the path in workspace.repo_dir.
Health check failing
Verify the health port in your config matches the port in the docker-compose.yml health check. Default ports: sprint-master 9100, worker 9200+, monitor 9106, webhook-receiver 9800.
Claude Code CLI not found
The production image installs @anthropic-ai/claude-code globally. If builds fail at that step, check npm registry access from your build environment.
Permission denied on /colony/workspaces
The named volume (and the image’s default /colony/workspaces directory) is owned by the non-root colony user (UID 501 by default), and the agent process always runs as that same user, never root. If you bind-mount a host directory instead of the named volume, it must be writable by the colony user’s UID — pass --build-arg COLONY_UID=$(id -u) at build time to match your host user.