Roll Out Self-Improvement Tracks
Self-improvement tracks periodically seed an issue that asks Colony to inspect a repository and propose a slate of useful improvements. By default, a person selects a candidate before development begins. Tracks use the same pipeline and Worker capacity as ordinary issues.
Start with one narrow track
Section titled “Start with one narrow track”Choose a repository where one additional open issue will be easy to observe. Use one track, a seven-day cooldown, and a prompt focused on a single concern such as missing tests or stale documentation.
Important field semantics:
- Name is the stable track key.
- Label is the actual code-host label applied to seeded issues, not a display title.
- Cooldown runs from the track’s latest seed or latest completion, whichever is later. A track can hold only as many open linked issues as its slots allow.
- Cron schedules use UTC.
- Weight sets the track’s share of its pool, which decides its slots and how often it is picked. It does not move its work ahead in the shared task queue.
Create the disabled track
Section titled “Create the disabled track”Open Autonomy → Tracks (/autonomy/tracks), filter to the repository, and add a track with:
- name
test-coverage; - label
colony:si-test-coverage; - cooldown Weekly (10,080 minutes);
- enabled off.
People with the Autonomy grant can create and edit tracks. The repository-specific view uses the same underlying track records.
First, enable the SI scheduler in the Colony installation’s colony.config.yaml:
self_improvement: enabled: true cooldown_minutes: 10080Apply the configuration, then open the monitor dashboard at http://localhost:9106 (or the configured monitor port). Sign in if agents.monitor.auth is configured and select the Tracks tab.
- Select the target repository when the dashboard manages more than one.
- Click Add Track.
- Enter
test-coveragefor Name andcolony:si-test-coveragefor Label. - Set Cooldown to
10080minutes. - Clear Enabled so the track cannot seed before its prompt is ready.
- Click Create Track.
The dashboard writes the track to Colony’s Postgres-backed track store. See Monitor Dashboard Reference for dashboard access and authentication.
Add a prompt version
Section titled “Add a prompt version”A prompt version has three operational fields:
seed_title: the title of the SI seed issue;seed_body: stable context placed in that issue;instructions: how the analyzer should search the repository and construct candidates.
The seed issue exists before candidate analysis, so prompt instructions cannot prevent the scheduler from creating it. Instead, make the instructions demand evidence, bounded candidates, and honest reporting when no worthwhile candidate is found.
Use Manage Prompts on the track, add a prompt version, and confirm that it is active.
In the monitor dashboard’s Tracks tab:
- Click the
test-coveragetrack row to expand Prompt Versions. - Click Add Version.
- Enter a seed title such as
Propose a focused test-coverage improvement. - Use the seed body for stable issue context and expected candidate qualities.
- In Instructions, ask Colony to find untested behavior with user-visible risk, cite file and test evidence, and prefer work that fits one pull request.
- Add a short version note and click Add Version.
Adding a Postgres-backed prompt version makes it the track’s active version. The expanded row displays the active content and version history; click Activate on an older version to roll back.
Enable and evaluate the first cycle
Section titled “Enable and evaluate the first cycle”In Cloud, enable the track’s toggle. In a self-hosted deployment, return to the monitor dashboard’s Tracks table and click the track’s Off button so it changes to On.
A newly eligible track can seed on the next scheduler loop. Colony then analyzes repository signals and presents a candidate slate for human selection. Review the candidates for evidence, scope, expected value, and overlap with existing work; select one only when it is worth developing.
Do not evaluate the track from the seed title alone. The useful artifact is the analyzed candidate slate and, after selection, the completed implementation outcome.
Expand cautiously
Section titled “Expand cautiously”After a full completion:
- refine instructions if candidates were vague or repetitive;
- keep the long cooldown until candidate quality is consistent;
- add a second track only when it covers a genuinely different concern;
- shorten cooldown based on repository need and Worker capacity;
- disable a track before editing a risky prompt or investigating unwanted output.
Monitor open SI issues, latest completion, next eligibility, candidate selection latency, completion quality, and cost. A track with no free slot, or with a seed still waiting for selection, is correctly ineligible even if its nominal interval has elapsed.