Roll Out Self-Improvement Tracks
Self-improvement tracks periodically seed an issue that asks Colony to inspect a repository and propose a slate of useful improvements. A person selects a candidate before development begins. Tracks use the same pipeline and Worker capacity as ordinary issues.
Start with one narrow track
Section titled “Start with one narrow track”Choose a repository where one additional open issue will be easy to observe. Use one track, a seven-day cooldown, and a prompt focused on a single concern such as missing tests or stale documentation.
Important field semantics:
- Name is the stable track key.
- Label is the actual code-host label applied to seeded issues, not a display title.
- Cooldown starts after the latest linked SI issue completes. A track can also have only one open linked issue at a time.
- Cron schedules use UTC.
- Weight can influence strategic ranking; it does not make the scheduler run a track more often or move its work ahead in the shared task queue.
Create the disabled track
Section titled “Create the disabled track”Open SI Tracks (/dashboard/si-tracks), filter to the repository, and add a track with:
- name
test-coverage; - label
colony:si-test-coverage; - cooldown 10,080 minutes;
- enabled off.
Owners and administrators can create and edit tracks. The repository-specific view uses the same underlying track records.
First, enable the SI scheduler in the Colony installation’s colony.config.yaml:
self_improvement: enabled: true cooldown_minutes: 10080Apply the configuration, then open the monitor dashboard at http://localhost:9106 (or the configured monitor port). Sign in if agents.monitor.auth is configured and select the Tracks tab.
- Select the target repository when the dashboard manages more than one.
- Click Add Track.
- Enter
test-coveragefor Name andcolony:si-test-coveragefor Label. - Set Cooldown to
10080minutes. - Clear Enabled so the track cannot seed before its prompt is ready.
- Click Create Track.
The dashboard writes the track to Colony’s Postgres-backed track store. See Monitor Dashboard Reference for dashboard access and authentication.
Add a prompt version
Section titled “Add a prompt version”A prompt version has three operational fields:
seed_title: the title of the SI seed issue;seed_body: stable context placed in that issue;instructions: how the analyzer should search the repository and construct candidates.
The seed issue exists before candidate analysis, so prompt instructions cannot prevent the scheduler from creating it. Instead, make the instructions demand evidence, bounded candidates, and honest reporting when no worthwhile candidate is found.
Use Manage Prompts on the track, add a prompt version, and confirm that it is active.
In the monitor dashboard’s Tracks tab:
- Click the
test-coveragetrack row to expand Prompt Versions. - Click Add Version.
- Enter a seed title such as
Propose a focused test-coverage improvement. - Use the seed body for stable issue context and expected candidate qualities.
- In Instructions, ask Colony to find untested behavior with user-visible risk, cite file and test evidence, and prefer work that fits one pull request.
- Add a short version note and click Add Version.
Adding a Postgres-backed prompt version makes it the track’s active version. The expanded row displays the active content and version history; click Activate on an older version to roll back.
Enable and evaluate the first cycle
Section titled “Enable and evaluate the first cycle”In Cloud, enable the track’s toggle. In a self-hosted deployment, return to the monitor dashboard’s Tracks table and click the track’s Off button so it changes to On.
A newly eligible track can seed on the next scheduler loop. Colony then analyzes repository signals and presents a candidate slate for human selection. Review the candidates for evidence, scope, expected value, and overlap with existing work; select one only when it is worth developing.
Do not evaluate the track from the seed title alone. The useful artifact is the analyzed candidate slate and, after selection, the completed implementation outcome.
Expand cautiously
Section titled “Expand cautiously”After a full completion:
- refine instructions if candidates were vague or repetitive;
- keep the long cooldown until candidate quality is consistent;
- add a second track only when it covers a genuinely different concern;
- shorten cooldown based on repository need and Worker capacity, remembering that eligibility begins after completion;
- disable a track before editing a risky prompt or investigating unwanted output.
Monitor open SI issues, latest completion, next eligibility, candidate selection latency, completion quality, and cost. A track that has an open issue is correctly ineligible even if its nominal interval has elapsed.