Data Consistency

Data Consistency is the cross-store audit surface for Colony Cloud. Route: /dashboard/consistency

It reconciles four independent stores that must agree for the pipeline to operate correctly: Postgres pipeline state (pipeline_issues), dependency edges (issue_dependencies), self-improvement linkage (si_issues), and projected GitHub labels (pipeline_issues.labels). When any of these stores drift out of agreement, a finding appears in this table so an operator can resolve it.

What the audit checks

Colony keeps several representations of each issue in sync. Normally they converge automatically, but background failures, crashed Workers, or edge cases in state transitions can leave one store stale relative to another. The audit compares:

• Pipeline state vs. dependency edges — whether the is_blocked flag on an issue reflects an actual active dependency, and whether subtask edges are cleaned up when an epic completes.
• Pipeline issues vs. GitHub — whether every open GitHub issue is tracked in pipeline_issues.
• SI completion linkage — whether a self-improvement issue that reached done had its completion event recorded in si_issues.
• Postgres state vs. projected labels — whether the labels stored in pipeline_issues.labels match what the authoritative Postgres fields (state, is_blocked, is_paused) say they should be.

For the canonical metric definitions and SQL for each finding kind, see Metrics — Data Consistency.

The audit table

The page shows findings in a single card with a table. Columns:

Column	What it shows
Issue	The GitHub issue number. Links to the issue detail page. Shows — when no issue number is associated (e.g. for orphan findings).
Repo	The owner/repo the finding belongs to. Links to the repository page. Hidden when a specific repository is selected in the global repo filter.
Kind	A neutral badge naming the finding type, plus a tooltip explaining the specific drift.
Detail	A compact human-readable summary of what is out of agreement — which state, labels, edges, or SI records have drifted.
Actions	A fix button for actionable finding kinds (see below). Shows (deferred) for kinds that require external API calls.

The table shows up to 50 findings per page. When there are more than 50, Previous and Next buttons appear below the table with a count of the current range.

Finding kinds

stale_blocked

An issue has is_blocked = true and is not in done state, but there is no active depends_on edge in issue_dependencies that would justify the flag. The blocked flag is stranded — it was set when a dependency existed but was not cleared when the dependency resolved.

Fix button: Clear stale flag — sends POST /pipeline/consistency/:id/clear-flag. The blocked flag is cleared immediately; no re-analysis is needed.

stale_subtask_edge

An active subtask dependency edge exists whose parent epic has already reached done state. The edge was not cleaned up when the epic completed, leaving dangling relationships that can confuse dependency resolution.

Fix button: Resolve subtask edges — sends POST /pipeline/consistency/:id/resolve-subtask-edges. All dangling subtask edges for the epic are marked inactive.

orphan_missing_from_pg

An open GitHub issue has no corresponding row in pipeline_issues — it is visible on GitHub but has not been ingested into the Colony pipeline.

Note

The orphan_missing_from_pg detector requires a live GitHub API call to enumerate open issues and compare them against Postgres. This detector is deferred and optional in Cloud. Rows of this kind render (deferred) in the Actions column with no fix button. If you see this kind in a future release, use the repo’s Connect a Repository flow to trigger re-ingestion.

si_linkage_gap

A self-improvement issue in si_issues is associated with a pipeline issue that has reached done, but si_issues.completed_at is still NULL — the SI completion event was not recorded when the pipeline issue finished.

Fix button: Backfill SI completion — sends POST /pipeline/consistency/:id/link-si. The completed_at timestamp is backfilled from the pipeline issue’s done transition.

label_drift

The authoritative Postgres state fields (state, is_blocked, is_paused) disagree with the projected GitHub labels stored in pipeline_issues.labels. Done issues are excluded. Drift can involve the state label (colony:<state>), the blocked label, or the paused label.

Fix button: Reconcile labels — sends POST /pipeline/consistency/:id/reconcile-labels. A label-reconciliation task is enqueued; the write-back to GitHub occurs asynchronously.

Empty state

When all consistency checks pass, the table is replaced with:

All consistency checks passed — no stale flags, dangling edges, linkage gaps, or label drift detected.

A link to Pipeline Live appears below, showing what is currently processing. This is the expected steady state for a healthy org.

Repository scoping

The global repository filter in the sidebar applies to the Data Consistency table. When a specific repository is selected, only findings for that repository are shown and the Repo column is hidden.

Operator guidance

Triage by urgency:

1. label_drift — label mismatches cause GitHub collaborators to see incorrect issue states. Reconcile these first.
2. stale_blocked — stranded blocked flags prevent issues from being picked up by Workers. Clear them to unblock the pipeline.
3. si_linkage_gap — missing SI completion records affect track overdue calculations and cadence scheduling. Backfill after clearing blocked issues.
4. stale_subtask_edge — dangling subtask edges are usually cosmetic but can affect dependency resolution for new work. Resolve these last.

Recurring findings indicate a systemic issue rather than a one-off race condition. If the same finding kind reappears after each fix, review the relevant pipeline transition (Worker crash during flag clearance, label write-back failures visible in the write queue) rather than applying fixes repeatedly.

Pagination — the table shows 50 findings per page. If a large number of findings appear at once, navigate through all pages before concluding the audit is complete.