Recover a Dead-Lettered Issue
A dead letter is a recorded pipeline transition that Colony could not complete permanently—for example, a failed state transition or task enqueue after retries were exhausted. It is not a synonym for the issue state failure-blocked, and one issue can have more than one historical dead-letter record.
List unresolved records
Section titled “List unresolved records”The Cloud API exposes unresolved records at GET /pipeline/dead-letters. The current dashboard can surface dead-letter counts and issue-level failure context, but it does not provide an Inbox DLQ category or row-level Retry/Resolve controls. Use an authenticated API client for direct dead-letter operations, or use the issue detail page for a separate workflow action when that is the intended recovery.
List the authoritative records with the CLI:
colony dead-letterscolony dead-letters --repo owner/repocolony dead-letters --jsonThe table includes the record ID, issue, source and target states, agent, error, and age. GitHub labels and log searches can help with diagnosis, but they cannot tell you whether the dead-letter record itself is unresolved.
Diagnose the failed transition
Section titled “Diagnose the failed transition”Read the dead-letter error together with the issue timeline and current state. The current state may have advanced by another route since the record was created.
Before retrying, verify:
- the target transition is still appropriate for the issue;
- the Worker pool and database are healthy;
- provider credentials and permissions are valid;
- any missing workflow definition, task executor, or queue dependency is now available;
- a permanent validation error has actually been corrected.
For a normal issue-stage failure rather than a dead-letter transition, use Triage a Stuck Issue.
Retry or acknowledge
Section titled “Retry or acknowledge”Retry the recorded transition:
POST /pipeline/dead-letters/{id}/retryOr acknowledge it without retrying:
POST /pipeline/dead-letters/{id}/resolveUse resolve when the issue already recovered, was intentionally cancelled, or the recorded transition is no longer appropriate. These endpoints require an authenticated Cloud session and are tenant-scoped.
Retry the exact recorded transition:
colony dead-letters retry <id>A successful retry applies the transition, enqueues any derived task, and resolves the record. To acknowledge without retrying:
colony dead-letters resolve <id>resolve-all is available, optionally with --repo, but use it only after reviewing the records; it acknowledges them without replaying their transitions.
Verify the result
Section titled “Verify the result”- List unresolved dead letters again and confirm the record is gone.
- Inspect the issue’s current state and timeline.
- Confirm any task derived from the transition is pending, claimed, or completed.
- If replay fails again, diagnose the new error before another attempt.