Skip to content

Recover a Dead-Lettered Issue

A dead letter is a recorded pipeline transition that Colony could not complete permanently—for example, a failed state transition or task enqueue after retries were exhausted. It is not a synonym for the issue state failure-blocked, and one issue can have more than one historical dead-letter record.

The Cloud API exposes unresolved records at GET /pipeline/dead-letters. The current dashboard can surface dead-letter counts and issue-level failure context, but it does not provide an Inbox DLQ category or row-level Retry/Resolve controls. Use an authenticated API client for direct dead-letter operations, or use the issue detail page for a separate workflow action when that is the intended recovery.

Read the dead-letter error together with the issue timeline and current state. The current state may have advanced by another route since the record was created.

Before retrying, verify:

  • the target transition is still appropriate for the issue;
  • the Worker pool and database are healthy;
  • provider credentials and permissions are valid;
  • any missing workflow definition, task executor, or queue dependency is now available;
  • a permanent validation error has actually been corrected.

For a normal issue-stage failure rather than a dead-letter transition, use Triage a Stuck Issue.

Retry the recorded transition:

POST /pipeline/dead-letters/{id}/retry

Or acknowledge it without retrying:

POST /pipeline/dead-letters/{id}/resolve

Use resolve when the issue already recovered, was intentionally cancelled, or the recorded transition is no longer appropriate. These endpoints require an authenticated Cloud session and are tenant-scoped.

  1. List unresolved dead letters again and confirm the record is gone.
  2. Inspect the issue’s current state and timeline.
  3. Confirm any task derived from the transition is pending, claimed, or completed.
  4. If replay fails again, diagnose the new error before another attempt.