Troubleshoot Webhooks and Provider Permissions

Inbound provider webhooks tell Colony that an issue, label, comment, pull request, or check changed. They are different from the outbound subscriptions under Colony Cloud Settings → Webhooks, which send Colony notifications to your own endpoints.

Identify the failure boundary

Follow one event from provider to pipeline:

The provider emitted a supported event.
The event reached Colony’s public receiver.
Colony authenticated the request.
The receiver mapped the repository and event.
The Mayor applied the event or scheduled a catch-up poll.
A Worker claimed any resulting task.

Do not restart Workers first when the event never reached the receiver, and do not rotate provider credentials when the task is already pending in Postgres.

Check Colony Cloud GitHub connectivity

Open Settings → VCS Connections and confirm the coder App is healthy for the correct GitHub organization. Confirm the operations App as well when issue mutation, review, or merge is failing.
Verify the repository is included in the App installation’s selected repository scope and enabled in Colony Cloud.
In GitHub, inspect the App installation’s recent deliveries. Confirm the event was sent, note its delivery ID, response status, and response body.
Compare the event time with Pipeline Events, the repository timeline, and connection-health messages in Colony Cloud.
Reconnect or reinstall an App only when its installation was revoked, moved, or granted the wrong repositories. Reinstallation does not fix branch policy or an unhealthy Worker.

Cloud manages the inbound endpoint and signature secret. You should not create a separate GitHub webhook for a normal Cloud-connected repository.

Configure the receiver and an HMAC secret:

webhook:
  enabled: true
  port: 9800
  secret_env: GITHUB_WEBHOOK_SECRET

Start the optional Compose profile with docker compose --profile webhook up -d, expose the receiver over HTTPS, and register the public /webhook URL in GitHub. The secret entered in GitHub must match the environment variable available to the receiver container.

GitHub requests must include a valid X-Hub-Signature-256. A missing or mismatched signature returns an authentication error before repository routing.

Add Basic authentication fields to the literal ado configuration:

ado:
  organization: acme
  project: Platform
  repo: payments
  token_env: ADO_CODER_PAT
  webhook_username: colony-webhook
  webhook_password_env: ADO_WEBHOOK_PASSWORD

Create Azure DevOps service-hook subscriptions that send supported work-item and pull-request events to the receiver. Configure the same username and password in the subscription. ADO authentication failures return ADO_AUTH_NOT_CONFIGURED or INVALID_CREDENTIALS before event handling.

Provider and webhook configuration are startup-only. Restart the receiver, Mayor, and any process whose provider identity changed; config hot reload does not replace these clients.

Validate reachability and health

Confirm the receiver’s health endpoint is reachable inside the deployment before testing its public URL.
Send a provider test delivery. Do not use an unsigned curl body as proof that GitHub signature handling works.
Inspect receiver logs for the delivery ID, event name, repository mapping, status, and authentication result.
Inspect Mayor logs or pipeline events for the applied wake event.
Confirm the resulting task appears in the task queue and that a Worker is available for that repository.

Validate permissions by operation

Failure	Identity and permission to inspect
Clone, branch, commit, push, or open PR	Coder GitHub App/PAT or ADO coder PAT; repository/code write access.
Add labels/tags, comments, or issue state	Operations identity where configured; issue/work-item write access.
Publish formal review or approve	Operations identity; pull-request review permission; identity must differ from author when required.
Merge	Operations API permission plus coder git permission for any rebase/force-push step; branch policy must allow the result.
Read CI checks	Provider checks/build read permission and exact configured check names.
Administer branch policy	Repository administration on GitHub or policy administration on ADO.

For self-hosted installations, run both checks after changing credentials:

colony check --stage credentials
colony check --stage repository

The first validates identity access; the second exercises the target repository. Container credentials must also be present inside the relevant process, not only in the host shell.

Understand polling fallback

The self-hosted Mayor periodically performs a full provider sync. When webhook activity is absent, the inactivity fallback forces full syncs rather than trusting an empty event cache. This keeps the pipeline functional without webhooks, but increases provider API traffic and response latency.

Webhooks are healthy when events arrive and the Mayor can use cache-first cycles between scheduled full syncs. If logs repeatedly report webhook inactivity despite provider deliveries, fix receiver-to-Mayor dispatch rather than lowering the poll interval.

Recovery sequence

Correct endpoint, secret, App installation, PAT scope, or repository mapping.
Restart only processes that consume the changed startup credential or provider configuration.
Re-run provider and Colony checks.
Redeliver the original provider event when the provider supports it, or wait for the Mayor’s full-sync safety net.
Confirm exactly one pipeline row exists and the expected task is pending, claimed, or complete.