Skip to content

webhooks: add durable signed lifecycle delivery - #179

Merged
yanpgwang merged 2 commits into
mainfrom
codex/durable-webhooks
Aug 24, 2026
Merged

webhooks: add durable signed lifecycle delivery#179
yanpgwang merged 2 commits into
mainfrom
codex/durable-webhooks

Conversation

@yanpgwang

Copy link
Copy Markdown
Owner

Problem

Mango applications currently have to poll or keep an SSE recovery loop open to notice major Session and scheduled Deployment Run lifecycle changes. A self-hosted operator needs an application-owned, signed notification path that survives API and worker replacement without delegating runtime behavior to a hosted service.

Solution

  • add Workspace-scoped /v1/webhooks CRUD, pagination, status changes, deletion, and explicit signing-secret rotation
  • return whsec_ signing secrets only on create/rotation and encrypt them with the existing operator keyring
  • snapshot enabled subscriptions in the same PostgreSQL transaction as each source lifecycle change, with no backfill
  • persist exact thin-event payload bytes and leased per-endpoint deliveries
  • deliver Standard Webhooks HMAC headers with stable event IDs and fresh per-attempt timestamps
  • retry up to three times with jittered 5–120 second backoff; never follow redirects; auto-disable redirect and non-public-address endpoints
  • retain terminal delivery state for 30 days with bounded cleanup
  • emit the selected Session lifecycle and scheduled-only Deployment Run success/failure events
  • document the API, operational boundary, supported event subset, delivery semantics, limitations, and CMA/Standard Webhooks provenance

CMA design decisions

Adopted: thin fetch-after-notification payloads, established event names, Standard Webhooks headers/signing, one-time secrets, subscription-at-emission/no-backfill semantics, duplicates and unordered delivery, three attempts, and immediate redirect/private-address disable.

Changed for Mango: endpoint management is a Mango HTTP resource because there is no hosted Console; secrets and delivery state remain inside Mango; organization_id is omitted because Mango has no Organization resource; only lifecycle events backed by current Mango behavior are exposed.

Deferred/rejected: no Anthropic auth/beta headers, no SDK compatibility requirement, no synthetic deployment_run.started, and no invented sustained-failure duration because CMA does not publish its threshold. Mango records the continuous-failure window for a future operator policy.

Acceptance criteria

  • endpoint and source event changes commit atomically and survive worker replacement
  • the signing secret is never returned by get/list/update and ciphertext is AAD-bound to its endpoint
  • retries preserve event ID/body and refresh timestamp/signature
  • concurrent workers cannot complete another worker's lease
  • endpoint disable/delete races cannot leave a newly inserted pending delivery behind
  • private-network and redirect targets cannot be followed
  • API, OpenAPI, capabilities, operations docs, and provenance agree

Verification

  • make verify
  • make docs-check
  • make security
  • make test-service
  • MANGO_TEST_DATABASE_URL='postgres://postgres:postgres@localhost:5432/mango?sslmode=disable' go test ./internal/pg -run Webhook -count=1

@yanpgwang
yanpgwang merged commit c11d675 into main Aug 24, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant