Idempotency Keys and Webhooks for Generative Media APIs

Generative media APIs are async and expensive — exactly the combination that breaks naive HTTP clients. Timeouts, queue retries, and CI re-runs must not spawn duplicate credit holds. Wavemaker treats idempotent dispatch and signed webhooks as first-class on the workflow API surface (slug runs, private spec runs, Training Studio, and DCO). This post explains the mechanics integrators rely on in production — not managed POST /videos (see Automate video creation with the API for that path).

The failure mode you are preventing

Imagine a nightly job that POSTs five hundred slug runs. Halfway through, Kubernetes evicts the pod. The job restarts from SKU 1. Without idempotency, SKUs 1–250 submit again — double holds, double renders, angry finance.

Idempotency keys fix submit duplication. They do not replace memoization inside a run (identical block inputs can still memo-hit on a new run with a new key). Both layers matter: keys for transport retries, memo keys for creative iteration (Memoized workflow runs).

Idempotency-Key semantics

Send Idempotency-Key: <string> on dispatch POSTs (≤255 characters; UUID v4 recommended). Rules:

Behavior	Detail
First request	Creates the run, places the credit hold, returns 202 with run_id.
Retry with same key + same body	Returns the original run; idempotent_replayed: true; HTTP 200/202 depending on route.
Same key + different body	Rejected — keys bind to the first payload shape.
Scope	Per organization (API key’s org), not global.

Workflow slug example:

curl -X POST https://api.wavemaker.io/api/v1/w/product-teaser/runs \
  -H "Authorization: Bearer mcp_your_key" \
  -H "Idempotency-Key: catalog-2026-08-07-sku-4412" \
  -H "Content-Type: application/json" \
  -d '{"inputs":{"product_url":"https://shop.example/p/4412"}}'

MCP run_hub_workflow exposes the same header as idempotency_key. Agents retry tool calls aggressively — pass a stable key derived from your business id, not random() per attempt.

Key design patterns

• SKU + content hash: teaser-{skuId}-{hash(url, heroImageEtag)} — product updates naturally get new keys.
• CI build id: gha-{repo}-{run_id}-smoke — one smoke run per pipeline execution.
• User click: never reuse across users; include tenant id in the key string.

If you lose the HTTP response but still hold the key, replay with the same key before minting a new one — otherwise you may create a second run.

Metadata correlation (webhooks + reads)

Every kernel submit accepts metadata: up to 16 string key/value pairs, echoed on:

• GET /api/v1/kernel-runs/:id
• Batch GET /api/v1/kernel-runs?ids=…
• Every kernel_run.* webhook

Use metadata for SKU, order id, or feature flag — not secrets. Webhook handlers should route on metadata before hitting your database.

Webhook delivery contract

Pass webhook_url (HTTPS, no embedded credentials) and optional webhook_secret on submit. When the run reaches a terminal or approval state, Wavemaker POSTs JSON with:

Header	Purpose
X-Webhook-ID	Dedupe deliveries in your consumer
X-Webhook-Timestamp	Reject replays outside your skew window
X-Webhook-Signature	v1,<hmac-sha256> over id.timestamp.body

Verify before side effects. Store processed X-Webhook-ID values with TTL ≥ your retry horizon.

kernel_run event family

Event	When	Payload highlights
kernel_run.completed	Success, settled to actual	outputs, credits_charged, metadata
kernel_run.failed	Terminal failure	error, metadata
kernel_run.cancelled	User/API cancel	partial spend retained
kernel_run.awaiting_approval	approvalGate node	instance_key, message

Resume approval:

curl -X POST https://api.wavemaker.io/api/v1/kernel-runs/$RUN_ID/approval \
  -H "Authorization: Bearer mcp_your_key" \
  -H "Content-Type: application/json" \
  -d '{"instance_key":"from-webhook","decision":"approved"}'

Human-in-the-loop patterns: Human approval gates in AI pipelines.

Training jobs and DCO

POST /training-jobs accepts the same webhook fields (training_job.awaiting_checkpoint, failures, completion). DCO uses a separate dco.* subscription model on POST /api/v1/dco/webhooks — do not assume kernel headers on catalog events.

Render webhooks (POST /renders) ship unsigned today — poll GET /api/v1/renders/:id when authenticity matters.

Poll vs webhook in production

Pattern	When
Webhook primary	Server-side apps, CI with ingress, queue workers
Poll fallback	Local scripts, air-gapped staging
Tight poll loops	Avoid — use exponential backoff on GET /kernel-runs/:id

Slug-run architecture diagram: Every workflow is an API endpoint (/blog/diagrams/slug-run-api.svg).

Estimate / quote before a credit hold on a slug run.

Workspace tool runs and static ads

Idempotency extends beyond kernel slug submits:

• POST /api/v1/workspaces/:id/tools/:tool_name — upscale, reviews, and workspace composites accept Idempotency-Key on dispatch. MCP dynamic workspace tools mirror the header when exposed.
• POST /api/v1/static-ads and variant actions — batch creative pipelines retry safely when keys include variant id + action name.
• DCO produce/export/sync — catalog video operators rely on keys for autopilot loops; events use the dco.* family with subscription secrets rotated via POST /api/v1/dco/webhooks/:id/rotate-secret.

Kernel integrators can ignore DCO until they need catalog slots — but the same HMAC verification code often handles both once you normalize event type prefixes.

Clock skew and replay attacks

Reject webhooks when X-Webhook-Timestamp differs from server time by more than your chosen window (five minutes is common). Store processed webhook ids in Redis or SQL with TTL. For idempotency keys, the server — not the client — decides replay; never “increment” a key on retry.

Observability fields to log

On dispatch: Idempotency-Key, slug or spec id, metadata keys (not values if PII). On webhook receipt: X-Webhook-ID, event type, run_id, settled credits_charged. Correlate with GET /kernel-runs/:id node arrays when debugging partial failures — each node records spend and memo hit flags in structured logs documented under observability.

Comparison with managed video webhooks

Managed POST /videos jobs emit workspace-style completion events with the same signature scheme when webhook_secret is set — but job ids, pricing, and polling URLs differ. Workflow slug integrators should not assume video job handlers parse kernel_run.completed payloads without schema updates. Keep handlers separate or branch on event type string.

Failure handling checklist

1. Dispatch 5xx or timeout: retry with the same Idempotency-Key and body.
2. Webhook 5xx from your server: Wavemaker retries; make handlers idempotent on X-Webhook-ID.
3. 429 KERNEL_DAILY_RUN_CAP: backoff; cap is 200 runs/org/24h on kernel submits.
4. Partial batch failure: metadata lets you reconcile which SKUs completed without listing all runs.

Idempotency in multi-region consumers

If you run webhook consumers in two regions, both may receive retries — X-Webhook-ID dedupe must be global (DynamoDB, Postgres unique index), not in-memory only. Idempotency keys are org-global too: failover submit from secondary region with same key must replay, not duplicate.

Training job webhooks

training_job.awaiting_checkpoint pauses until pick_checkpoint — agents and CI should not poll training jobs every second. Webhook + email notification defaults reduce noise. Cancelled training jobs may not emit kernel_run events — use training-specific handlers.

Batch GET for reconciliation

Nightly jobs list pending SKUs from your DB, batch GET /kernel-runs?ids= for stale run_ids, reconcile status without waiting for webhooks lost during deploy. Complement webhooks — do not replace unless traffic is low.

Load testing idempotency

Before Black Friday catalog runs, script 100 identical submits with one key in staging — assert 99 replays and single hold. Then script 100 distinct keys — assert cap handling and webhook throughput on your receiver. Staging org keys avoid production royalty side effects.

Where to go next

• Slug runs end-to-end: /workflow-api and Workflows as API endpoints
• CI without duplicate submits: Slug-run workflows from CI
• OpenAPI reference for all routes: OpenAPI for generative media
• Developer tables: /developers