Quality Contracts for AI Pipelines: Retry Until, Review Facts, and No Silent Give-Ups

Quality contracts are how you tell an AI pipeline “do not ship this frame until it passes review” — not as a comment in a prompt, but as structure in the graph. On Wavemaker, that means combinators like Retry Until and Best of N, wired to review_image / review_video outputs and conditions over facts such as review.score >= 7. They are the workflow-platform equivalent of the chat pipeline’s bounded regen loops — visible, budgeted, and memo-safe.

Answer-first: what you are buying

When you wrap generate_image in retryUntil, you declare:

• A condition (parsed micro-expression, never eval).
• maxAttempts (≤ 5).
• onExhausted: ship lastAttempt or fail the run.

When you use bestOfN, you declare up to six parallel candidates, a scoreBy expression, and the scheduler keeps the winner. Both patterns show up in production templates — the visual builder tour zooms into a Music Video map where Best of N sits inside per-scene branches.

That is a quality contract: generation is subordinate to review facts, with a declared exhaustion policy.

Why visible structure beats hidden retries

Hidden retries fail three audiences:

1. Operators cannot see why spend spiked (was it map fan-out or six invisible regens?).
2. Integrators cannot rely on API behavior — the same slug run might pass or fail depending on undocumented luck.
3. Creators cannot remix responsibly — forks inherit mystery behavior.

Combinators fix this. The compile gate includes retry and Best-of-N multipliers in the cost envelope, so quotes reflect worst-case attempts before hold. Run telemetry lists node instance keys like scenes[2]/frame@3/gen — attempt three is legible in logs and support.

Chat’s orchestrator still performs quality loops for one-off generations (quality gates doc); workflows freeze the loop for products. If your agency ships the same ad scaffold daily, the contract belongs in the spec.

Wiring conditions to review facts

Block adapters expose facts on artifacts — e.g. review_image publishes score and structured issues for conditions. Typical patterns:

review.score >= 7 && review.issues == 0

The expression parser resolves fact paths against the artifact in scope; it does not execute JavaScript. MAX_EXPRESSION_LENGTH bounds input size — another compile-time guard.

retryUntil re-enters the body until the condition passes or attempts exhaust. bestOfN runs N candidates (scheduler keys pick~2/...) and selects the highest scoreBy value — useful when variance is high and you want parallel exploration, not serial correction.

Pair with Map carefully: a quality contract inside a map multiplies attempts by scene count. The envelope compiler uses documented fan-out assumptions — design maps with scene caps when budgets are tight.

Memoization and the cache-loop invariant

Wavemaker memoizes block outputs by content hash: same block ref, params, inputs, bindings → same artifact, no re-bill. That is essential for iterative editing (change the last node, upstream memo hits).

Quality contracts would break without instance salt:

• Attempt 1 generates image A; review rejects A.
• Attempt 2 with identical inputs would memo-hit A — infinite rejection loop, zero progress.

The scheduler stamps instanceSalt for retryUntil attempt ≥ 2 and bestOfN candidate ≥ 2 ("@2", "~3"). First attempts stay unsalted so innocent replays dedupe; contract retries always diverge in the memo key. Kernel invariant: retry/bestOfN re-executions are memo-salted — a quality contract can never cache-loop on its own rejected artifact (building workflows).

Style salt is separate: editing WorkflowSpec.style changes the look anchor and busts memo for styled generation — so a contract cannot pass old-looking frames after you lock a new visual medium in the spec panel (style-locked workflows).

Approval gates vs. automated contracts

approvalGate is a different species of contract: human judgment, not model score. The run parks on waiting-approval, emits kernel_run.awaiting_approval webhooks, and resumes on approve (timeout up to 72 hours). Use it when legal, client sign-off, or brand safety requires eyes before generate_video_clip or publish steps — human approval gates goes deeper.

Automated retryUntil/bestOfN handle stochastic model failure; approvalGate handles policy.

Authoring in the builder

On the AI workflow builder canvas, Control Flow blocks drag like any other node. Nest them inside Map groups for per-scene policies — e.g. retry seed frames but accept bestOfN on hero scenes only.

The workflow copilot is trained to recommend quality patterns when you ask for reliable ad stills or consistent character frames — always through validate_spec, never by inventing hidden retries (copilot post).

Visual builder canvas — typed blocks, wires, and quality contracts.

Publish and slug runs inherit contracts

Publish freezes the graph into an immutable workflow_spec_versions row. Foreign runners on /w/{slug} execute the same contracts — premium graphs stay hidden, behavior does not. Open-access workflows still earn 10% royalty on foreign settled platform credits; premium runs use creator-set pricing with 80/20 split on success (marketplace economics).

Design checklist

1. Pick retryUntil when failures are correctable (prompt adherence, anatomy) and serial attempts help.
2. Pick bestOfN when variance dominates and parallel candidates beat repeated edits.
3. Set onExhausted to fail when shipping a bad frame is worse than no output.
4. Keep map fan-out × attempts inside your envelope; quote before batch campaigns.
5. Add approvalGate when automation ends and human accountability begins.

Where to go next

• Kernel combinators reference: Production kernel § IR
• Best of N mechanics: Best of N generation explained
• Memoized re-runs: Memoized workflow runs
• Build: /workflows/new · AI workflow builder

Worked example: seed frame contract inside Map

Imagine a social ad workflow with four scenes in a Map over board.scenes. Inside each iteration you wire enrich_prompt → generate_image → review_image, then wrap the generator and reviewer in retryUntil with condition review.score >= 7, maxAttempts 4, and onExhausted: fail. The compile envelope multiplies image+review credits by four scenes times up to four attempts — the quote you see before Run is pessimistic on purpose. When scene two fails all four attempts, the run fails closed instead of animating a frame that would fail review_video later — that is the contract doing its job early.

If you switch the inner combinator to bestOfN with n: 3 and scoreBy: review.score, you pay for three reviews per scene up front but often skip serial regen latency. Pick retry when edits fix adherence; pick Best of N when the model explores composition. Both patterns appear in shipped templates — open Music Video on /workflows/new and compare node labels in the run rail after a failed scene.

Envelope hard stop and operator visibility

Runs stop if actual spend approaches 1.25× the compiled envelope. Quality contracts are the main reason envelopes inflate — that is desirable. Operators debugging spend should sort run nodes by instance key: @3 on a retryUntil body is attempt three, not a mysterious duplicate block. Support can trace memo hits separately: a memo hit shows no provider call but still marks the node complete with prior costCredits preserved on replay.

Slug runs and MCP inherit the same contracts

Foreign runners executing POST /api/v1/w/{slug}/runs do not get a “lite” quality path. Premium graph visibility is hidden, not behavior. Integrators should document exhaustion policy in their own API docs — onExhausted: lastAttempt ships the best failed frame; fail returns a terminal error your webhook must handle. Pair with approvalGate when your customers need sign-off even after a passing score (human approval gates).