Genace
Sign inGet started

generation job transactional outbox: preserve intent without promising exactly once

An application can lose a generation request between committing a business action and publishing a queue message. A transactional outbox closes that local gap by storing the operation and its dispatch intent in the same database transaction. It does not make the subsequent network POST part of that transaction, and it does not create remote exactly-once semantics.

Commit the local intent atomically

Imagine an editor approving a storyboard frame. Store the approved revision and a generation operation in your own database, then insert an outbox row before committing. A dispatcher can find unsent rows after a restart. The following SQL is a conceptual transaction for application-owned tables, not Genace’s internal schema.

BEGIN;
INSERT INTO app_generation_operations
  (operation_id, owner_id, specification_hash, phase)
VALUES ($1, $2, $3, 'prepared');
INSERT INTO app_outbox (event_id, operation_id, event_type)
VALUES ($4, $1, 'generation_requested');
COMMIT;

Use unique constraints for operation identity and event identity. Store the reviewed payload or an immutable reference to it so a dispatcher does not reconstruct a different prompt from mutable application data. Include a schema version for the event itself.

Dispatch under a conditional claim

The dispatcher claims a row, loads the operation, checks allowance and current phase, then submits once. Save the returned job_id on the operation before acknowledging local dispatch completion. A recovered operation with that ID moves to observation rather than another creation attempt.

Genace’s own reservation transaction covers its credit deduction and local queued job insertion. That is distinct from your outbox transaction and does not span your database, queue, and the remote provider. Keep these boundaries explicit in architecture diagrams and incident records.

Represent the uncertain network interval

If the connection fails after the POST may have reached the server but before you save an identifier, mark the operation uncertain. A lease expiration must not blindly return it to prepared. The remote creation route has no published request idempotency contract, so a second dispatch may create another billable job.

An HTTP 502 submission failure can contain error.job_id; preserve it and inspect the local job’s accounting. A successful accepted response instead uses top-level job_id. Generic HTTP errors and lost connections require their own evidence review rather than a universal retry rule.

Separate collector events from creation events

Once an identifier is durable, schedule a collection event containing only your operation reference and known remote ID. Repeating that event should perform GET observation. A succeeded render can then create a separate delivery operation, whose retries download and store existing outputs.

Do not delete the operation because the outbox row was sent. The operation is the durable record needed for support, ownership checks, and budget reconciliation. Retain event attempt metadata without treating every queue delivery as a new generation attempt.

Prove the local guarantees

Crash before commit, after commit but before dispatch, after acceptance, and after saving the remote ID. Verify the first leaves no partial intent, the second remains discoverable, and the last resumes observation. The acceptance-to-persistence gap must remain an explicit uncertain case rather than an exactly-once claim.

See batch coordination, unique operation keys, and durable acceptance.

Continue with production operations

Where these facts come from