generation API deadline propagation: budget each layer without losing jobs

A browser, proxy, application server, and generation API can each impose a different timeout. Without a shared budget, an inner request may wait longer than the outer layer can tolerate. Propagate your application’s remaining time deliberately, while keeping a request deadline separate from the remote job’s lifetime. Genace does not consume a custom application deadline header in the routes described here.

Define the budget at the application boundary

Choose the maximum time a particular operation may occupy its current request. A submission budget covers network dispatch and acceptance, while an observation budget covers waiting for a known job. Store the accepted job ID durably so neither budget needs to encompass the entire render.

Within one process, use a monotonic clock for elapsed time. Across services, absolute timestamps can be affected by clock differences. If your infrastructure propagates a deadline header, validate it, cap it to your own policy, and never let an untrusted caller request unlimited processing.

export function observationWindow(remainingMilliseconds) {
  if (!Number.isFinite(remainingMilliseconds) || remainingMilliseconds < 5000) {
    return { kind: 'defer' };
  }
  const transportReserve = 4000;
  const waitSeconds = Math.max(0, Math.min(30,
    Math.floor((remainingMilliseconds - transportReserve) / 1000)));
  return {
    kind: 'read',
    waitSeconds,
    clientTimeoutMilliseconds: remainingMilliseconds,
  };
}

This is an illustrative local allocation policy. The four second reserve is not a Genace guarantee. Measure response transfer and proxy overhead in your deployment, and leave enough margin for returning a useful response before the outer request expires.

Fit long polling into the remaining window

Genace’s job GET clamps its wait parameter to zero through sixty seconds and returns the latest state when its waiting window ends. The polling helper can still report queued or running. Choose a shorter wait value when the outer budget is small, or return a resumable local pending response instead of starting a request that cannot complete usefully.

A client timeout should exceed its requested server waiting window. A slightly longer network timeout alone does not guarantee success, but setting it shorter guarantees needless aborts under normal waiting behavior. Keep transport timeouts, pool acquisition limits, and total operation deadlines visible as separate diagnostics.

Preserve outcomes at every deadline

If a known job’s GET times out, retain the identifier and last observed state. If submission times out before an accepted ID is received, record uncertainty. Automatically repeating the POST can create another job because remote creation has no documented idempotency contract.

Aborting a local request does not cancel the remote render. An intentional cancellation uses the separate authenticated job operation and has its own race with completion. Do not relabel a running job as failed merely because the current browser request ran out of time.

Verify nested deadlines offline

Use a local mock and controlled clocks to test an outer budget shorter than the requested long poll, a slow response body, and connection acquisition pressure. Confirm the helper defers when no useful budget remains and that the saved ID survives all observation failures.

Test shutdown just after acceptance and just before persistence. Your recovery record should make those branches distinguishable. These tests reveal budget mistakes without creating paid generations or waiting for a real model.

Continue with aborted submissions, serverless collection, and controlled timer tests.

Continue with code and integration

• AI generation API Docker: inject secrets when the worker starts
• AI generation API env variables: configure local and deployed clients consistently
• AI generation API Express: validate requests before forwarding them
• Browse all code and integration guides