UPSTREAM_SUBMISSION_UNCERTAIN: reconcile acceptance before retrying
UPSTREAM_SUBMISSION_UNCERTAIN means the gateway cannot establish whether an upstream queue accepted a creation request. A connection reset, a timeout, an unclassified HTTP response, or a successful response without a usable request identifier can produce this result. None proves that the upstream rejected the work. Do not automatically repeat the generation POST.
Save the local identifier
The HTTP 502 response includes a local job ID. This illustrative envelope uses a placeholder ID rather than a customer record:
{
"error": {
"type": "upstream_error",
"code": "UPSTREAM_SUBMISSION_UNCERTAIN",
"message": "Submission acceptance is unknown. Keep this job ID; do not resubmit automatically.",
"job_id": "local-job-id"
}
}
Persist error.job_id, the operation timestamp, model, and your own attempt
identifier. GET /api/v1/jobs/{id} identifies that resource using id.
When the marker was saved, the snapshot remains queued with
error.submissionOutcome set to uncertain; held credits remain reserved and
credits_actual stays unsettled. Queued does not establish upstream acceptance
or permission to send another creation request.
If saving the marker also fails, the job can remain queued without an error object. The returned local ID still supports investigation. Missing metadata is not evidence of rejection and does not authorize a refund.
Why submission is sent only once
Queue creation makes one HTTP POST, uses a 20-second client deadline, and rejects redirects. It does not use the SDK queue submission retry loop. Status and result reads retain their separate SDK path. Disabling creation retries prevents a lost response from silently becoming several upstream jobs; it does not create an exactly-once guarantee across acceptance and local persistence.
The wrapper treats missing local credentials or serialization failure before dispatch, and selected explicit HTTP rejections, differently. HTTP 400, 401, 403, 404, 422, and 429 are currently classified as rejection. Transport errors, 408, 409, and server errors are not included. Inspect the actual gateway code instead of deriving refund eligibility from an arbitrary 502.
Refunds and cancellation require evidence
New confirmed submission failures carry UPSTREAM_SUBMISSION_FAILED together
with submissionOutcome: rejected. Only that combination with unsettled
accounting enters automatic submission-refund recovery. Historical failure
markers without this evidence need investigation; they are not bulk rewritten
or automatically assumed to describe rejected work. Already returned credits
are not clawed back by this change.
Cancellation is not a workaround. A nonterminal task with an unknown outcome
or either missing upstream handle returns HTTP 409 with
SUBMISSION_RECONCILIATION_REQUIRED, preserving state and held credits.
Ownership checks still run first. Normal known-handle cancellation keeps its
documented behavior, but does not promise to physically stop upstream work.
Escalate the existing attempt
Provide the local ID, timestamp, model, and saved response to the operator. Reconciliation must use upstream or operational evidence before deciding the outcome or accounting. There is no public creation-idempotency contract that makes a second POST refer to the same attempt. Do not test recovery by creating another paid render or editing balances directly.
Read upstream failures, submission boundaries, and explicit cancellation for the related, distinct paths.
Where these facts come from
- codebase: src/ai/providers/fal-client.ts
- codebase: src/ai/providers/errors.ts
- codebase: src/ai/jobs/service.ts
- codebase: src/ai/jobs/db.ts
- codebase: src/ai/api/errors.ts
- https://fal.ai/docs/documentation/model-apis/inference/queue