generation job trace correlation: link application spans to saved job identifiers
Generation spans several temporary HTTP requests and may outlive the process that submitted it. Trace correlation therefore needs a durable operation mapping, not one indefinitely open span. Link your application’s request and worker telemetry to the saved Genace job ID while preserving the distinct meaning of each identifier.
Define the identity map
Keep a business operation ID, an HTTP attempt ID, a local trace or span context, the Genace job ID, and an optional upstream provider identifier. The first three belong to your application. The job ID comes from accepted or recoverable API responses. The provider identifier exists only when actually returned.
{
"operation_id": "application-operation-42",
"submission_attempt_id": "application-attempt-1",
"application_trace_reference": "stored-trace-reference",
"remote_job_id": "saved-job-id",
"provider_request_id": null,
"identifier_source": "accepted_response.job_id"
}
This is an application mapping example, not a claim that Genace returns a trace reference. Store it with the operation under appropriate access controls so a restarted collector can attach observations to the same business action.
Instrument short spans around meaningful actions
Create a span for submission transport and another for saving acceptance. Later workers create observation and delivery spans linked through the operation record. This reveals whether latency comes from API calls, queue delay, database writes, or asset transfer without holding a span open for the entire render.
If your services use W3C Trace Context, propagate validated context within your own trusted boundaries according to your instrumentation stack. The public routes described here do not promise to echo traceparent, propagate your trace into a provider, or return an upstream trace identifier. Avoid drawing a continuous distributed trace line where no supporting contract exists.
Preserve actual error identifiers
A normal submission returns job_id. HTTP 502 with UPSTREAM_SUBMISSION_FAILED can provide error.job_id; preserve it even though the request failed. GET snapshots identify the same local job with id.
HTTP error envelopes may include error.upstream_request_id. Job diagnostic errors use an optional upstreamRequestId field. Save the original field and source category when present. Do not derive a provider request ID from the local job ID or fill it with a random application UUID just to satisfy a dashboard column.
Explain sampling and missing history
Trace sampling can omit spans, and SSE reconnection does not replay an event sequence in the current protocol. A missing observation span does not prove the job stopped. Your durable operation record remains the recovery source, while traces explain selected execution paths.
Keep prompts, bearer keys, and signed output URLs out of span attributes. Use safe model IDs and state categories, and watch high-cardinality fields in metrics. A correlation ID may be useful in restricted traces without being appropriate as a metric label.
Acceptance checks
Test successful acceptance, recoverable submission failure, a response without provider ID, and a collector restart. Verify each maps to the correct saved job without inventing fields. Confirm delivery traces reuse the existing operation and no reconnect path creates a generation. Finally, inspect exported telemetry for synthetic secrets and document where cross-service trace continuity intentionally ends.
See structured log schema, durable acceptance, and SSE recovery.
Continue with production operations
Where these facts come from
- Editorial keyword hypothesis; search volume not measured
- https://www.w3.org/TR/trace-context/
- codebase: src/ai/api/errors.ts
- codebase: src/ai/providers/types.ts
- codebase: src/ai/jobs/http.ts