image seed reproducibility test: Test a saved seed without assuming identical bytes

An image seed reproducibility test should begin with a precise question. Are you checking that the client preserves the seed, that repeated requests resemble each other, or that files are identical? Those are different claims and require different evidence. This article proposes a test method rather than reporting an experiment.

Verify serialization first

The current image schemas accept an optional integer seed, including zero. The route copies seed directly and removes only undefined properties. Test your request builder offline before scheduling any generation work.

Save the complete serialized request with an explicit model, prompt, ratio, image count, optional reference, negative prompt, and reviewed extra. A seed alone cannot explain a result when the rest of the request changes.

Preserve reference bytes or a content hash, because a stable URL can later serve a different image. Also record the application and adapter revision where available.

Define the comparison

If the question is byte identity, archive output files and compare their hashes. Different encodings or metadata can produce different hashes even when images look similar, so interpret that result according to the stated question.

If the question is visual repeatability, define a human comparison rubric before looking at the pair. Choose concrete features such as composition, object placement, or major colors. A vague conclusion that both look similar is difficult to audit.

Do not transfer a seed between models and expect the same visual result. The shared integer field is an input convention, not a shared random-number space or a cross-model reproducibility guarantee.

Plan controlled observations

Hold the complete request constant for the repeated-condition group. If you intentionally vary the seed, put that in a separate group. Keep the number of planned samples visible and obtain the ordinary budget authorization before real submissions.

Record actual submission times, job identifiers, terminal outcomes, and all returned outputs. Do not silently discard failed or unresolved attempts and report only the appealing pairs.

A provider or upstream implementation can change without the public model identifier changing. When repeatability changes over time, state that the available provenance may be insufficient to attribute the difference to a particular upstream revision.

Interpret findings narrowly

A matching pair would establish an observation for those requests and conditions. It would not prove all future requests are deterministic. A differing pair would show a difference under the recorded conditions, not automatically prove that the seed was ignored.

For this proposed protocol, leave result fields empty until someone performs and reviews the experiment. Avoid inventing sample images or numerical repeatability scores.

Acceptance requires a preserved request, a defined comparison method, archived artifacts, and conclusions limited to the observed set. Continue with seed zero handling, prompt test sets, and regression scorecards when building a longer-term evaluation.

Continue with quality and testing

• generation model regression checklist: Maintain a release scorecard with real evidence
• generation prompt revision diff: Review the change as a field-level hypothesis
• AI image color drift test: Control viewing conditions before judging color
• Browse all quality and testing guides