image generation evaluation rubric: Give human scores observable anchors

An image generation evaluation rubric makes a human judgment easier to inspect. It does not turn a subjective creative task into a universal model ranking. Define observable evidence for your intended asset and preserve disagreements rather than hiding them behind a single average.

Divide hard requirements from preferences

Start with requirements that can invalidate delivery: the correct product, the required number of objects, readable approved text, or sufficient room for a later overlay. Give each a direct check with a clear failure description.

Then add graded preferences such as background distraction or lighting suitability. Keep these separate from hard failures. A beautiful background should not compensate mathematically for a misspelled product name if the name is mandatory.

For a fictional catalog brief, product identity might fail when a handle or closure changes, remain uncertain when the relevant area is obscured, and pass when the visible distinguishing details match the approved source. These anchors describe the decision, not measured performance from any model.

Write score anchors before viewing outputs

Choose a small scale and define what each level means for each criterion. For subject separation, a low anchor might describe an edge lost against the background, while a high anchor describes a clearly readable outline at the intended display size.

Avoid an anchor such as looks professional. Two reviewers can agree with those words while checking entirely different features. Ask for a short note identifying the region or detail supporting the score.

Include an unable-to-assess option when the evidence is insufficient. Forcing a numeric judgment on a hidden product detail creates false precision. Resolve the missing evidence or reject the asset for that use.

Control the review context

Present images at the same display size and under consistent viewing conditions. Hide model labels when practical if brand expectations might influence judgment. Keep the approved brief and relevant reference available to every reviewer.

Record reviewer identity, rubric revision, artifact identifier, and review timestamp. If reviewers disagree on a hard requirement, ask them to point to the evidence and have an assigned decision owner resolve it. Preserve the original judgments alongside the decision.

The current image adapters support explicit model IDs and standard request settings, but they do not provide this human rubric as a response field. Store review records in your application linked to jobs and archived outputs.

Acceptance criteria

Pilot the rubric with a few existing or deliberately prepared example assets. Check whether another reviewer can apply the anchors without asking the author what each label means. Revise ambiguous wording before using scores for a release decision.

Report the task set and review method with any future scores. Do not describe this proposed rubric as a published benchmark or claim results that were never collected.

Continue with test set design, text legibility review, and reference fidelity review for criteria requiring more specific evidence.

Continue with quality and testing

• generation model regression checklist: Maintain a release scorecard with real evidence
• generation prompt revision diff: Review the change as a field-level hypothesis
• AI image color drift test: Control viewing conditions before judging color
• Browse all quality and testing guides