Generative production has a specific evaluation problem: the best possible output of a competent operator and the best possible output of a beginner are much closer together than they used to be. Anybody can produce a beautiful frame. The distance between a studio and a person with a subscription shows up at volume, under a deadline, with a client’s legal team reading.
These are the questions we would ask, and they are the questions we expect to be asked.
On measurement
- What is your acceptance rate on the hardest shot type in this brief? A studio that has run this before will have a number and will volunteer that it is lower for legible packaging type than for environment plates. "It depends" without a follow-up means they have not logged it.
- What is your cost per accepted asset, and does it include the rejected renders? If the number excludes rejects it is a cost per generation, which flatters everybody and predicts nothing.
- Show me a run log from a real campaign. Not a case study — the log. Timestamps, attempt counts, what failed and why.
On consistency
- What do you lock, and where is it written? The answer should be a file, not a description of good intentions. Palette, light behaviour, materials, wardrobe rules, lens family.
- How do you keep a face or a product identical across forty shots? Reference conditioning, a master plate, a trained identity — any of them is a real answer. "Careful prompting" is not.
- What happens when the model you use is deprecated? A studio that has thought about it will describe assets that survive the change. One that has not will look surprised.
On control
- What are your gates, and what do they reject automatically? A pipeline without automated rejection is a person watching everything, which is a cost you are paying.
- What is your spend ceiling per asset and what happens when a shot hits it? "The run halts and a person decides" is the answer. "We keep going until it works" is a blank cheque with your name on it.
- Who signs off, and at what points? Human review at named checkpoints, at any volume, or the volume is the product.
On rights and disclosure
- What does your consent file contain for a synthetic presenter? If the answer does not mention a derivative-training clause, the release is a photography release and it does not cover this.
- Who decides whether this campaign has to be labelled, and when? The correct answer is "at brief stage, to the strictest of platform policy, advertising code and applicable law". Deciding at delivery is how a campaign gets pulled.
- What do we own at the end, and in what form? Frames, plates, project files and the lock file. If the deliverable requires a specific model or account to still exist, you do not own the work.
The answers that should end the conversation
| WHAT THEY SAY | WHAT IT MEANS |
|---|---|
| "We don’t really track acceptance rate." | The quote is a guess and the overrun is yours. |
| "Consistency is just about good prompting." | They have not run a sequence longer than a few shots. |
| "We can do unlimited revisions." | There is no ceiling, so there is no budget control, so there is a number later. |
| "Disclosure is the client’s side." | True in part, and a studio that will not advise on it has not read the rules. |
| "We use the latest model." | Not a capability. Everyone has the same access. |
| "The AI does most of it." | The generation is the cheap part. If that is the pitch, the expensive parts are not being done. |
What a good answer sounds like
Specific, quantified, and slightly unflattering. A studio that tells you the packaging shots will need three times the renders of the environment plates, that the acceptance rate on faces at that shot size is lower than they would like, and that they will halt at a ceiling and come back to you, is describing a process they have actually run.
Studios that have not run one describe outcomes. It is a reliable tell, and it costs nothing to check.
Twenty-five scored questions across locks, gates, measurement, compliance and handover. Use it on a supplier or on yourself.
THE READINESS SCORECARD →