How to design a retest you can trust: five moves so noise does not pass as progress
AI visibility is a distribution, not a point: fix the prompts, repeat the runs, add controls, paraphrase, validate by hand — or you cannot attribute the change.
Key numbers
- Why repeats are mandatory: one GEO measurement study concludes answers vary across runs, prompts and time, making one-off observations unreliable and visibility a distribution; a brand-recommendation evaluation study likewise requires repeated sampling because the same query yields different brands and rankings.
- The reproducible protocol from the survey has five moves: repeated measurement, paraphrased prompts, controls, human validation, and accounting for multi-actor interference.
- Only two levers survive scrutiny as reliably reproducible: topical relevance and context position; generic heuristics transfer poorly and competition erodes individual gains.
- State the boundary condition: already-retrieved content can causally change its citation or use, but no technique has been shown to produce a stable long-term effect on organic discoverability — so one good week does not mean you became easier to find.
Method and sample
Synthesis of three studies: the critical GEO survey (arXiv:2607.14035, 45 studies plus a reproducible protocol), the GEO measurement methodology paper (arXiv:2604.07585, visibility as a distribution) and the brand-recommendation evaluation framework (arXiv:2609.16304, repeated sampling for recommendation probability and prominence).
What it means for brands
Optimisation without a retest design is indistinguishable from doing nothing: you cannot tell whether the change worked or the model simply moved. Fix the prompt set and the number of runs, leave a control untouched, and noise separates from real movement.
How SourceSight handles it
SourceSight's retest practice follows those five moves: the same prompt set is re-run in full each audit, results are reported per engine, and two snapshots are compared directly — with mention rate, position and sources side by side instead of one blended score.
Limits
The protocol comes from academic recommendations and runs into budget and time limits in practice (more repeats cost more); volatility differs by engine, so the number of repeats needs calibrating on your own project.
References
Run it on your own project
What the same prompts return across 10 engines is only knowable by running them.
Related research
Same engine, different surface, different answer: test app, web and API separately
On the same platform and the same questions, app and web cite systematically different source sets — and the API shows something else again.
Methods · 2026-10-07What GEO methods actually hold up: 45 studies and a cold shower
The widely cited “40% visibility lift” comes with a condition: the source was already in a fixed context. It proves neither discoverability nor durable traffic.