Ask “which should I buy” and ChatGPT vs Gemini share 0% of their sources in 76.7% of cases
AI shopping advice is neither stable nor consistent: ask again and the products change, and the API shows something different from the product.
Key numbers
- Audit: 2,528 real commercial-advice queries, with 1,536 product-answer responses collected from ChatGPT (product and API) and Google Gemini (product and API).
- For the same query, the ChatGPT and Gemini interfaces share only 5.4% of cited domains on average — and in 76.7% of comparisons they share no domain at all.
- Tone differs too: ChatGPT states a first-person product preference (“I'd pick…”) in 79% of product-recommending responses, versus 7% for Gemini.
- APIs show a different picture from their interfaces (mean domain overlap 12.0% for ChatGPT, 14.8% for Gemini, with different source layers), and the products recommended change across repeated requests.
Method and sample
arXiv paper “If I Had to Buy Just ONE: Galaxy S26 Ultra”: Auditing AI-Generated Product Recommendations (2026-09-16): 2,528 curated real commercial-advice queries and 1,536 evaluated product responses, comparing recommendation stance, cited-domain overlap and repeat stability across engines and access paths (interface vs API).
What it means for brands
Being recommended in one engine says nothing about another, and what you measure through an API is not what users see in the product. AI also speaks in the first person — “I'd pick” reads far more like neutral advice than a list of options does.
How SourceSight handles it
SourceSight measures through the same surfaces consumers use, reports mentions and citations per engine instead of one blended number, and repeats prompts so a single fluctuation is not mistaken for a finding.
Limits
The sample is consumer-electronics commercial advice and covers only ChatGPT and Gemini; the paper proposes an audit method rather than a universal conclusion across industries and engines.
References
Run it on your own project
What the same prompts return across 10 engines is only knowable by running them.
Related research
What 2 million AI citations show: prompt-content match matters most, checklists matter less
Once you control for the domain, the usual AEO checklist (FAQ blocks, structured data, Core Web Vitals) reverses or collapses to zero — domain authority and prompt alignment dominate.
How engines work · 2026-10-07Whose side is the AI shopping assistant on? Change its role and the penalty for sponsored results disappears
The same sponsored listings get treated very differently once the prompt names the platform as the agent's principal — disclosure rules built for humans do not carry over.