Does AI judge content or clout? A causal test of authority bias
Hold the title and abstract constant, change only author prestige, venue and citation counts, and eight models change their recommendation — while claiming they do not.
Key numbers
- Design: authority metadata — author prestige, venue and citation counts — varied across three counterfactual conditions (original, flipped, boosted) with titles and abstracts held constant.
- Subjects: eight LLMs (five open-weight, three frontier closed-weight) in an in-context, single-turn, top-1 recommendation setting.
- Findings: authority bias is substantial and directional, varies markedly across models, and is only partially addressable through prompt-level debiasing.
- The say-do gap: debiasing instructions suppress mentions of authority far faster than authority-driven flips, so surface auditing systematically underestimates behavioural bias.
Method and sample
arXiv paper “Authority Bias in Conversational Search Engines for Academic Paper Recommendation” (2026-08-31): counterfactual metadata conditions applied across eight models to causally test whether recommendations follow content or authority signals, with prompt-level debiasing also tested.
What it means for brands
For brands this means good content does not win by itself: authority signals such as institution, author and citation counts already move outcomes. It also warns that telling a model to ignore authority changes what it says, not what it chooses.
How SourceSight handles it
SourceSight's off-site list is effectively an authority-signal plan — presence in directories, reviews and authoritative media affects whether a model shortlists you — and per-engine reporting tracks that movement over snapshots.
Limits
The experiments cover academic paper recommendation rather than commercial recommendation; model versions move quickly, so the absolute size of the bias changes over time.
References
Run it on your own project
What the same prompts return across 10 engines is only knowable by running them.
Related research
What Chinese generative search actually cites: 8.3% brand selection, and app vs web cite different sources
Across 160,860 citation-level records, the same platform surfaced systematically different source sets on its app and web interfaces.
Sources · 2026-10-06Incumbent advantage: with identical specs, a well-known brand is recommended 100% of the time
Three experiments on skincare: identical specifications produce near-total dominance for known brands — and less than a +0.1-star edge for a competitor breaks it.