National bias across languages: ask in another language, get another answer
In university applications, travel and relocation tasks, model recommendations shift with the language — local language bias is prevalent.
Key numbers
- Scope: three personalised-advice scenarios — university applications, travel and relocation — tested across multiple languages for decision-making tasks.
- Quantification: bias in model-generated scores was measured, along with how demographic factors and reasoning strategies such as chain-of-thought prompting affect bias patterns.
- Finding one: local language bias is prevalent across different tasks.
- Finding two: GPT-4 and Sonnet reduced bias toward English-speaking countries compared with GPT-3.5, but neither achieved robust multilingual alignment — with direct implications for multilingual AI in education and similar settings.
Method and sample
arXiv paper “Assessing Agentic Large Language Models in Multilingual National Bias” (2025-02-25): multilingual prompts across three decision scenarios, quantifying national bias in model-generated scores and testing the influence of demographic factors and chain-of-thought reasoning.
What it means for brands
If you operate across markets, differences in how the same brand is described in different languages are not random — they are a systematic model-level bias. Multilingual markets must be asked and measured separately, then checked for consistency of the same fact.
How SourceSight handles it
SourceSight runs one project per market and language, records mentions, position and sources per language, and groups multi-market projects so three-language differences can be compared directly.
Limits
The public abstract reports directional bias analysis without per-language figures; the finding is directional, and the size of the effect has to be verified with your own project data.
References
Run it on your own project
What the same prompts return across 10 engines is only knowable by running them.
Related research
What Chinese generative search actually cites: 8.3% brand selection, and app vs web cite different sources
Across 160,860 citation-level records, the same platform surfaced systematically different source sets on its app and web interfaces.
Sources · 2026-10-06Incumbent advantage: with identical specs, a well-known brand is recommended 100% of the time
Three experiments on skincare: identical specifications produce near-total dominance for known brands — and less than a +0.1-star edge for a competitor breaks it.