High-stakes AI advice shifts with conversation context — not with who you are
For legal, medical and financial questions, a single conversation history is enough to give different users different advice. The strongest predictor is the topic.
Key numbers
- Background: in high-stakes settings such as legal, medical and financial advice, even a single conversation history is enough to drive outcome differences between users, and prior work shows socio-demographic groups receive advice of differing quality.
- The correction this study adds: models actually struggle to infer user socio-demographics from one conversation, and group-level disparities are minimal in magnitude.
- What predicts the advice is the conversation topic — compared against emotion and readability — which partly acts as a proxy for socio-demographic groups and often shifts advice in unpredictable directions.
- Conclusion: conversational context is itself a variable that changes high-stakes advice and needs to be evaluated on its own.
Method and sample
arXiv paper “Topics as Proxies for Sociodemographics: How Conversational Context Affects LLM Answers” (2026-06-01): across high-stakes legal, medical and financial advice scenarios, the study compares user socio-demographics against a range of psycho-linguistic features (topic, emotion, readability) for predictive power over generated advice, and tests whether models can recover user profiles from a single conversation.
What it means for brands
In these industries AI does not deliver an objective answer: it delivers an answer shaped by how the question was asked and what came before. The same applies to how brands are described — which is why customer asking patterns, not a single verdict, are what to measure.
How SourceSight handles it
SourceSight groups prompts by how customers actually ask (pricing, risk, credentials and compliance, which to choose) and reports mentions and position per group, with one project per market so a single prompt never stands in for all customers.
Limits
The study runs in controlled tasks and conversational conditions rather than auditing real financial or legal consultations; it shows context influences output, not that any particular phrasing yields better advice.
References
Run it on your own project
What the same prompts return across 10 engines is only knowable by running them.
Related research
Cross-category recommendations: +36% add-to-cart on item pages, +15% on cart pages
Recommending general merchandise to grocery shoppers — a milk frother with the milk — has long been hard; LLM retrieval plus real-time cart context moved production A/B metrics.
Industries · 2026-10-07Home services: 22% of homeowners already use AI to find a provider, and 80% of businesses don't know how to be found
The local services funnel is moving upstream: before a homeowner opens a website or calls, AI has already synthesised reviews, reputation and service descriptions into one answer.