AI career advice is generic: local context covered only 35.4% of the time across ten countries
The “described correctly” problem applies to education too: models know the tech stack but frequently skip local ecosystems, language requirements and national policy.
Key numbers
- Design: standardised prompts tested across six models (ChatGPT-4, DeepSeek, Gemini, Claude 3.5, Llama 3, Mistral) describing entry-level computing careers in ten African countries — 60 responses analysed against the Computing Curricula 2020 framework.
- Technical competencies were covered consistently (cloud computing, programming), while non-technical competencies diverged, especially ethics and responsible AI use.
- Recognition of country-specific factors varied widely — local technology ecosystems, language requirements and national policies averaged only 35.4% contextual awareness.
- Scores: Llama 4.47/5 and DeepSeek 4.25/5 beat ChatGPT-4 at 3.90/5 and Claude at 3.46/5, while open-source Mistral scored just 0.00/4 on context — openness alone does not guarantee local relevance.
Method and sample
arXiv paper “Evaluating LLMs for Career Guidance: Comparative Analysis of Computing Competency Recommendations Across Ten African Countries” (2025-10-20): 60 responses from six models to standardised prompts about entry-level computing careers across ten countries, content-analysed with the Computing Curricula 2020 framework and contextual dimensions.
What it means for brands
When AI describes an education or training provider, the risk is not only omission but getting flattened into a generic version — local strengths, language capability, policy and certification all erased. The more explicitly those facts appear on your site and in directories, the more likely AI gets them right.
How SourceSight handles it
For education projects SourceSight builds prompts from student questions (which to choose, is it right for me, pricing, credentials) and runs one project per market, because the concerns around the same programme differ by country.
Limits
The study covers computing careers across ten African countries with 60 responses; the scores come from the authors' coding framework and do not transfer to other countries, industries or the latest model versions.
References
Run it on your own project
What the same prompts return across 10 engines is only knowable by running them.
Related research
Cross-category recommendations: +36% add-to-cart on item pages, +15% on cart pages
Recommending general merchandise to grocery shoppers — a milk frother with the milk — has long been hard; LLM retrieval plus real-time cart context moved production A/B metrics.
Industries · 2026-10-07Home services: 22% of homeowners already use AI to find a provider, and 80% of businesses don't know how to be found
The local services funnel is moving upstream: before a homeowner opens a website or calls, AI has already synthesised reviews, reputation and service descriptions into one answer.