Global AI giants need sovereign data partners for inclusion

This article is authored by Supriya Paul, CEO, Josh Talks.

A woman in a village in Bihar tries to seek medical advice for her child. She speaks in Bhojpuri, blending in Hindi, describing symptoms the way she normally would, without structuring her thoughts for a machine. The system responds, but the advice does not fully align with what she is trying to explain. She repeats the query, adjusts her phrasing, slows down her speech, hoping clarity will improve the outcome. It does not. Eventually, she disengages and turns to someone local.

AI

Nothing in this interaction appears broken. The system responds each time. It processes input, generates output, and completes the loop it was designed for. Yet, it fails to deliver a dependable outcome. This gap, between functioning and actually being useful, is increasingly shaping how AI is experienced across real-world contexts.

AI systems are now widely accessible across sectors such as health care, financial services, and public welfare. India alone has seen rapid adoption of voice-led interfaces, with regional-language internet users accounting for nearly 75% of the user base and voice search usage growing significantly in recent years. At the same time, voice-first behaviour is becoming deeply embedded, India processes billions of voice interactions daily across platforms like messaging and search.

However, increased access has not translated into consistent usability. Users are engaging with AI systems, but the interaction often breaks down at the level of interpretation. Inputs are shaped by context, habit, and linguistic fluidity, while systems attempt to map them to structured patterns. The result is a mismatch that is not always visible immediately but becomes evident over repeated interactions, especially in high-stakes scenarios.

The limitation is not simply about model capability; it is about representation. In India, language is inherently fluid. Code-switching between Hindi and English, or across multiple regional languages, is not occasional but dominant. In fact, large-scale analysis of deployed voice systems shows that Hindi-English code-switching is the most common interaction pattern, rather than pure single-language usage.

At the same time, India’s linguistic complexity is immense, with 22 official languages and over 19,500 dialects shaping how people communicate. Yet, most AI training datasets continue to rely on standardised, text-heavy, and urban-centric sources. These datasets capture language in its most structured form, not in its lived, spoken reality. This creates a fundamental disconnect. Models are trained on clarity and consistency, while users operate within variation and context.

The way AI systems are evaluated reinforces this disconnect. Benchmarks are typically built on clean, well-structured inputs, where systems perform with high accuracy. But real-world deployment introduces variability that is rarely captured in these evaluation frameworks.

Evidence from India’s emerging AI benchmarks highlights this gap. A large-scale evaluation across 15 languages and over 35,000 speakers found that global multilingual AI systems struggle significantly with Indian accents, dialects, and code-switched speech. Even within a single language like Hindi, dialects such as Bhojpuri remain challenging despite being spoken by tens of millions.

This divergence between controlled testing and real-world performance means that systems can appear robust in development environments while behaving inconsistently in practice.

India’s diversity amplifies these limitations in ways that are difficult to overlook. Variation exists not only across languages but within them, across regions, communities, and contexts. Pronunciation, phrasing, and meaning shift continuously.

From a technical standpoint, this variability directly affects performance. For instance, even advanced speech recognition systems show word error rates of 8–12% in clean Hindi audio, rising to 15–20% in noisy, real-world conditions, with further drops in accuracy for regional dialects. Similar gaps persist across other Indian languages, particularly when models encounter domain-specific vocabulary or informal speech patterns.

As AI systems expand into non-urban and multilingual environments, these inconsistencies become more pronounced, not because the systems are failing outright, but because they are operating outside the narrow conditions they were optimised for.

Addressing this gap requires more than incremental model improvements. It requires a shift in how data is sourced, governed, and evaluated. Sovereign data partners are emerging as a critical part of this transition.

These partners, often national platforms, sectoral data aggregators, or regulated institutions, operate within local legal and cultural frameworks. Their role is to curate datasets that reflect real-world usage across languages, sectors, and environments, while ensuring compliance with evolving data governance requirements.

Equally important is their role in evaluation. By incorporating real-world variability into testing frameworks, they enable a more accurate understanding of system performance. This shifts the focus from theoretical accuracy to operational reliability.

India’s own ecosystem is beginning to reflect this shift, with locally developed AI systems and benchmarks increasingly influencing how performance is defined and measured in multilingual contexts.

For global AI companies, the implications are both strategic and immediate. As deployment expands into markets like India, Africa, and Southeast Europe, systems trained on generic datasets encounter limitations in interpretation and consistency.

This affects product performance, user trust, and regulatory alignment simultaneously. Governments are placing greater emphasis on data localisation and accountability, while enterprises are increasingly factoring in data governance and country-of-origin considerations when selecting AI partners.

Sovereign data partnerships offer a way to navigate this complexity. They allow global AI firms to retain their core strengths, model development, infrastructure, and scale, while embedding those capabilities within locally governed data ecosystems. This enables models to be fine-tuned and evaluated in environments that reflect actual usage, rather than idealised conditions.

The next phase of AI adoption will be shaped not by how widely systems are deployed, but by how effectively they function across diverse contexts. Scale increases reach, but it does not automatically improve relevance. Inclusion depends on whether systems are trained on data that reflects real-world diversity and evaluated against conditions that mirror actual usage. Sovereign data partnerships provide a structured way to integrate both.

As AI becomes more deeply embedded in everyday decision-making, the ability to operate within complexity, across languages, environments, and user behaviours, will define its usefulness. Systems that cannot adapt to this complexity will continue to function, but their role will remain limited. For global AI companies, the question is no longer whether they can scale. It is whether they can scale in a way that remains usable, reliable, and contextually grounded across the markets they aim to serve.

(The views expressed are personal)

This article is authored by Supriya Paul, CEO, Josh Talks.