Multilingual language models often struggle to recognize the same person across different scripts, a problem identified in new research submitted to arXiv on October 22, 2026. One paper found that the Qwen2.5-7B model’s accuracy for matching same-person identities dropped from 35.1% when names were in Latin script to just 8.5% when they were presented in Devanagari script, even with Marathi prompts. This suggests a fundamental difficulty in handling script variations for entity recognition. That is a core task for many AI applications.

The paper, titled “Same Entity, Different Script: Diagnosing Cross-Script Entity Matching Failures in Language Models,” was authored by Athira Gopal, Patil Prathamesh Pravin, Ankush Kadu, Arpit Jain, and Ashwanth Krishnan. The researchers constructed a dataset of 601 audited identity pairs, each with attested Devanagari and Latin forms, maintaining fixed questions and relations. While the model’s accuracy for identifying different people remained high, above 98%, its ability to link the same person across scripts deteriorated sharply. The problem extended beyond Devanagari. A replication of the study on 374 audited Malayalam pairs, built from the same entities, showed a similar effect. For Malayalam, same-person accuracy fell to 5.3%, approximately half the rate observed for Devanagari. Despite this low accuracy, the models still ranked matching names above non-matching names with an AUROC of 0.925.

The findings reveal a significant hurdle for building reliable AI systems in multilingual, multi-script environments like India. Applications such as customer relationship management, knowledge base population, or even identity verification depend on accurate entity resolution. If a model cannot reliably identify “Ramesh” in Latin script as the same “रमेश” in Devanagari, its utility for processing real-world data across various sources diminishes. The authors also explored whether reduced tokenization fragmentation could bridge this gap, but found it did not reliably improve the cross-script matching performance. This suggests the issue runs deeper than tokenization, affecting how models represent and relate entities across diverse orthographies. Previous research from this publication, including pieces on tokenizer efficiency and models for language identification, has consistently pointed to the complexities arising from India’s script diversity. The current study provides specific performance numbers that quantify this challenge for identity matching.

Addressing Code-Switching in Speech Recognition

Another paper, “Synthetic-to-Real Code-Switched ASR: A Spontaneous Nepali-English Podcast Benchmark,” authored by Firoj Paudel and submitted on October 22, 2026, tackles the related problem of code-switched speech recognition. Code-switching, the practice of alternating between two or more languages within a single conversation, is common in many multilingual communities, including those in India and neighboring regions. For Nepali-English code-switched conversational speech, automatic speech recognition (ASR) systems face significant challenges, particularly from script confusion—the inappropriate mixing of Devanagari and Latin orthography in transcriptions—and elevated error rates at language switch points.

This work’s main contribution is a new benchmark dataset of 13,044 spontaneous podcast utterances. This dataset totals 15.45 hours of audio, split into 10,214 test utterances and 2,830 development utterances. The dataset was meticulously audited and verified to remove common pitfalls like machine translation leakages and acoustic loops. This ensures its quality for evaluating ASR systems. Creating such high-quality human-annotated code-switched audio is expensive and time-consuming, which has historically limited research in this area.

To address the data scarcity, the research investigates a two-stage synthetic-to-real adaptation framework. Stage 1 involves supervised fine-tuning (SFT) on 0.95 hours of speech synthesized using a multi-speaker, dual-engine text-to-speech (TTS) pipeline. Stage 2 further explores Group Relative Policy Optimization (GRPO), incorporating both transcription and script rewards. The study found that the synthetic SFT stage provided the primary cross-domain transfer, effectively adapting models to spontaneous human speech. This result suggests a practical pathway for improving code-switched ASR performance by using cheaper, synthetically generated data to augment limited human-annotated resources. Creating reliable benchmarks for challenging linguistic phenomena, much like those for bias and conversational AI previously covered here, is essential for improving AI capabilities across diverse language use cases.

Both papers highlight the specialized challenges of building AI for real-world use in multilingual contexts. Whether identifying people across scripts or transcribing code-switched conversations, general-purpose multilingual models often fall short, requiring targeted diagnostics, bespoke datasets, and innovative training strategies. The entity matching study specifically quantified a performance gap of over 25 percentage points for Qwen2.5-7B between Latin and Devanagari script for Marathi prompts, while the ASR work provided a 15.45-hour benchmark to tackle Nepali-English code-switching.

Sources

Compiled by Launch91 Desk from the sources linked above. More about Launch91.