Developers adapting large language models for Indian languages are making progress on reducing the ‘tokenizer tax’ that has previously slowed inference and increased costs. Two papers, submitted this week, detail an in-place tokenizer expansion that cut tokens per word by 38.6% for eight Indian languages. This work builds on prior efforts, including those discussed in our August 2026 coverage of new tokenizers addressing an 8x ‘tokenizer tax’ and September 2026 reporting on Devanagari tokenizer advancements. Concurrently, a separate research effort has produced a 2.5-billion-parameter Tamil-first model trained from scratch. This model offers an alternative approach to building language-specific AI.
Adapting Existing Models for Indic Languages
Two related papers, both submitted this week, describe efforts to adapt NVIDIA’s Nemotron-3-Nano-30B-A3B for improved performance in Indian languages.
The first, submitted by Avinash Benki, Prasanjith Pasunuti, Sutanshu Saksena Raj, Parijat Chatterjee, Nayanjit Sarkar, and Daksh Maan, details an in-place tokenizer expansion. This method adapted Nemotron-3-Nano-30B-A3B for eight Indian languages without altering its backbone architecture. The vocabulary expanded from 131,072 to 163,072 entries. Using continued byte-pair encoding and AdaptMix rebalancing, the researchers reduced tokens per word by 38.6%. New input and output rows were initialized by averaging their original constituent rows, then trained on 90% Indic and 10% English text before the backbone was updated. Continual pretraining proceeded over approximately 1.27 trillion tokens, through broad training, high-quality annealing, and long-context training. The annealed checkpoint showed a gain of 12.9 points on the Indic MILU benchmark but incurred a loss of 3.4 points on English MMLU. The authors also reported that direct token addition performed worse than continued byte-pair encoding, and AdamW weight decay altered rows intended to remain fixed.
A second paper, submitted by Sutanshu Saksena Raj, Prasanjith Pasunuti, Parijat Chatterjee, Daksh Maan, Purusharth Malik, Nayanjit Sarkar, and Avinash Benki, introduces Gnani-Evon-v3.3-30B-A3B. This is a long-context hybrid Mamba-Transformer mixture-of-experts model designed for reasoning, instruction following, and tool use across English and eight Indian languages. It starts from the same NVIDIA Nemotron-3-Nano-30B-A3B checkpoint, retaining its 52-layer backbone but modifying the vocabulary, training data, and post-training objectives. An identical in-place expansion from 131,072 to 163,072 tokens also reduced the mean Indian-language fertility (tokens per word) by 38.6%. New embedding rows were initialized from their upstream decompositions and warmed before the backbone was updated. Continual pretraining added broad English-Indic data, high-quality annealing, and direct training at a 131,072-token sequence length. Post-training, the model’s performance improved through reinforcement learning across four verifiable task environments, boosting mathematics and tasks described as ‘gradua’.
Agaram 2.5B: A Tamil-First Language Model
In a different approach to Indic language AI, Ramchand Kumaresan, Madhusudhanan G, and Kaushik Kezendirane have developed Agaram 2.5B, a 2.5-billion-parameter Tamil-English language model, as first reported by arXiv. This model was trained from scratch on approximately 50 billion tokens, with Tamil comprising 58% of the dataset, alongside English, mathematics, code, and parallel text. Training occurred on one node of eight H100 GPUs over about six days.
Agaram 2.5B uses a new tokenizer that preserves line breaks. This improves on the team’s initial model. Supervised fine-tuning was performed using 134,428 conversations, where an open-weight teacher model generated the replies, followed by three short targeted rounds. This turned Agaram 2.5B into a chat model capable of answering in Tamil script, romanized Tamil (Tanglish), and English. It also supports tool calling. On 25 tool prompts whose schemas appeared in training, the model picked the correct tool 23 times and provided exactly the expected arguments 13 times.
For translation tasks, Agaram 2.5B outperforms Sarvam-1, a 2-billion-parameter Indic base model. On all six FLORES and IN22 directions, Agaram 2.5B achieved a chrF score of 54.3 when translating into Tamil and 55.2 when translating into English. Sarvam-1, for comparison, scored 49.2 and 44.2 respectively.
The papers collectively show two strategies for advancing Indian language AI. One focuses on adapting large, existing models by improving tokenizer efficiency and continually pretraining them on vast Indic datasets, accepting some trade-offs in general English performance for gains in specific Indic benchmarks. The other shows that training a smaller, language-specific model from the ground up, like Agaram 2.5B for Tamil, yields strong performance in translation and specific tasks for that target language. The Agaram model’s superior translation scores against a general Indic model show the benefits of a focused, language-first training regimen.
Compiled by Launch91 Desk from the sources linked above. More about Launch91.
