India needs language technology that works beyond English and Hindi, across mixed-language conversations, local scripts, dialects, noisy speech transcripts, and low-connectivity environments. Small language models (SLMs) are a practical route: they cost less to train and serve, can run on private infrastructure or devices, and are easier to adapt to a specific domain than a large general-purpose model.
This guide explains how to build an SLM for Indian languages with a focus on measurable utility rather than language-count marketing. Start with a narrow use case—such as customer support, public-service search, education, claims processing, or voice-agent assistance—then expand language coverage after the first workflow is reliable. For product context, see this guide to building AI apps for the next billion users in India.
Define the product and language scope
Do not begin by attempting to support every Indian language. Define:
- Users and setting: urban or rural, smartphone or feature phone, online or intermittent connectivity.
- Tasks: classification, retrieval, summarisation, question answering, translation, structured extraction, or dialogue.
- Language modes: native script, Romanised text, code-mixed input, and speech transcripts.
- Quality threshold: what error rate is acceptable for each task, and when must the system defer to a human?
- First languages: choose based on user demand, available data, and business or public-service impact—not only speaker count.
A restaurant assistant, for example, may need Hindi-English and regional-language code-switching, while a benefits assistant may prioritise factual retrieval, named entities, and safe refusal behaviour. Voice products should be designed jointly with speech recognition and text generation; the voice-agent architecture guide covers the surrounding system.
Build a trustworthy Indic data pipeline
Data quality usually matters more than adding parameters. Combine several sources, while documenting licence, provenance, language, domain, date, and consent status for every dataset.
Useful sources include:
- Government and public-service documents with clear reuse terms.
- Licensed news, books, educational material, and customer-support conversations.
- Community contributions reviewed by native speakers.
- Synthetic examples used for coverage, not as a replacement for real language.
- Speech transcripts and query logs collected with explicit consent and privacy controls.
Filter aggressively. Remove duplicated pages, boilerplate, spam, personally identifiable information, unsafe content that is not required for the task, and machine-translated text with no quality review. Deduplicate at document and near-paragraph level so repeated web content does not dominate training.
For low-resource languages, use language experts to create a small, high-quality benchmark before scaling the corpus. The low-resource Indic NLP builder’s guide is useful when working with limited labelled data, orthographic variation, and regional language resources.
Design tokenisation for Indian scripts
Tokenisation can determine whether a compact model is genuinely efficient. A tokenizer trained mainly on English may split Indic words into excessive fragments, increasing sequence length and reducing useful context.
Evaluate candidate tokenizers on:
- Fertility: average tokens per word in each target language.
- Coverage of common words, names, numbers, punctuation, and emojis.
- Romanised and code-mixed text.
- Script variants, spelling variation, and Unicode normalisation.
- Memory and latency at the target context length.
SentencePiece or byte-level approaches are common starting points, but benchmark them on your own data. Preserve meaningful boundaries where possible, and include representative samples from every target language in tokenizer training. Avoid aggressive stemming or lowercasing: these can damage meaning in scripts and languages where case, suffixes, or diacritics carry information.
Choose an efficient model strategy
For most teams, the fastest path is continued pretraining or supervised fine-tuning of an existing open model, rather than training from zero. Select a base model with a compatible licence, architecture, context length, tokenizer, and hardware footprint.
Practical options include:
- A compact decoder model for generation and instruction following.
- An encoder model for classification, search, and information extraction.
- A multilingual encoder-decoder model for translation and transformation tasks.
- A domain-adapted model paired with retrieval for changing factual content.
Use parameter-efficient fine-tuning such as LoRA or adapters when compute or data is limited. Quantisation can reduce serving cost, but test quality after quantisation in every language; degradation is often uneven. Distillation from a stronger teacher can transfer task behaviour into a smaller student, provided the synthetic data is checked by native speakers and domain reviewers.
Train for transfer without erasing languages
Balance batches by language and task rather than allowing high-resource languages to overwhelm training. Track loss and downstream performance separately for each language. Useful methods include:
- Continued pretraining on clean, domain-relevant multilingual text.
- Instruction tuning with parallel task formats across languages.
- Translation and transliteration augmentation for underrepresented forms.
- Code-mixed examples that reflect real user input.
- Contrastive or alignment objectives for cross-language retrieval.
- Replay data from weaker languages during later training to prevent forgetting.
Synthetic translation is valuable for bootstrapping, but it can amplify errors, unnatural phrasing, and majority-language assumptions. Keep human-written evaluation sets and periodically sample outputs for expert review.
Evaluate usefulness, safety, and equity
A single multilingual average hides failures. Build a test matrix by language, script, task, domain, and input type. Measure:
- Task accuracy, macro-F1, exact match, or structured-output validity.
- Retrieval recall and answer faithfulness for knowledge applications.
- Translation quality with human adequacy and fluency ratings, not BLEU alone.
- Hallucination, refusal, toxicity, privacy leakage, and harmful advice.
- Latency, memory use, throughput, and cost per request.
- Performance on code-mixed, Romanised, misspelled, and dialectal inputs.
Create language-specific red-team sets for names, locations, caste and community references, health, finance, and government schemes. Have native speakers score whether an answer is understandable, culturally appropriate, and actionable. If a model cannot answer reliably, retrieval, constrained generation, or human escalation may be safer than further scaling.
Deploy for Indian operating conditions
Benchmark on the hardware your users will actually have. A model that performs well on a data-centre GPU may be unusable on a low-cost Android device or a regional call centre.
Consider:
- 4-bit or 8-bit quantisation with language-wise regression tests.
- Distilled models for on-device classification or autocomplete.
- Hybrid edge-cloud routing based on privacy, latency, and complexity.
- Caching and batching for predictable workloads.
- Offline or store-and-forward operation where connectivity is unreliable.
- Observability for language, latency, fallback rate, and user corrections.
Keep personal data out of training by default. Encrypt logs, minimise retention, obtain consent for feedback collection, and provide deletion and correction pathways. For production systems, separate model improvement data from operational data and document who can access each.
A practical build sequence
1. Select one high-value workflow and two or three languages.
2. Create a documented dataset and a small native-speaker benchmark.
3. Compare tokenizers and one or two compact base models.
4. Establish a retrieval or rules baseline before fine-tuning.
5. Fine-tune with balanced multilingual batches and parameter-efficient methods.
6. Evaluate by language, task, safety, latency, and cost.
7. Pilot with real users, capturing corrections with consent.
8. Improve the weakest language or failure mode before adding another language.
The strongest Indian SLM projects are not necessarily the largest. They are the ones that make a specific workflow faster, clearer, and safer for people using the language they are most comfortable with.