Apply for AI Grants India

What you are building—and what fine-tuning cannot solve

Fine-tuning a Telugu model for the Indian legal domain is not simply a matter of adding legal vocabulary to a general-purpose model. The system must handle Telugu script, English legal phrases, transliterated names, court-specific formatting, citations, dates, sections of legislation, and regional variations without inventing authorities or changing the meaning of a document.

The safest objective is a narrow, auditable capability: for example, extracting case metadata, classifying petitions, retrieving relevant passages, translating with human review, or producing a grounded summary. Do not begin with an open-ended legal chatbot. If the product must support broader research or drafting, pair the model with retrieval, citation checks, access controls, and lawyer review. A practical companion is this guide to AI legal document automation in India, especially for mapping model outputs to real workflows.

Define the task before choosing the model

Write a task specification that answers five questions:

• Input: Telugu judgments, bilingual pleadings, scanned PDFs, contracts, or user questions?
• Output: classification, named-entity extraction, translation, summarisation, retrieval, or structured fields?
• Users: advocates, court staff, paralegals, legal-aid organisations, or citizens?
• Risk level: can an incorrect answer cause missed deadlines, a faulty filing, or denial of legal assistance?
• Success condition: what must the system get right, and what errors are unacceptable?

For instance, “summarise a Telugu judgment” is too broad. A usable specification might require a 300-word summary containing the court, date, parties, issues, holding, and cited provisions, with every material claim linked to a source passage. For legal research, retrieval accuracy and citation faithfulness matter more than fluent prose.

Select a Telugu-capable base model

Compare models on language coverage, licence terms, context length, tokenizer behaviour, inference cost, and the availability of commercial support. A multilingual model may provide stronger general reasoning, while an Indic-focused or open-weight model may tokenise Telugu more efficiently. Test candidate models on representative legal text rather than relying on benchmark claims.

Create a small diagnostic set containing:

• Telugu-only passages and Telugu-English mixed text.
• Section numbers, case citations, abbreviations, names, and dates.
• OCR errors and common spelling variants.
• Long sentences typical of judgments and statutes.
• Questions requiring the model to distinguish facts, arguments, and holdings.

Inspect token counts and truncation. If a judgment is routinely cut off, increasing the context window or using hierarchical retrieval may be more useful than fine-tuning. Teams evaluating open-weight options can also review Indian open-source AI developer projects for relevant tooling and implementation patterns.

Build a lawful, representative dataset

Data quality is the main determinant of legal-model performance. Potential sources include legislation, government publications, publicly accessible judgments, authorised legal databases, synthetic templates, and consented examples from partner organisations. Confirm the right to collect, process, store, and redistribute every source. Public availability does not automatically grant unrestricted commercial or training rights.

Keep a dataset register with the source, licence or permission, collection date, language, document type, court or authority, and permitted uses. Remove personal data that is not necessary for the task. Mask phone numbers, addresses, identity documents, medical details, and other sensitive information unless the use case specifically requires them and appropriate safeguards are in place.

Do not randomly split pages from the same judgment across training and test sets. That creates leakage and inflated scores. Split by case, matter, source, and time period where possible. Maintain separate training, validation, and locked test sets, with difficult examples deliberately included.

Prepare Telugu legal text carefully

Legal documents often arrive as scanned PDFs, poorly encoded text, or bilingual files. Use OCR only as a first pass and validate it against the original page. Preserve section numbers, paragraph boundaries, headings, tables, footnotes, and citations. A corrupted “Section 302” can change the meaning of an entire example.

Normalise Unicode consistently, but retain the original text for audit. Build a terminology table covering Telugu equivalents, English terms, abbreviations, transliterations, and known variants. Label entities such as courts, statutes, sections, parties, dates, locations, case numbers, and legal outcomes. For summarisation or question answering, store the supporting passage and document identifier alongside each example.

A useful dataset record includes:

• document_id and source metadata;
• original Telugu text and any translated version;
• task instruction and expected output;
• evidence span or citation;
• annotator identity, confidence, and review status;
• privacy, licence, and retention labels.

Use at least two legally literate reviewers for high-risk labels. Resolve disagreements with written guidelines instead of silently choosing the majority answer.

Choose supervised fine-tuning, retrieval, or both

Fine-tuning teaches behaviour, formatting, and task patterns. It does not reliably update changing law or guarantee that the model knows every relevant judgment. For current legal research, use retrieval-augmented generation over a controlled corpus, with document dates, jurisdiction filters, passage citations, and an explicit “insufficient evidence” response.

A sensible architecture is:

1. OCR and clean the source document.
2. Detect language and classify the legal task.
3. Retrieve authoritative Telugu, English, or bilingual passages.
4. Ask the model to answer only from retrieved evidence.
5. Validate citations, sections, names, and dates.
6. Route high-risk outputs to a qualified reviewer.

For training configuration and parameter-efficient methods such as adapters or LoRA, follow best practices for fine-tuning LLMs on custom data. Parameter-efficient tuning can reduce GPU memory and make experiments easier to roll back, but it still requires careful data governance and evaluation.

Fine-tune with reproducibility and restraint

Start with a baseline: the untuned model plus a prompt and, if applicable, a retrieval system. Then run a small pilot using a fixed data version. Record the model checkpoint, tokenizer, hyperparameters, random seed, hardware, training duration, and evaluation results.

Use a low learning rate, early stopping, and a held-out validation set. Compare full fine-tuning with parameter-efficient tuning. Monitor both training and validation loss; a widening gap usually signals overfitting. Do not optimise solely for fluency. A model that sounds authoritative while changing legal meaning is worse than a cautious model that asks for evidence.

Include negative and abstention examples: unclear scans, missing context, conflicting provisions, unknown case names, and questions outside the corpus. Teach the system to state when it cannot verify an answer.

Evaluate legal and Telugu performance separately

Report results by task, document type, jurisdiction, language mix, and difficulty. Useful measures include:

• extraction precision, recall, and F1;
• retrieval recall at relevant ranks;
• citation and evidence accuracy;
• factual consistency of summaries;
• translation adequacy reviewed by Telugu legal experts;
• abstention rate on unanswerable questions;
• latency, cost, and failure rate in production conditions.

Use expert review for semantic errors that automatic metrics miss. Ask reviewers to flag omitted exceptions, altered negation, incorrect section numbers, fabricated authorities, privacy leaks, and unjustified confidence. Test adversarially with spelling variants, OCR noise, code-switching, long documents, and misleading prompts.

Deploy with legal safeguards

Treat the model as decision support, not a lawyer or judicial authority. Display the source passages used, document dates, confidence limitations, and a clear review requirement. Maintain role-based access, encryption, audit logs, retention controls, incident reporting, and a process for deleting or correcting training data.

Before launch, run a pilot with a small group of practitioners. Measure time saved and error patterns, not just user satisfaction. Establish a rollback path for every model and retrieval-index update. If the system handles compliance workflows, connect the technical controls to a documented process; the guide on automating legal compliance with AI in India offers a useful framing.

A practical 30-day build plan

• Days 1–5: define one task, users, risk boundaries, and evaluation criteria.
• Days 6–12: secure data permissions, create the dataset register, and prepare a reviewed sample.
• Days 13–18: benchmark two or three base models and build a retrieval baseline.
• Days 19–24: run parameter-efficient fine-tuning with versioned experiments.
• Days 25–27: conduct Telugu legal-expert evaluation and adversarial testing.
• Days 28–30: pilot with safeguards, document failures, and decide whether to iterate or stop.

The strongest Telugu legal systems are usually not the largest. They are the ones with clean, permitted data; explicit evidence links; realistic Telugu evaluation; narrow product claims; and a reliable human-review path.