What fine-tuning means—and when to use it
Fine-tuning large language models means updating a pretrained model with examples that teach it a repeatable task, tone, format, or domain pattern. It is not the same as adding documents to a search index or writing a longer prompt. Fine-tuning changes model behaviour; retrieval-augmented generation (RAG) supplies current facts at query time.
For most Indian startups, the right first step is to define the failure you want to fix. Fine-tuning is useful when the model consistently needs to:
- Follow a strict output schema for workflows or APIs.
- Use a brand voice across high-volume interactions.
- Classify tickets, documents, or leads consistently.
- Handle domain-specific phrasing, code-switching, or transliterated language.
- Produce concise summaries or structured decisions from recurring inputs.
Use RAG instead when answers depend on changing policies, catalogues, prices, regulations, or internal documents. A hybrid system—fine-tuned behaviour plus retrieval—often provides the best balance. Teams new to the stack can review these best practices for fine-tuning LLMs on custom data before committing engineering budget.
Start with a measurable business case
Do not fine-tune simply because an open model is available. Establish a baseline using a representative test set and connect improvement to a business metric. Examples include first-contact resolution, document-processing time, successful task completion, escalation rate, or cost per resolved interaction.
Write a short model specification covering:
- Users and languages: Include English, Hindi, regional languages, transliteration, and code-switching where they occur in production.
- Task boundaries: State what the model must answer, refuse, escalate, or leave blank.
- Quality targets: Set thresholds for factuality, format compliance, latency, and safety.
- Operational constraints: Define cloud, on-premise, data residency, hardware, and monthly inference limits.
- Human fallback: Identify when a support agent, clinician, lawyer, or financial professional must review the output.
For voice-heavy products, text fine-tuning is only one part of the system. Speech recognition, turn-taking, latency, and call routing can dominate the user experience; compare the model work with the practical guidance on voice agent services for Indian businesses.
Build training data that reflects India
A small, clean dataset usually beats a large, noisy one. Collect examples from actual workflows, then remove personally identifiable information, secrets, irrelevant boilerplate, and contradictory labels. Preserve the variation that matters: accents, spelling differences, Roman-script Indic text, local names, currency formats, dates, and mixed-language queries.
Each example should show the desired response, not merely the source document. For a support assistant, include the customer message, relevant context, ideal answer, escalation decision, and—where applicable—the exact tool or API action. Add difficult cases deliberately: ambiguous requests, missing information, abusive language, prompt injection, and questions outside the product’s scope.
Indic-language projects require extra care because benchmark performance can conceal poor real-world behaviour. Test dialects, script variation, low-resource languages, and transliteration separately. The low-resource Indic NLP builder’s guide is a useful companion for teams preparing multilingual data.
Choose the least expensive adaptation method
Start with prompting and RAG, then test parameter-efficient fine-tuning before full model updates. Common options include:
- Supervised fine-tuning: Trains on labelled input-output examples and is suitable for formats, classifications, and repeatable task behaviour.
- LoRA or QLoRA: Updates a small set of adapter parameters, reducing memory and making experiments easier to store and roll back.
- Preference optimisation: Uses ranked responses to improve helpfulness, tone, or policy adherence after supervised training.
- Full fine-tuning: Updates most or all model weights. It may help at larger scale but requires more compute, stronger evaluation, and tighter version control.
Select an open-weight model or hosted model based on licence terms, commercial use, language coverage, context length, hardware requirements, and data-handling commitments. A cheaper model that fails on Hindi or Marathi may cost more after human review and escalation are included.
Evaluate before deployment
Split data into training, validation, and a locked test set. Keep near-duplicates out of the test set, and include a production-like “hard set” that the model has never seen. Automated scores can check classification and formatting, but human review remains essential for factuality, politeness, bias, and harmful advice.
Track at least:
- Task accuracy and schema compliance.
- Hallucination and unsupported-claim rates.
- Performance by language, script, geography, and user segment.
- Refusal quality and escalation accuracy.
- Latency, tokens per request, GPU utilisation, and total cost.
- Regression results against the current production model.
Use blinded human evaluation with a clear rubric. For regulated areas such as lending, insurance, health, and legal services, preserve input, output, model version, retrieval context, and reviewer decisions so incidents can be investigated.
Deploy with safeguards and cost controls
A fine-tuned model should not be the only control layer. Put authentication, rate limits, input filtering, tool permissions, retrieval citations, output validation, and human escalation around it. Never allow a model to approve credit, prescribe treatment, issue legal conclusions, or execute sensitive transactions without the controls required by the use case.
Quantisation, batching, caching, shorter prompts, and smaller specialist models can reduce inference costs. Deploy adapters separately when possible so you can roll back a behaviour change without replacing the base model. Run a shadow test or limited pilot, compare it with the baseline, and expand only after monitoring shows stable results.
For sensitive Indian customer data, document consent, retention, access controls, vendor terms, and cross-border processing. Maintain a data lineage record for every training source, including licences and removal requests. Treat prompts, outputs, annotations, and evaluation sets as production data—not disposable experiment files.
A practical 90-day implementation plan
Weeks 1–2: Define the use case, baseline, risk classification, success metrics, and fallback path.
Weeks 3–4: Audit data, remove sensitive information, create annotation guidelines, and assemble a multilingual test set.
Weeks 5–7: Compare prompting, RAG, and LoRA/QLoRA on the same evaluation harness. Record quality, latency, and cost.
Weeks 8–9: Red-team the strongest approach for leakage, prompt injection, bias, unsafe advice, and language-specific failures.
Weeks 10–12: Pilot with a limited user group, monitor business metrics, collect reviewer feedback, and decide whether to scale, retrain, or revert.
The builder’s decision rule
Fine-tuning is worthwhile when a stable, high-volume behaviour gap remains after good prompting and retrieval, and when you have enough representative examples to measure improvement. It is a poor shortcut for missing knowledge, weak product requirements, or unreliable source data.
Indian startups can move faster by treating the model as one component in a tested product system. Begin with a narrow workflow, preserve user and expert feedback, evaluate languages independently, and optimise for total business value rather than benchmark scores alone. Teams exploring the broader ecosystem can also study Indian open-source AI developer projects for reusable tooling and deployment ideas.
Apply for AI Grants India
Funding can help cover data preparation, evaluation infrastructure, compute, and responsible deployment. Apply to AI Grants India to explore support for your AI project and turn a validated prototype into a production-ready system.