Fine-tuning starts with a pre-trained transformer and adapts it to a narrower job: classifying support tickets, extracting fields from invoices, answering in a consistent format, or handling Indian-language and code-switched conversations. You are not teaching a model everything from scratch. You are changing its behaviour using examples that represent the task you need it to perform.
For most teams in 2026, the sensible path is to begin with prompting and retrieval, establish a measurable baseline, and fine-tune only when the baseline fails for a repeatable reason. A smaller open model adapted to your workflow can reduce latency, inference cost, and vendor dependence—but only if the data and evaluation process are disciplined.
Decide whether fine-tuning is the right tool
Fine-tuning is useful when the model must consistently learn a behaviour, format, tone, or decision procedure. Examples include:
- Returning structured JSON with a fixed schema.
- Following a domain-specific classification policy.
- Producing concise answers in a brand voice.
- Handling recurring instruction patterns across many requests.
- Improving performance on Hindi, Tamil, Bengali, Hinglish, or other target-language workflows.
Fine-tuning is usually the wrong first choice for information that changes frequently. Product catalogues, current regulations, account records, and internal documents are better supplied through retrieval or tools. A practical architecture often combines retrieval for facts with fine-tuning for response behaviour. Before training, compare the options in best practices for fine-tuning LLMs on custom data.
Run a baseline first. Create a fixed evaluation set, test a strong prompt, and record accuracy, format compliance, latency, and cost. If prompt changes or a small retrieval layer solve the problem, training may add unnecessary complexity.
Choose the model and adaptation method
Select a model according to task, language coverage, licence, context length, serving stack, and hardware, not parameter count alone. Encoder models such as BERT-style architectures suit classification and token-level labelling. Decoder-only models suit instruction following, generation, and conversational applications. Check commercial-use terms, attribution requirements, acceptable-use restrictions, and whether the model has meaningful coverage of your target Indian languages.
There are three broad approaches:
- Full fine-tuning: Updates every model parameter. It can deliver strong results but requires substantial memory, storage, training time, and operational discipline. It is rarely the best starting point for a small team.
- LoRA: Adds small trainable low-rank matrices to selected layers while keeping the base weights frozen. The resulting adapter is inexpensive to train and easy to version.
- QLoRA: Loads the base model in low-bit precision and trains LoRA adapters. This reduces memory use and is often the most practical route for experimentation on a single rented GPU.
Adapter methods are not automatically better. They still depend on task quality, sequence length, target modules, and evaluation. Start with LoRA or QLoRA, then consider full fine-tuning only when experiments show a clear need.
Build a reliable training dataset
Your dataset is the product specification expressed as examples. Each record should show the input, the desired output, and—where relevant—the rules that determine a correct answer. For instruction tuning, JSONL is common, but the exact conversation template must match the base model and training library.
A robust preparation workflow includes:
- Define the task contract: Specify allowed labels, output fields, language, tone, refusal behaviour, and citation expectations.
- Remove leakage: Keep near-duplicates and documents from the same case out of both training and test sets.
- Clean carefully: Fix malformed records, inconsistent punctuation, broken Unicode, accidental prompt injections, and personally identifiable information.
- Balance examples: Include common cases, edge cases, ambiguous requests, and safe refusal examples. Do not let a large majority class hide poor minority performance.
- Preserve Indian context: Retain relevant currency formats, names, addresses, local units, regional scripts, and code-switching. Do not “clean” away the very signals the model must learn.
- Split by source or entity: For legal, healthcare, finance, or customer data, ensure the same person, document, organisation, or template cannot appear across splits.
A few hundred excellent examples can teach formatting or tone. Domain coverage and factual reliability generally require more. Do not use synthetic data without review: generated examples can amplify errors and create an artificial style that fails in production.
Prepare the training run
A typical open-source stack uses transformers, datasets, peft, trl, accelerate, and, where compatible, bitsandbytes. Pin package versions and save the complete configuration so a result can be reproduced.
Before launching, confirm:
1. The model’s chat or prompt template is correct.
2. The tokenizer matches the base model exactly.
3. Long records are truncated deliberately rather than silently.
4. Training, validation, and test data are separate.
5. Checkpoints and logs are written to durable storage.
6. Sensitive data is not sent to an unapproved cloud region or third-party service.
For a first QLoRA experiment, use a modest rank such as 8–32, a conservative learning rate, a small number of epochs, and gradient accumulation to achieve a workable effective batch size. These are starting points, not universal settings. Sequence length can dominate memory; shorten examples or pack compatible samples before buying a larger GPU.
A single 16–24 GB GPU may support QLoRA for smaller models, depending on sequence length and batch configuration. Larger models, longer contexts, and full fine-tuning quickly require more memory. Cloud costs in India vary by provider and availability, so estimate the full run—including storage, failed experiments, evaluation, and inference—before committing.
Evaluate beyond training loss
Training loss only tells you how well the model fits the examples. Maintain a held-out test set and report metrics that match the product:
- Exact match or F1 for extraction and classification.
- JSON validity and field-level accuracy for structured output.
- Human ratings for helpfulness, tone, and language quality.
- Hallucination, refusal, and unsafe-answer rates.
- Performance by language, script, customer segment, and edge case.
- Latency, throughput, and cost per request.
For Indian-language applications, test spelling variation, transliteration, mixed scripts, regional terminology, and code-switching. Have native or expert reviewers assess high-impact outputs. If your project involves visual inputs, compare text-only training with suitable open-source vision-language models for Indian languages.
Watch for overfitting: training loss may fall while validation quality worsens. Reduce epochs, improve the split, add harder examples, or lower the learning rate. Keep the base model, adapter, dataset version, tokenizer, configuration, and evaluation report together. This makes rollback possible and supports auditability.
Deploy adapters safely
You can serve a LoRA adapter alongside a compatible base model or merge it into a model artefact. Keeping adapters separate is often better during experimentation because it simplifies rollback and allows multiple task-specific versions. Validate merged and unmerged outputs; quantisation or serving conversions can change behaviour.
For production, test batching, concurrent requests, streaming, timeouts, observability, and fallback behaviour. Frameworks such as vLLM or specialised inference servers may improve throughput, but benchmark with your actual prompts and sequence lengths. Apply access controls, redact logs, encrypt datasets, and document whether customer data can be used for future training.
A practical first project
Start with a narrow, measurable task such as classifying support requests or extracting fields from GST invoices. Build a small reviewed dataset, establish a prompt baseline, train a QLoRA adapter, and compare it against the baseline on an untouched test set. This is a more useful learning path than attempting to fine-tune a general chatbot immediately. You can also develop supporting skills through machine learning portfolio projects for beginners in India and study deployment patterns in how to deploy deep learning models on GKE.
Fine-tuning is an engineering loop, not a one-time command: define the failure, curate representative data, run a controlled experiment, evaluate by slice, and deploy only when the improvement is material. For Indian founders, the strongest advantage often comes from proprietary, consented data and careful local evaluation—not from choosing the largest available model.