How to Fine-Tune a Chatbot on Hugging Face with Non-PII Indian Data

What fine-tuning can—and cannot—solve

Fine-tuning adapts a pretrained language model to a defined conversational style, domain, or response format. It can help a chatbot answer recurring questions in Indian languages, follow a support workflow, or use terminology familiar to a specific audience. It does not automatically make the model factually reliable, compliant, or current.

For changing information—such as government schemes, prices, admission deadlines, or product catalogues—use retrieval-augmented generation (RAG) or an API alongside fine-tuning. If you are deciding between these approaches, the principles in best practices for fine-tuning LLMs on custom data are a useful starting point.

Define the use case and success criteria

Write a short model specification before collecting data. Include:

Audience: for example, Indian SaaS customers, students, patients, or internal support staff.
Languages and scripts: Hindi in Devanagari, Hinglish in Roman script, Tamil, Marathi, Bengali, or another target mix.
Tasks: FAQ answering, intent classification, troubleshooting, form filling, or escalation.
Boundaries: topics the model must refuse, transfer to a human, or answer only from approved sources.
Success metrics: task completion, groundedness, language quality, refusal accuracy, latency, and cost.

Create a held-out test set before training. Do not use it to tune prompts or inspect repeated examples during development. A realistic test set should include spelling variation, code-switching, short messages, abusive inputs, ambiguous requests, and regional terminology.

Build a genuinely non-PII dataset

“Public” does not mean “safe to train on”. A public support forum, WhatsApp export, survey, or scraped page may contain names, phone numbers, email addresses, addresses, Aadhaar or PAN references, account numbers, medical details, or indirect identifiers.

Prefer data that was intentionally created for training, such as synthetic conversations, licensed datasets, publicly released government material, or de-identified internal FAQs. Keep a data card recording the source, licence, collection date, permitted use, language, transformations, and known limitations.

Practical de-identification workflow

1. Inventory fields and sources. Separate user text, agent text, metadata, timestamps, and labels.
2. Detect direct identifiers. Use pattern matching for phone numbers, email addresses, URLs, bank details, government ID formats, and common address patterns.
3. Review indirect identifiers. A rare job title combined with a locality, date, and incident may identify someone even without a name.
4. Replace rather than merely delete. Convert values to tokens such as <PHONE>, <EMAIL>, <CITY>, or <ORDER_ID> so the model learns the workflow without memorising the value.
5. Run human review. Sample records from every source and language; automated detection will miss context and transliterated identifiers.
6. Remove duplicates and memorised text. Near-duplicate conversations can inflate evaluation scores and increase leakage risk.

Under India’s Digital Personal Data Protection framework, assess whether your collection and processing have a lawful basis, appropriate notices, retention controls, and security measures. Non-PII training data still needs licensing and provenance checks.

Choose a model and training method

Select a causal language model that supports your target languages, hardware, and licence. Check the model card for training data, known biases, context length, tokenizer coverage, commercial restrictions, and safety limitations. A model that performs well in English may handle Indian scripts poorly if its tokenizer splits words into excessive fragments.

For most teams, parameter-efficient fine-tuning is the practical option. LoRA or QLoRA trains a small set of adapter weights instead of updating the entire model, reducing GPU memory, cost, and deployment complexity. Full fine-tuning is usually justified only with a large, high-quality corpus and a strong evaluation pipeline. Teams exploring local development can also review Indian open-source AI developer projects for relevant implementation patterns.

Format conversations for supervised fine-tuning

Use a consistent conversational schema. For example:

{"messages":[
  {"role":"system","content":"You are a support assistant. Do not request sensitive personal information."},
  {"role":"user","content":"Mera order kab aayega?"},
  {"role":"assistant","content":"Order status dekhne ke liye order ID share karein. Phone number ya OTP share na karein."}
]}

Keep system instructions stable and avoid placing private operational secrets in them. Include positive examples, refusal examples, escalation examples, and corrections of common misunderstandings. Balance languages, intents, regions, and response lengths; otherwise the model may overfit to the most frequent language or style.

Split data by conversation or source—not by individual turns—to prevent leakage. A useful first split is 80% training, 10% validation, and 10% test, but source-based and time-based splits are often more realistic.

Fine-tune with Hugging Face

Install a current, compatible stack and pin versions for reproducibility:

pip install -U transformers datasets peft trl accelerate bitsandbytes evaluate

Load and inspect the dataset before training:

from datasets import load_dataset

dataset = load_dataset("json", data_files={
    "train": "data/train.jsonl",
    "validation": "data/validation.jsonl"
})
print(dataset)

For chat models, apply the tokenizer’s chat template rather than manually concatenating arbitrary markers. With TRL’s supervised fine-tuning trainer, configure a small learning rate, a short initial run, gradient accumulation, checkpointing, and evaluation at regular intervals. Use LoRA through PEFT and enable 4-bit loading only after verifying that quantisation does not damage your target-language quality.

Start with a pilot run on a representative subset. Track training loss and validation loss, but do not treat a falling loss as proof of a useful chatbot. Save the base model, adapter, tokenizer, dataset version, configuration, random seed, and hardware details.

Evaluate Indian-language quality and safety

Automated scores are useful for regression testing, not as the sole decision criterion. Evaluate with a fixed, human-reviewed set covering:

Accuracy and groundedness against approved answers.
Hindi-English code-switching, transliteration, spelling variation, and regional vocabulary.
Refusal of requests for OTPs, passwords, Aadhaar numbers, financial credentials, and unnecessary personal data.
Toxicity, stereotypes, caste and religious bias, and unsafe advice.
Consistent escalation when confidence is low or the request is out of scope.
Prompt injection, jailbreaks, data extraction, and memorisation of training examples.

Use native speakers or trained bilingual reviewers, with a rubric and adjudication process. Compare the fine-tuned model with the base model and a strong prompt-only baseline. For customer-facing systems, connect model evaluation to operational outcomes such as resolution rate and human handoff quality; automated user feedback categorization for Indian SaaS can help turn support interactions into structured improvement signals without feeding raw conversations back into training.

Deploy with safeguards

Keep the adapter and base model versioned separately, and expose the model behind an authenticated service rather than sharing unrestricted inference access. Add input and output filtering, rate limits, audit logs with sensitive content minimised, and a human escalation path. Never ask users to send passwords, OTPs, full card numbers, or government ID documents through the chatbot.

For changing factual content, retrieve answers from approved documents and show source links where appropriate. Red-team before launch, monitor language-specific failures, and maintain a rollback path. If the chatbot supports professional or regulated workflows, treat it as an assistant—not an autonomous decision-maker. A private architecture may be more appropriate for sensitive teams; see how to build a private AI chatbot for lawyers for an example of stricter controls.

A practical launch checklist

Confirm dataset licences, provenance, retention, and de-identification evidence.
Test tokenizer coverage for every target script and language.
Freeze a leakage-free evaluation set.
Compare LoRA, prompting, and RAG against the same test cases.
Review safety, bias, refusal, and escalation behaviour with native speakers.
Record model, adapter, code, data, and dependency versions.
Start with a limited pilot, monitor failures, and retrain only on approved, redacted examples.

Fine-tuning is most valuable when it teaches a repeatable behaviour or communication style. For Indian deployments, disciplined data governance, multilingual testing, and retrieval-backed answers matter as much as the training command itself.

Apply for AI Grants India