Apply for AI Grants India

Fine-tuning can make a language model more useful for a narrow domain such as India’s e-way bill system—but only when the training data is accurate, consistently formatted, and evaluated against current rules. This guide explains how to fine-tune a model using e-way bill FAQ data on Hugging Face, from dataset preparation to deployment and maintenance.

An FAQ fine-tune is best suited to a model that must answer recurring questions in a consistent style. It is not a substitute for live validation against the GST e-way bill portal, official notifications, or a retrieval system for frequently changing thresholds and exemptions. For high-stakes workflows, combine fine-tuning with the principles in Data Veracity Infrastructure for High-Stakes AI.

Choose the right adaptation method

Before training, decide whether fine-tuning is actually necessary:

• Prompting works when the model already understands the subject and needs a clear answer format.
• Retrieval-augmented generation (RAG) is preferable when answers depend on changing notifications, state-specific rules, or source citations.
• Supervised fine-tuning (SFT) helps the model learn tone, structure, terminology, and repeatable responses from curated examples.
• Parameter-efficient fine-tuning, such as LoRA or QLoRA, reduces GPU memory and is usually the practical starting point for an FAQ assistant.

For a production GST support tool, a strong pattern is to fine-tune response behaviour while retrieving the latest source documents at inference time. Review Best Practices for Fine-Tuning LLMs on Custom Data before selecting a base model or training configuration.

Prepare a trustworthy FAQ dataset

Start with authoritative material: official GST and e-way bill guidance, approved internal support answers, and carefully reviewed user questions. Do not scrape or publish personal data, invoice identifiers, vehicle numbers, GSTINs, phone numbers, or unredacted business records.

Create examples that reflect how users actually ask questions. Include spelling variations, Hindi-English phrasing, abbreviations, and incomplete queries, but keep the target answer precise. Useful fields include:

• question: the user’s query
• answer: the approved response
• source: notification, portal guidance, or internal reference
• topic: generation, cancellation, validity, transporter, or blocking
• language: English, Hindi, or another supported language
• reviewed_on: the date of the latest review
• requires_live_check: whether the answer must be verified before use

A conversational JSONL format works well with Hugging Face:

{"messages":[{"role":"user","content":"When is an e-way bill required?"},{"role":"assistant","content":"An e-way bill may be required for the movement of goods subject to applicable GST rules and thresholds. Verify the current requirement on the official e-way bill portal before dispatch."}]}

Avoid contradictory answers in the same training split. Deduplicate near-identical questions, remove obsolete guidance, and separate training, validation, and test examples by question intent, not just by random rows. Otherwise, nearly identical FAQs can leak into evaluation and produce an inflated score.

Load and inspect the data on Hugging Face

Install the core libraries in a reproducible environment:

pip install -U transformers datasets peft trl accelerate evaluate

Load a local JSONL file and inspect its schema:

from datasets import load_dataset

files = {"train": "data/eway_faq_train.jsonl", "validation": "data/eway_faq_valid.jsonl"}
dataset = load_dataset("json", data_files=files)
print(dataset)
print(dataset["train"][0])

If you publish the dataset to the Hugging Face Hub, add a dataset card describing provenance, licence, redaction steps, intended use, known gaps, and the date of the last legal review. Treat the Hub repository as a governed asset: restrict write access, enable versioning, and do not place secrets in notebooks or configuration files.

Select a base model and training strategy

Choose an instruction-tuned model that fits your language and deployment constraints. For English-only support, a compact open model may be sufficient. For Hindi or code-mixed queries, test multilingual capability rather than assuming it from the model name. Teams building regional-language assistants can also compare approaches in Fine-Tuning Llama for Indian Regional Languages.

Use LoRA or QLoRA when GPU resources are limited. A typical starting point is a low learning rate, two to four epochs, a small rank such as 8 or 16, and early stopping based on validation loss and task quality. These are starting points—not universal settings. Overtraining a small FAQ set can make the model repeat memorised wording, mishandle unfamiliar questions, or become overconfident.

A simplified TRL-style workflow looks like this:

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

model_id = "your-instruction-model"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

lora = LoraConfig(
    r=16, lora_alpha=32, lora_dropout=0.05,
    target_modules=["q_proj", "v_proj"],
    task_type="CAUSAL_LM"
)

args = SFTConfig(
    output_dir="eway-faq-adapter",
    learning_rate=2e-5,
    num_train_epochs=3,
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
    eval_strategy="steps",
    logging_steps=10,
    save_steps=100,
    report_to="none",
)

trainer = SFTTrainer(
    model=model,
    args=args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
    peft_config=lora,
    processing_class=tokenizer,
)
trainer.train()
trainer.save_model()

Library APIs change, so pin tested versions and confirm the current TRL and Transformers documentation before running this code. Use a GPU for practical training; a smaller quantised model can reduce memory requirements.

Evaluate for correctness, not just loss

Validation loss alone cannot tell you whether an answer is safe or legally current. Build a test set containing:

• common generation and cancellation questions;
• validity and distance-related scenarios;
• transporter and vehicle-update cases;
• ambiguous, incomplete, and adversarial questions;
• Hindi, Hinglish, and spelling variants;
• questions whose answer has changed since earlier guidance;
• requests for confidential invoice or taxpayer information.

Score responses for factual accuracy, completeness, citation or source behaviour, refusal quality, language consistency, and calibration. Require the model to say when it cannot verify a current rule. Compare the fine-tuned model with the untuned baseline and a RAG baseline. Have GST practitioners review a representative sample, especially answers involving penalties, exemptions, thresholds, or blocked documents.

Deploy with guardrails

Do not let a fine-tuned model silently approve transport documents or provide definitive legal advice. Put a retrieval layer and source links behind time-sensitive answers, log model and dataset versions, and route uncertain cases to a human. Mask sensitive inputs before logging and define retention rules for chat transcripts.

For mobile or low-cost field deployments, quantisation and adapter merging can help, but measure latency and accuracy on the actual device. The broader trade-offs are covered in AI Model Optimization for Mobile Devices: 2026 Deployment Guide. If your audience includes first-time internet users, prioritise short answers, low-bandwidth performance, and multilingual fallback; see Building AI Apps for the Next Billion Users in India.

Maintain the model after launch

E-way bill guidance changes, and user questions will expose gaps that were absent from the original dataset. Maintain a review queue for unanswered or low-confidence queries, label corrections by topic, and release new dataset versions rather than overwriting the old one. Re-run the same fixed evaluation set after every data, prompt, adapter, or retrieval change.

A reliable release checklist includes:

• verified source documents and review dates;
• redacted and licensed training data;
• separate train, validation, and test sets;
• multilingual and adversarial evaluations;
• retrieval for changeable rules;
• human escalation for uncertain answers;
• model, adapter, dataset, and dependency versioning.

The goal is not merely to make a model sound knowledgeable. It is to build an assistant that answers routine e-way bill questions clearly, cites or retrieves the right authority, and knows when a current GST check is required.