Apply for AI Grants India

Start with the support problem, not the model

The goal is not to make a model “speak Malayalam” in the abstract. It is to resolve a defined set of customer-support tasks reliably: order status, refunds, warranty questions, account access, service availability, and escalation to a human agent. A small model can be faster and cheaper than a large general-purpose model, but only when the scope, knowledge sources, and hand-off rules are explicit.

Decide first whether you need classification, retrieval, generation, or a combination:

• Intent classification routes a message to the right workflow.
• Entity extraction identifies order IDs, dates, product names, locations, and amounts.
• Retrieval-augmented generation (RAG) supplies current policy and product information without retraining the model for every update.
• Supervised fine-tuning teaches the model your preferred response format, tone, Malayalam terminology, and escalation behaviour.

For a low-resource language project, review this builder’s guide to low-resource Indic NLP before selecting a base model. It covers data, tokenisation, script variation, and evaluation issues that generic multilingual tutorials often miss.

Select a model and an efficient training method

Choose a compact instruction-tuned model with a licence that permits your intended commercial use. Compare models on Malayalam comprehension, Malayalam generation, context length, inference cost, quantisation support, and available tokenizer quality—not just parameter count. A model that splits Malayalam words inefficiently may require more tokens and deliver worse latency than a slightly larger alternative.

For most teams, parameter-efficient fine-tuning is the practical starting point:

• LoRA or QLoRA updates a small set of adapter parameters rather than all model weights.
• 4-bit quantisation reduces GPU memory requirements during training and inference, subject to quality testing.
• Instruction tuning is appropriate for producing structured support replies.
• Classification fine-tuning is preferable when the model only needs to route tickets or detect intent.

Follow established best practices for fine-tuning LLMs on custom data, especially around held-out evaluation, data leakage, learning rates, and checkpoint selection. A Colab GPU may be enough for a pilot; production training may require a rented NVIDIA GPU or a managed training service.

Build a Malayalam-first dataset

Historical support data is useful only after it has been made safe and consistent. Remove phone numbers, email addresses, account identifiers, addresses, payment details, and internal notes. Obtain the necessary permissions and document how data is collected, retained, and used. For India-facing deployments, involve legal and security teams early rather than treating privacy as a post-training task.

Create examples that represent how customers actually write:

• Malayalam script, including spelling variation and colloquial phrasing.
• Malayalam-English code-mixing, such as product names and technical terms in Latin script.
• Transliterated Malayalam typed in English characters.
• Short, incomplete, emotional, and misspelled messages.
• Voice-transcription errors if the product accepts calls or voice notes.
• Regional vocabulary and realistic support scenarios from Kerala and Malayalam-speaking users elsewhere.

Each example should include the customer message, the intended action, the approved answer or answer template, and an escalation label where relevant. Include negative examples: requests for confidential information, unsupported refunds, abusive language, prompt injection, and questions the system must not answer.

Keep training, validation, and test sets separated by conversation or customer, not by random message. Otherwise, near-duplicate tickets can inflate results. Maintain a Malayalam terminology sheet for product names, policy terms, transliterations, and words that must remain in English.

Design the response contract

A support model should not improvise business policy. Give it a strict response contract, for example:

1. Identify the customer’s intent and required entities.
2. Retrieve the relevant policy or account information.
3. Answer in the requested language and channel style.
4. State the next step clearly.
5. Ask only for information that is necessary and safe to collect.
6. Escalate when confidence is low or the request is sensitive.

Use structured outputs internally, such as JSON containing intent, language, entities, answer, citations, and escalate. Render only the approved customer-facing fields. This makes it easier to connect the model to ticketing, CRM, order, and authentication systems.

Fine-tuning should teach behaviour and phrasing; it should not be the only source of changing facts. Put refund rules, prices, service areas, and operating hours in a versioned knowledge base. For multilingual customer journeys, a voice layer may be useful; compare the model workflow with guidance on voice agent software for small businesses, particularly if Malayalam support will include phone calls.

Train with LoRA or QLoRA

A typical training pipeline uses a Malayalam-capable causal language model, a chat template, and supervised examples containing user instructions and ideal assistant responses. Keep the first run conservative:

• Use a small learning rate and monitor validation loss.
• Start with one to three epochs; more training can memorise phrasing and policies.
• Truncate only after measuring how much context is being lost.
• Mask loss on system and user turns when the training framework supports it.
• Save checkpoints and compare them on a fixed Malayalam test suite.
• Track the exact base model, adapter, dataset version, tokenizer, quantisation settings, and prompt template.

Do not judge success by training loss alone. A model can produce fluent Malayalam while giving an incorrect refund instruction. Keep retrieval, tools, permissions, and escalation outside the model wherever possible.

Evaluate Malayalam support quality

Build an evaluation set with both automated metrics and human review. Accuracy and F1 are useful for intent classification, but they are insufficient for open-ended answers. Measure:

• Intent and entity accuracy.
• Correct use of current policy and retrieved sources.
• Malayalam fluency, naturalness, and code-mixed comprehension.
• Factuality and resistance to unsupported claims.
• Appropriate refusal and escalation behaviour.
• Privacy leakage and prompt-injection resistance.
• First-contact resolution, average handling time, and human correction rate.
• Latency, memory use, and cost per conversation.

Use bilingual reviewers who understand customer-support policy, not only general language fluency. Score responses against a rubric and record the exact failure category. Include adversarial tests such as contradictory policy documents, ambiguous transliteration, multiple issues in one message, and requests to reveal internal instructions.

Deploy with retrieval, controls, and monitoring

Begin with agent-assist or a limited FAQ workflow before enabling autonomous resolution. Add authentication before exposing account-specific information, enforce tool permissions, redact logs, and define a hard escalation path. The model should say it does not know or transfer the case when evidence is missing—not invent an answer in Malayalam.

Quantise the model only after establishing a quality baseline. Benchmark on the hardware you will actually use, including CPU or edge devices if relevant. The 2026 guide to AI model optimisation for mobile devices is useful when support must run under tight memory, connectivity, or latency constraints.

Monitor production by language and intent, not just aggregate metrics. Look for rising fallback rates, repeated customer messages, low-confidence clusters, new spelling patterns, policy-related errors, and differences between Malayalam, English, and code-mixed traffic. Route corrected conversations into a review queue; retrain only after deduplication, approval, and regression testing.

A practical pilot plan

A credible first release can follow this sequence:

• Define 10–20 high-volume intents and explicit escalation rules.
• Assemble and redact a few thousand representative conversations, supplemented by carefully authored examples.
• Establish a retrieval-backed baseline before fine-tuning.
• Fine-tune a small adapter and compare it with the baseline on a locked test set.
• Run human review with Malayalam-speaking agents.
• Launch in agent-assist mode, then expand only where quality and safety targets are met.

The best Malayalam support system is rarely the model with the most parameters. It is the system with representative data, current sources, disciplined evaluation, safe tool access, and a clear path to a human. Indian founders building this kind of infrastructure can apply for AI Grants India to support experimentation, evaluation, and deployment.