Apply for AI Grants India

Intent extraction is the process of identifying what a person wants from a message, voice command, search query, or support conversation. It is a core capability in conversational AI: a system must recognise whether someone wants to check a balance, cancel an order, report fraud, or speak to an agent before it can take the right action.

For Indian builders, the challenge is rarely just choosing a model. Users switch between English and Indic languages, spell words phonetically, omit context, and use product-specific terms. A production-grade system therefore needs a clear intent taxonomy, entity extraction, dialogue context, confidence thresholds, and an evaluation process that reflects real traffic.

What intent extraction includes

A useful intent pipeline usually produces more than one label:

• Intent: The user’s goal, such as track_order, reset_pin, or apply_for_loan.
• Entities: Details needed to complete the task, such as an order ID, city, date, amount, or account type.
• Context: Earlier turns, user history, channel, and workflow state.
• Confidence: The model’s estimate that its prediction is reliable.
• Fallback status: Whether the system should ask a clarifying question, route to a human, or refuse to act.

Consider: “Kal Bengaluru se Pune wali train cancel kar do.” The intent is cancellation, while the date, origin, destination, and possibly a booking reference are entities. “Cancel” could also refer to a hotel, subscription, or payment, so the surrounding conversation matters.

Intent extraction differs from sentiment analysis. Sentiment describes emotional tone; intent describes the action or information need. It also differs from named-entity recognition, which identifies spans of text but does not determine the user’s overall objective.

Design the intent taxonomy before choosing a model

Poor label design creates more problems than a modest model. Start from the actions your product can actually support, not from every possible phrasing users may produce.

A practical taxonomy should be:

• Mutually distinguishable: Labels such as payment_failed and payment_pending need clear operational definitions.
• Actionable: Each intent should map to a workflow, API, answer, or escalation path.
• Balanced enough to evaluate: Very rare intents may need hierarchical classification or a fallback queue.
• Stable: Avoid labels tied to temporary campaign names or internal team structures.
• Versioned: Record when labels are added, merged, split, or deprecated.

Define boundaries with positive and negative examples. For a banking assistant, “Why did my UPI payment fail?” should not be grouped with “How do I set up UPI?” even though both mention UPI. Include hard negatives, code-mixed text, spelling variations, and incomplete requests from actual users after removing sensitive information.

For short, noisy queries, a dedicated workflow is often better than treating every input like a full sentence. See this practical guide to intent extraction from short text for data and modelling patterns that work with sparse context.

Choosing an approach

Rules and classifiers

Rules, keyword lists, and regular expressions remain useful for deterministic cases: account numbers, OTP-related phrases, known commands, and compliance-sensitive flows. They are transparent and quick to deploy, but brittle when users paraphrase a request.

Classical supervised models such as logistic regression or support vector machines can perform well with clean, narrow datasets. They are inexpensive to run and easier to inspect, making them suitable for high-volume support categories with stable language.

Transformer and multilingual models

Transformer encoders can capture context and paraphrases more effectively. Fine-tuning a multilingual model may be appropriate when labelled data exists across Hindi, Bengali, Tamil, Telugu, Marathi, or other target languages. A lightweight encoder can often meet latency and cost requirements better than sending every message to a large generative model.

Generative models are useful for zero-shot routing, taxonomy discovery, and difficult long-tail cases, but they should not be trusted to trigger high-impact actions without validation. A robust architecture may use a classifier for common intents, retrieval or rules for known entities, and an LLM only for fallback interpretation or response generation.

India-specific performance depends heavily on data quality. Plan for transliteration (“mera paisa kab aayega”), code-mixing (“refund kab tak process hoga?”), regional vocabulary, and speech-recognition errors. Pair intent work with low-resource Indic NLP techniques and carefully documented low-resource language datasets.

Build the training dataset

Collect examples from search logs, support tickets, call transcripts, failed chatbot sessions, and moderated user research. Keep the original wording, including typos and mixed scripts, because cleaning everything into formal language produces unrealistic benchmarks.

For every example, store:

• Intent label and label version
• Entity spans and normalised values
• Language, script, and code-mixing indicators
• Conversation turn and channel
• Personally identifiable information handling status
• Annotator decision and disagreement notes

Use at least two annotators for ambiguous categories and measure agreement. Review disagreements to improve the taxonomy rather than forcing uncertain examples into arbitrary labels. Split data by conversation or user, not random utterance alone, to avoid leakage between training and test sets.

Preprocessing should be conservative. Normalise Unicode and obvious formatting noise, but preserve meaningful distinctions such as negation, numbers, currency, and language choice. Build reusable pipelines; small Python utilities for automating data preprocessing can make experiments reproducible.

Evaluate what matters in production

Accuracy can hide serious failures when one intent dominates the dataset. Track per-intent precision, recall, F1 score, and a confusion matrix. Also measure:

• Macro F1: Gives rare intents equal weight.
• Top-k recall: Useful when a downstream reranker or clarification step can choose among candidates.
• Abstention quality: Whether low-confidence cases are safely rejected.
• Entity exact match and span F1: Measures whether required details were captured.
• End-to-end task completion: The most important business metric.
• Latency, cost, and availability: Essential for voice and high-volume systems.

Test separately on English, Indic scripts, transliterated text, code-mixed inputs, short queries, noisy ASR transcripts, and adversarial phrasing. Monitor performance by language and user segment; an aggregate score can conceal poor results for a smaller but important population.

Confidence thresholds should be tied to risk. A wrong FAQ answer may be recoverable, while a mistaken money transfer or account closure is not. For sensitive actions, require confirmation, verify entities, and provide a human escalation path. Study ways to improve intent recognition in conversational AI for practical strategies around context, ambiguity, and fallback handling.

Deployment architecture for Indian products

A production service commonly follows this sequence:

1. Receive text or speech transcript.
2. Detect language, script, and channel quality.
3. Apply safety, privacy, and basic normalisation checks.
4. Predict intent and extract entities.
5. Resolve entities against product data or APIs.
6. Check confidence, permissions, and workflow state.
7. Ask a clarifying question, execute a confirmed action, or escalate.
8. Log an anonymised outcome for monitoring and retraining.

Keep sensitive inference close to the data when possible. Local or private deployment may be preferable for regulated workloads, especially when data residency, predictable latency, or offline operation matters. For voice agents, intent quality depends on transcription quality and turn-taking; review guidance on natural-sounding TTS for voice agents alongside NLU evaluation.

Common failure modes

• Overlapping labels: Rewrite definitions or introduce a hierarchical taxonomy.
• Training on synthetic text only: Add real, messy user language and human review.
• Ignoring context: Pass relevant conversation state instead of classifying each turn independently.
• No unknown class: Include out-of-scope examples and an explicit abstention path.
• Silent model drift: Monitor new phrases, changing products, and language-specific error rates.
• Executing without confirmation: Add confirmation for irreversible or financial actions.

The best intent systems are not those that force every message into a label. They are systems that know what they understand, ask useful questions when they do not, and connect predictions to safe, measurable workflows. For Indian AI products, that means treating multilingual data, privacy, human escalation, and operational reliability as first-class design requirements—not later enhancements.