RAG for Information Retrieval: Architecture, Evaluation and Use Cases

What RAG means for information retrieval

Retrieval-augmented generation (RAG) connects a search system to a large language model (LLM). Instead of asking the model to answer from its training memory alone, the system first retrieves relevant passages from an approved knowledge base and then uses those passages to draft an answer.

This distinction matters. A conventional search engine returns links or documents; a standalone LLM produces fluent text that may be outdated or unsupported. RAG combines the two: retrieval supplies current evidence, while generation turns that evidence into a useful response. For Indian enterprises, this can mean answering questions from policy manuals, bilingual service documents, product catalogues, legal records, or internal operational data without retraining a model every time content changes.

RAG is not automatically accurate. Its quality depends on the complete pipeline: source governance, document parsing, indexing, query understanding, ranking, prompt construction, model behaviour, and evaluation.

How a RAG retrieval pipeline works

A production system usually has two connected paths.

1. Indexing path

Collect sources: Bring together PDFs, web pages, tickets, spreadsheets, databases, manuals, and structured records.
Parse content: Extract text, headings, tables, metadata, page numbers, and document relationships. Scanned Indian-language documents may require OCR and language-specific validation.
Chunk documents: Split content into meaningful sections rather than arbitrary character counts. Preserve titles, section labels, dates, and access permissions with every chunk.
Create representations: Generate embeddings for semantic search, while also maintaining an inverted index for exact terms, identifiers, and citations.
Store metadata: Record source, owner, effective date, language, department, geography, sensitivity, and document version.

2. Query path

Understand the question: Detect language, intent, entities, filters, and whether the user needs a fact, comparison, calculation, or document lookup.
Retrieve candidates: Combine keyword search such as BM25 with vector search. Hybrid retrieval is often stronger than either method alone, especially for policy numbers, scheme names, product codes, and names in Indian languages.
Rerank results: Use a cross-encoder or another relevance model to reorder the top candidates. Apply access control before content reaches the model.
Build context: Select compact, non-duplicative passages and include source labels, dates, and page references.
Generate with constraints: Instruct the LLM to answer only from supplied evidence, distinguish unknowns, and cite sources.
Validate and log: Check citations, confidence signals, policy violations, latency, and user feedback before returning the answer.

This architecture is also relevant to fine-tuning LLMs for enterprise knowledge retrieval, although fine-tuning and RAG solve different problems. Fine-tuning changes model behaviour; RAG supplies changing or private knowledge at query time.

Design choices that determine retrieval quality

Chunking and metadata

Chunk by meaning. A section containing an eligibility rule should remain together with its conditions and exceptions. Very small chunks lose context; very large chunks dilute relevance and increase token cost. Test several sizes using real questions rather than adopting a universal setting.

Metadata improves both retrieval and governance. Filters for state, department, language, date, customer segment, or document status can remove irrelevant content before semantic ranking. For regulated workflows, retain the exact source passage used to answer a question.

Hybrid and multilingual retrieval

Dense embeddings handle paraphrases, while lexical search catches exact names, acronyms, numbers, and statutory language. Combining both is especially useful for Indian domains, where a query may mix English with Hindi, Tamil, Marathi, or transliterated terms.

Use multilingual embeddings only after testing them on the languages and scripts your users actually use. Build evaluation sets with code-mixed queries, spelling variations, transliteration, and regional terminology. Speech interfaces can feed RAG as well; systems using Hindi or other Indian languages should account for recognition errors before retrieval, as discussed in Hindi ASR and low-WER speech recognition.

Structured data and document retrieval

Do not force every question through vector search. Route calculations, filters, inventory checks, and tabular lookups to SQL or APIs. Use document retrieval for explanations and policy text, then combine the results when an answer requires both a record and its governing rule.

For sensitive public-sector or land-related workflows, provenance is essential. A project involving automated information extraction from land records in India should preserve document IDs, page locations, extraction confidence, and human review status rather than presenting an untraceable summary.

Measuring a RAG system properly

A fluent answer is not proof of a good system. Evaluate retrieval and generation separately, then assess the complete user task.

Useful retrieval measures include:

Recall@k: Whether the required evidence appears in the top-k results.
Precision@k: How much of the retrieved context is relevant.
MRR or nDCG: Whether the best evidence is ranked near the top.
Filter accuracy: Whether permissions, dates, language, and geography are correctly applied.

Useful answer measures include:

Faithfulness: Whether claims are supported by retrieved evidence.
Answer relevance: Whether the response addresses the actual question.
Citation correctness: Whether each citation supports the associated claim.
Completeness: Whether important conditions and exceptions are included.
Abstention quality: Whether the system says it lacks evidence instead of guessing.

Create a representative test set with easy, ambiguous, adversarial, multilingual, and unanswerable questions. Have domain experts review high-risk cases. Track latency, cost per query, retrieval failures, and escalation rates alongside accuracy.

Common failure modes and fixes

Irrelevant context: Improve chunking, metadata filters, hybrid retrieval, and reranking.
Correct passage, wrong answer: Tighten the prompt, reduce distracting context, and require claim-level citations.
Outdated answers: Add effective-date filters, document expiry workflows, and source ownership.
Duplicate or conflicting documents: Deduplicate content and show version precedence explicitly.
Prompt injection in retrieved text: Treat documents as untrusted data, isolate instructions, and test malicious content.
Permission leakage: Enforce access control during retrieval, not only in the final response.
High latency and cost: Cache stable queries, reduce candidate counts, use smaller rerankers, and route simple requests to cheaper models.
Poor performance on local language queries: Expand the evaluation set and consider language-aware query rewriting, not just translation.

For enterprise deployments, secure knowledge retrieval systems provide a useful reference point for identity, tenancy, audit logs, encryption, and data retention.

Practical deployment plan for Indian teams

Start with one narrow, high-value workflow: employee policy search, support resolution, compliance lookup, or scheme discovery. Establish an approved corpus and a human escalation path before adding more sources. Keep source content in India-approved infrastructure where organisational or regulatory requirements demand it, and document every external model and data-processing dependency.

A sensible pilot sequence is:

1. Define ten to twenty measurable user tasks.
2. Clean and label a small, authoritative corpus.
3. Implement hybrid retrieval with citations and access controls.
4. Test multilingual, unanswerable, and adversarial queries.
5. Compare against keyword search and a non-RAG baseline.
6. Review failures with domain experts and improve the corpus before changing the model.
7. Monitor quality, cost, latency, and escalation after launch.

For teams connecting RAG to operational workflows, the broader principles in AI for business efficiency are useful: automate only where outcomes can be measured, and keep people responsible for consequential decisions.

RAG versus fine-tuning and traditional search

Choose traditional search when users mainly need ranked documents and exact traceability. Choose RAG when users need a cited synthesis across multiple sources. Choose fine-tuning when the problem is consistent style, classification, extraction format, or domain behaviour rather than access to frequently changing facts. Many mature systems use all three.

RAG should be treated as an information product, not a prompt trick. Its defensibility comes from controlled sources, reproducible retrieval, transparent citations, and continuous evaluation. When these foundations are in place, it can make internal knowledge more accessible without pretending that the model knows more than the evidence supports.

FAQ

Is RAG the same as a vector database?

No. A vector database is one storage and retrieval component. RAG also requires parsing, chunking, metadata, ranking, prompting, generation, access controls, evaluation, and monitoring.

Does RAG eliminate hallucinations?

No. It can reduce unsupported answers by supplying evidence, but the model may still misread, combine, or invent claims. Citations, abstention rules, testing, and human review remain necessary.

How many documents should be retrieved?

There is no universal number. Retrieve enough evidence to answer the task without overwhelming the model. Measure recall and answer quality while varying the candidate count and reranking strategy.

Can RAG work with Indian languages?

Yes, but performance depends on parsing, OCR, embeddings, query handling, terminology, and evaluation data for each language. Test native-script and transliterated queries separately.

Apply for AI Grants India

If you are building a retrieval, language, or enterprise AI product for India, AI Grants India can help you identify funding opportunities and shape a stronger deployment case. Explain the problem, the data safeguards, the measurable impact, and why your approach is suited to Indian users and institutions.