React is a strong foundation for LLM products because it can turn unpredictable model behaviour into a responsive, inspectable user experience. But a production application is more than a chat component calling an API. It needs clear boundaries between the browser, orchestration layer, retrieval system, model provider, data stores, and background workers.
This guide explains how to design that system in 2026, with practical choices for Indian teams building multilingual assistants, internal copilots, customer-support tools, and domain-specific products.
Start with the right application architecture
A reliable full-stack LLM application usually has six layers:
- React or Next.js client: Renders conversations, citations, tool status, approvals, errors, and feedback.
- Application API: Authenticates users, validates requests, applies quotas, and hides provider credentials.
- LLM orchestration: Selects prompts, models, tools, retrieval steps, and fallback behaviour.
- Data and retrieval layer: Stores documents, embeddings, conversation state, tenant permissions, and audit records.
- Background workers: Process ingestion, document extraction, evaluations, report generation, and other long-running jobs.
- Observability and policy controls: Tracks latency, token usage, failures, unsafe outputs, and user feedback.
Next.js is convenient when one team owns both UI and API routes. A separate Node.js or FastAPI service is often better when several clients share the same backend, when Python libraries are central to the product, or when workloads require independent scaling. For larger systems, apply the principles in this guide to scaling backend infrastructure for AI applications rather than allowing the frontend framework to become the architecture.
Design the React experience around model uncertainty
LLM output is incremental, fallible, and sometimes slow. Your interface should expose that reality without making users manage the underlying complexity.
A useful chat screen should support:
- Token streaming with a visible connection and generation state.
- A cancel action so users can stop expensive or irrelevant runs.
- Markdown rendering with safe link and HTML handling.
- Citations that open the source passage, not merely a document title.
- Tool-status events such as “searching”, “checking account data”, or “drafting”.
- Retry and regenerate actions with clear error messages.
- Conversation persistence, branching, and feedback at message level.
Keep transient generation state separate from durable application state. A React query cache can manage server data, while local component state or a dedicated store can manage input text, optimistic messages, and active streams. Do not trust hidden client state for permissions, billing, or conversation ownership; the server must verify all of these.
For products serving Indian users, plan for multilingual input and output from the start. Test code-switching, transliterated Hindi, Tamil, Bengali, and other target languages with real users. A chatbot architecture is not automatically a multilingual product; retrieval quality, typography, token costs, and evaluation sets all change by language. See building multilingual chatbots for Indian startups for product-specific considerations.
Stream responses with an explicit event contract
Waiting for a complete JSON response creates a poor experience and increases the chance of gateway timeouts. For ordinary text generation, Server-Sent Events or a streaming fetch response is usually simpler than WebSockets. WebSockets become more useful when the client and server need continuous two-way communication, such as collaborative workspaces, live voice, or many concurrent events.
Define an event schema before writing the UI. For example:
{"type":"message.start","id":"msg_123"}
{"type":"text.delta","id":"msg_123","text":"The answer"}
{"type":"citation.add","source":{"id":"doc_7","title":"Policy"}}
{"type":"message.complete","usage":{"input_tokens":420,"output_tokens":180}}This is more durable than sending unstructured text. It lets the client distinguish a model token from a citation, tool call, warning, or final usage record. Include request IDs and sequence numbers so the client can detect dropped events. Define what happens when the stream disconnects: resume, show the partial response, or mark the run as failed.
Libraries such as the Vercel AI SDK can accelerate implementation, but keep your provider adapter behind a small internal interface. That makes model changes, regional routing, retries, and fallback policies easier without rewriting the React layer.
Build RAG as a data product, not a prompt trick
Retrieval-Augmented Generation works only when the ingestion and permission model are sound. A practical pipeline is:
1. Extract text and metadata from PDFs, web pages, tickets, or database records.
2. Remove boilerplate and preserve headings, tables, page numbers, language, and access controls.
3. Split content into meaningful sections rather than using an arbitrary character count everywhere.
4. Generate embeddings and store them in pgvector, a managed vector database, or another suitable index.
5. Retrieve candidates using semantic search, keyword search, or a hybrid approach.
6. Apply metadata filters and, where needed, rerank the candidates.
7. Pass only relevant, permission-checked context to the model.
8. Return citations and measure whether the answer is actually supported.
Do not assume the top three chunks are always enough. Retrieval quality depends on query type, document structure, language, and the cost of missing information. Add “I don’t know” or escalation behaviour when evidence is weak. In multi-tenant products, apply tenant and user permissions during retrieval—not after generation.
For lower infrastructure cost, PostgreSQL with pgvector can be a sensible first choice. A dedicated vector database may become worthwhile when index size, traffic, filtering, or operational requirements demand it. Benchmark with your own corpus instead of selecting a database from a feature checklist.
Choose Node.js, Python, or both deliberately
Node.js and Next.js work well for request handling, streaming, authentication, and product integration. FastAPI is often preferable for Python-heavy pipelines, evaluation jobs, scientific workloads, and model-serving integrations. A hybrid system is reasonable when the boundary is explicit: the product API owns identity and business rules, while a Python service owns specialised inference or retrieval jobs.
Avoid placing every operation in a synchronous request. Document ingestion, bulk summarisation, exports, and long reports should run through a queue such as BullMQ, Celery, or a managed task service. Store job status in a durable database and let React poll or subscribe to progress. The same separation is important when building distributed systems with AI agents, where retries and partial completion are normal rather than exceptional.
Secure the application before scaling it
Never expose model-provider keys in the browser. Authenticate every API request and authorise access to conversations, files, tools, and retrieved records on the server. Add per-user, per-organisation, and endpoint-level rate limits, with separate controls for expensive tools.
Treat uploaded files and retrieved text as untrusted input. Prompt injection can arrive through a user message, a PDF, a web page, or a tool result. Use allowlisted tools, structured arguments, least-privilege credentials, and confirmation steps for actions that change data or send messages. Log the prompt and output lineage needed for debugging, while minimising retention of sensitive content.
For Indian deployments, map data flows before selecting providers. Identify whether personal data, financial information, health records, or confidential business documents leave your controlled environment. Apply encryption, retention limits, access logging, deletion workflows, and contractual safeguards appropriate to your use case and obligations under India’s privacy framework. Mask sensitive fields where the model does not need them.
Measure quality, latency, and cost together
Track more than time to first token. Useful metrics include:
- Time to first token and time to final response.
- Retrieval hit rate, citation support, and no-answer accuracy.
- Task completion and user correction rates.
- Input and output tokens by feature, tenant, and model.
- Tool-call failures, retries, fallbacks, and abandoned streams.
- Safety incidents and policy-review outcomes.
Create a small evaluation set before launch. Include common questions, ambiguous requests, adversarial prompts, multilingual examples, and cases where the correct answer is “insufficient information”. Run it whenever you change prompts, chunking, embedding models, or providers.
Control cost with model routing, bounded context, caching, batching, and budgets. Use a smaller model for classification or query rewriting and reserve stronger models for difficult tasks. Cache embeddings and stable retrieval results, but do not cache personalised answers without checking authorisation and freshness.
A practical launch sequence
Start with one narrowly defined workflow and a measurable success criterion. Then:
- Build the API boundary and authentication before connecting production data.
- Add streaming, cancellation, citations, and structured errors.
- Create a versioned ingestion and retrieval pipeline.
- Add logs, traces, token accounting, and an evaluation set.
- Test prompt injection, data leakage, rate limits, and provider outages.
- Introduce queues for jobs that exceed normal request timeouts.
- Deploy with staged rollouts and a kill switch for problematic models or tools.
Teams that want to minimise infrastructure risk can prototype with open-source components and managed deployment, then replace individual services as usage justifies it. Building high-performance AI applications with open-source tools offers a useful path for keeping the first version economical without sacrificing engineering discipline.
Frequently asked questions
Should I use React or Next.js?
React is the UI library; Next.js adds routing, server rendering, API capabilities, and deployment conventions. Use React with a separate backend when the client and service need independent lifecycles. Use Next.js when a single team benefits from an integrated product stack.
Are WebSockets required for streaming LLM output?
No. SSE or streaming HTTP is usually sufficient for one-way token and event delivery. Choose WebSockets for genuinely bidirectional, persistent interactions rather than because the application contains a chat window.
How can I reduce hallucinations?
Improve retrieval and source permissions, require citations, constrain tool use, evaluate against a representative test set, and provide an explicit no-answer path. Prompt wording alone is not a reliable quality strategy.
What is the best database for an early product?
Use the database your team can operate reliably. PostgreSQL with pgvector is often a capable starting point; move to a specialised vector system when measured scale or retrieval requirements warrant it.
Build with support for the next stage
A React frontend can make an LLM product feel immediate, but durable value comes from trustworthy data access, clear event contracts, measurable quality, and disciplined operations. Build those foundations early, then add agents, voice, or richer automation only where they improve a defined user workflow. If you are building an AI company in India, AI Grants India can connect eligible teams with funding and support as the product moves from prototype to deployment.