GitHub is more than a place to store application code. For an AI web product, it can hold the application, prompt templates, evaluation datasets, infrastructure definitions, documentation, and automated deployment workflows. Used properly, it gives a small team a repeatable path from prototype to a production service.
This guide explains integrating artificial intelligence into web applications with GitHub. It focuses on decisions that matter in practice: where inference should run, how to connect a model to private data, how to stream useful responses, and how to test an AI feature before users depend on it.
Start with the product boundary
Do not begin by selecting an LLM. Begin by defining the job the AI feature must perform and the consequences of an incorrect answer. A support assistant, document classifier, coding copilot, and voice workflow need different models, permissions, latency targets, and evaluation methods.
Write a short feature contract covering:
- Input: accepted formats, maximum size, language, and sensitive fields.
- Output: free text, structured JSON, a classification, an action, or a citation-backed answer.
- Failure behaviour: when the system should refuse, ask for clarification, or hand off to a person.
- Success measures: accuracy, groundedness, response time, resolution rate, and cost per request.
A narrow, measurable first release is usually safer than a general-purpose chatbot. Teams that are new to this work can study open-source AI projects for beginners on GitHub before choosing a production architecture.
Choose an architecture that fits the workload
Most web applications use one of three patterns:
- Hosted model API: Your backend calls a provider through an SDK or HTTPS endpoint. This is the fastest route to a working product and avoids GPU operations.
- Self-hosted inference: Your team runs an open model on a cloud GPU or private server. This can improve control and predictable high-volume pricing, but adds serving, patching, and scaling work.
- Browser or edge inference: A compact model runs close to the user through technologies such as ONNX Runtime Web or WebGPU. This suits privacy-sensitive, low-complexity tasks, not every reasoning workload.
A hybrid architecture is often sensible for Indian startups: use a hosted model for complex reasoning, a smaller model for routing or classification, and retrieval from an application-controlled database. Keep provider calls behind your own backend so you can change models without rewriting the frontend.
For deeper infrastructure decisions, use this guide to scaling backend infrastructure for AI applications alongside your latency and traffic estimates.
A practical GitHub-based stack
A common implementation pairs a Next.js or React frontend with a TypeScript API layer, or a React frontend with a Python FastAPI service. TypeScript is convenient for streaming interfaces and shared schemas; Python remains strong for data pipelines, model tooling, and scientific libraries. If Python is your preferred backend, see how to integrate LLM APIs in Python web apps.
Useful building blocks include:
- Model SDKs: official provider clients, configured only on the server.
- AI interface libraries: tools such as Vercel AI SDK can simplify streaming and chat state.
- Workflow and retrieval frameworks: LangChain, LlamaIndex, or a small custom orchestration layer when the flow is simple.
- Data storage: PostgreSQL with pgvector, a managed vector database, or a search engine with hybrid retrieval.
- Validation: Zod, Pydantic, or JSON Schema to constrain model output.
- Observability: request IDs, latency, token usage, model versions, tool calls, and safe error logs.
Pin dependency versions, review licences, and inspect the repository’s maintenance history before adopting a popular project. GitHub stars are not a security or reliability guarantee.
Build retrieval-augmented generation carefully
RAG is useful when the application must answer from changing or private material. The pipeline normally has four stages:
1. Extract and clean documents, preserving titles, headings, dates, access rules, and source URLs.
2. Split content into meaningful passages rather than arbitrary character blocks.
3. Generate embeddings and store them with metadata such as tenant, language, document type, and permission scope.
4. Retrieve relevant passages, apply access checks, and send only the necessary context to the model.
Add citations or source links to the response wherever users need to verify an answer. Retrieval is not a substitute for authorization: a vector search must never return a document merely because its text is semantically similar. Re-index documents when their source changes, and retain an index version so you can reproduce an answer during debugging.
Design the request path for users
AI calls are slower and less predictable than ordinary database requests. Stream output when partial text is useful, but do not stream an unvalidated action or financial decision. Return structured events such as status, token, citation, tool_result, and error so the client can render each state explicitly.
Set timeouts, cancellation, retries with backoff, and per-user rate limits. Protect long-running work with a queue rather than holding an HTTP request open indefinitely. Cache only when the prompt, relevant data, user permissions, and model settings make the result safe to reuse. For background processing and high-volume systems, plan capacity using the principles in building high-performance AI applications with open-source tools.
Treat GitHub as an AI delivery system
Organise the repository so changes are reviewable:
- Keep prompt templates, schemas, retrieval settings, and model configuration in version control.
- Store small, representative evaluation cases separately from production user data.
- Use pull requests for prompt and model changes, not only application code.
- Keep secrets in GitHub Actions or deployment-provider secret stores; never commit API keys.
- Use Git LFS or an external model registry for large weights instead of placing them directly in Git history.
GitHub Actions can run unit tests, type checks, dependency scans, prompt evaluations, and deployment gates. A useful pipeline compares the proposed version against a fixed test set for answer quality, refusal behaviour, citation accuracy, latency, and estimated cost. For regulated or customer-facing features, require human approval before production promotion.
Security, privacy, and Indian deployment concerns
Assume that user input may contain prompt injection, personal data, malware, or instructions designed to manipulate tools. Keep system instructions separate from retrieved content, allow-list tools and arguments, validate tool results, and require confirmation for irreversible actions. Do not let the model decide its own permissions.
Apply authentication and tenant checks before retrieval. Redact or minimise personal data in logs, define retention periods, and document where prompts, embeddings, and outputs are processed. For Indian businesses, map the design to the Digital Personal Data Protection Act and contractual requirements before sending sensitive information to an external provider. Healthcare teams should also review the specific risks covered in integrating computer vision in healthcare apps, even when their feature is text-based.
Control cost and measure quality
Track cost per successful task, not just tokens. Route simple requests to smaller models, limit context length, deduplicate documents before embedding, and summarise long conversations. Set budgets and alerts by customer, feature, and environment. A cheaper model that produces more retries may cost more overall.
Evaluate with a mixture of automated checks and expert review. Test multilingual inputs, including Indian English and relevant regional languages, adversarial prompts, empty retrieval results, stale documents, and provider failures. Monitor real user feedback, but sample safely and remove personal information before analysis.
The strongest AI web applications are not the ones with the longest prompts or most agents. They are systems with a clear job, controlled data access, observable behaviour, and a GitHub workflow that makes every model-related change testable and reversible.