AI features rarely fail because the model cannot answer a question. They fail because every request is sent to an expensive model, prompts grow without control, and the team discovers unit economics only after launch. For Indian startups serving price-sensitive users, building AI apps without high API costs requires cost control to be part of the product architecture from the first prototype.
The objective is not to eliminate API usage. It is to reserve premium models for tasks that genuinely need them, while cheaper models, retrieval, caching, structured workflows, and evaluation handle the rest.
Start with a cost model, not a model choice
Before comparing providers, define the economics of one completed user task. Track:
- Input and output tokens per request
- Average requests per user and per workflow
- Model price, embedding price, and storage cost
- Retry, fallback, and tool-call frequency
- GPU, database, observability, and bandwidth costs
- Revenue or gross margin associated with the task
Use a simple formula: cost per successful task = inference + retrieval + tools + infrastructure + retries. A chatbot message is a poor unit if one customer conversation contains 20 calls. For a voice product, calculate the full call—including speech-to-text, reasoning, text-to-speech, and telephony. The same discipline applies when estimating voice agent pricing and ROI.
Set a budget per workflow and alert when it is exceeded. Cost data should be visible by customer, feature, model, and prompt version—not only as a monthly cloud bill.
Route requests by difficulty
A single premium model for every request is usually the fastest route to weak margins. Build a cascade instead:
1. Check an exact or semantic cache.
2. Use deterministic code for rules, lookups, validation, and calculations.
3. Send routine classification, extraction, rewriting, and summarisation to a small model.
4. Use retrieval with a capable but economical model for knowledge-grounded answers.
5. Escalate only ambiguous, high-risk, or reasoning-heavy requests.
A lightweight router can use intent, input length, confidence, customer tier, and risk category. Do not let a model decide everything: a rules-based gate is cheaper, faster, and easier to audit. For complex products built from multiple agents, define budgets per agent and cap recursion, retries, and tool calls; the principles in building distributed systems with AI agents are especially relevant here.
Use smaller models where accuracy permits
Small language models are effective for narrow tasks when the output contract is clear. Examples include lead classification, invoice-field extraction, moderation, language detection, email drafting, and support-ticket tagging. Test current open models in the 3B–14B range alongside hosted small models, rather than assuming a flagship model is necessary.
Evaluate on your own Indian-language, domain, and edge-case data. A cheaper model that fails on Hinglish, regional names, currency formats, or local regulatory terminology may create more operational cost than it saves. Keep a premium fallback for uncertain outputs and sample a portion of low-risk decisions for human review.
Open-source work is also a practical route for experimentation and capability building; Indian student developers building open-source AI offers useful context for teams starting with constrained budgets.
Control self-hosting economics
Self-hosting can reduce marginal cost at steady, high utilisation, but it is not automatically cheaper. Include GPU idle time, engineering, monitoring, storage, upgrades, security, and failover in the comparison. APIs often win at low or unpredictable volume; reserved or dedicated GPU capacity becomes more attractive when traffic is sustained and predictable.
For local development, Ollama is convenient. For production serving, engines such as vLLM can improve throughput through continuous batching and efficient attention management. Quantisation can reduce memory requirements, but benchmark quality and latency on your actual workload. A 4-bit model that needs extensive retries is not a saving.
For Indian deployments, compare Mumbai, Hyderabad, and other available regions on total delivered cost—not headline GPU rates. Measure latency for your target users, data-transfer charges, availability, support, and data-processing requirements. Keep a hosted fallback during outages and deployment changes.
Reduce tokens before reducing model quality
Prompt optimisation is one of the lowest-risk savings. Store reusable instructions efficiently and avoid repeating large policy blocks on every request where the provider supports prompt caching. Send only the fields a task needs, trim retrieved passages, and cap output length.
Use structured outputs with explicit schemas. Ask for the required fields, not an essay that your application later parses. Remove few-shot examples when a smaller model or a short rubric can do the job. Summarise long conversation history into state, retaining decisions and unresolved issues rather than replaying every message.
Treat retries as a defect signal. Validate JSON locally, make tool calls idempotent, and retry only transient failures. An automatic retry of a bad prompt can double cost without improving the result.
Cache repeated work and design efficient RAG
Exact caching is valuable for repeated system responses. Semantic caching can help support and documentation products where different questions have the same answer, but it needs safeguards. Cache only when similarity thresholds, tenant boundaries, permissions, language, and document versions match. Never return a cached answer across customers or roles without checking access control.
Retrieval-Augmented Generation is usually more economical than fine-tuning for changing business knowledge. However, poor retrieval creates oversized prompts and expensive retries. Improve the pipeline by:
- Chunking documents around meaningful sections
- Removing duplicate and stale content
- Applying metadata and tenant filters before vector search
- Reranking a small candidate set
- Passing only evidence relevant to the question
- Citing source passages and declining when evidence is weak
For high-stakes use cases, cost optimisation cannot weaken evidence controls. Teams working on data veracity infrastructure for high-stakes AI should treat provenance, freshness, and audit logs as core product requirements.
Measure quality and cost together
Create an evaluation set before changing models. Include normal requests, adversarial prompts, multilingual inputs, long-context cases, and production failures. Track task success, groundedness, refusal quality, latency, first-pass completion, and cost per successful task.
Run changes as controlled experiments: smaller model versus baseline, shorter prompt versus baseline, cached versus uncached, and self-hosted versus API. A 40% reduction in token spend is irrelevant if successful completion falls by 20%. Conversely, a slightly more expensive model may be cheaper overall if it avoids human escalation and retries.
A practical rollout plan
Week one: instrument tokens, latency, retries, model choice, and workflow-level cost. Set per-request and per-user budgets.
Weeks two and three: add deterministic shortcuts, trim prompts, constrain outputs, and build a representative evaluation set.
Weeks four and five: introduce caching, RAG filtering, and a small-model route. Keep premium fallback and compare successful-task economics.
After validation: test quantised self-hosted models at realistic traffic, negotiate volume pricing, and establish monthly reviews for prompts, models, and GPU utilisation.
The strongest architecture is rarely “API versus self-hosting.” It is a managed mix: code for predictable work, small models for routine work, retrieval for changing knowledge, premium APIs for difficult cases, and human review where errors are costly. That approach lets Indian builders scale useful AI products while protecting margins, reliability, and user trust.