Generative AI is moving from isolated experiments to customer support, software development, sales, internal search, finance, and operations. That shift changes the governance question. It is no longer enough to ask whether a model is accurate in a demo; businesses must know what data enters the system, who can act on its output, how failures are detected, and who is accountable.
For Indian companies, governance sits at the intersection of the Digital Personal Data Protection (DPDP) Act, sectoral rules, cybersecurity obligations, contractual commitments, intellectual-property risk, and practical concerns such as multilingual accuracy. The aim is not to create a committee that blocks useful work. It is to build repeatable controls that let teams ship AI products safely and measure whether they create value.
Start with an AI system inventory
Governance begins with visibility. Many organisations have more AI use than they realise: employees paste confidential material into public chatbots, marketing teams use content-generation tools, developers install coding assistants, and product teams connect models to internal databases.
Create a register for every GenAI use case, including:
- Business owner and technical owner
- Model provider, model version, hosting region, and subcontractors
- Input data categories, including personal, financial, health, confidential, and proprietary data
- Whether the system uses prompting, retrieval-augmented generation (RAG), fine-tuning, or autonomous tool use
- Users, affected individuals, and downstream systems
- Required human review and escalation paths
- Accuracy, safety, latency, cost, and incident metrics
Classify use cases by impact rather than by novelty. An internal brainstorming assistant is not equivalent to a credit, insurance, hiring, medical, education, or public-service decision tool. Higher-impact systems need stronger evidence, narrower permissions, more testing, and human decision authority.
For teams building autonomous workflows, the practical controls described in how to build generative AI agents are especially relevant: define tools, permissions, stopping conditions, and audit logs before adding autonomy.
Translate the DPDP Act into operating controls
The DPDP Act, 2023, and its implementation framework should be treated as part of product design, not a legal review at the end. A business should document its role as Data Fiduciary or Data Processor, identify the purpose for each processing activity, and confirm that notices, consent or other lawful grounds, retention, and user-rights processes work when AI is involved.
Key questions include:
- Purpose limitation: Is personal data being sent to a model for a purpose covered by the notice, or is a new AI use being introduced without an appropriate basis?
- Data minimisation: Can the task be completed with redacted, masked, synthetic, aggregated, or non-personal data?
- Vendor controls: Do contracts restrict provider training on customer data, define breach responsibilities, support deletion requests, and disclose subprocessors and retention?
- Access and deletion: Can the organisation locate personal data in prompts, conversation histories, vector databases, caches, logs, and fine-tuning datasets?
- Cross-border processing: Has the organisation assessed provider locations, transfer arrangements, sectoral requirements, and any government restrictions that apply to the data?
- Children’s data: Do products likely to be used by children apply the additional safeguards required under Indian law?
Do not promise that a model can simply “forget” a record. Deletion from a RAG index, application database, logs, and backups is different from removing information from model weights. Avoid training on personal data unless the necessity, lawful basis, technical feasibility, and deletion position are clearly documented.
Choose models and architectures by risk
There is no universally safest choice between a proprietary API, an open-weight model, and a self-hosted system. Compare options against the data and consequence of failure.
- Managed enterprise APIs: Often provide mature security features, but review retention, regional processing, training defaults, and availability commitments.
- Open-weight models: Can support greater control and deployment flexibility, but the business owns patching, evaluation, infrastructure security, and abuse prevention.
- RAG systems: Keep changing business knowledge outside model weights and can improve traceability, but retrieval quality, document permissions, and citation accuracy require testing.
- Fine-tuned models: May improve consistency for a narrow task, but increase data governance, versioning, rollback, and deletion complexity.
Use least privilege throughout the stack. A support assistant should not have unrestricted access to the CRM, payment system, or production database. Separate read and write actions, require confirmation for consequential actions, and log the identity, prompt, retrieved sources, tool calls, model version, and final result.
India-focused teams should also test language and modality choices rather than assuming English benchmarks transfer. Work on open-source vision-language models for Indian languages and AI-based tools for local Indian dialects illustrates why regional language coverage, transliteration, code-switching, accents, and low-resource contexts need dedicated evaluation.
Control hallucinations, prompt injection, and data leakage
Accuracy is a system property, not a model promise. Establish an evaluation set drawn from real Indian customer queries, internal documents, edge cases, and known failure modes. Measure more than answer quality:
- Unsupported claims and missing citations
- Refusal and escalation behaviour
- Personal-data leakage and secret exposure
- Prompt-injection resistance
- Toxic, discriminatory, or culturally inappropriate responses
- Performance across languages, scripts, accents, and code-mixed speech
- Cost, latency, and reliability under production load
Use layered controls: input classification, sensitive-data detection, retrieval filters, grounded generation, output validation, rate limits, and human review. Treat documents, web pages, emails, and retrieved text as untrusted input; prompt injection can instruct a model to ignore system rules or misuse connected tools.
Red-team tests should include requests for confidential records, unauthorised refunds, policy bypasses, malicious code, fabricated citations, and indirect attacks hidden inside documents. Re-test after model, prompt, tool, or knowledge-base changes. A dashboard should show incidents, blocked requests, escalation rates, hallucination samples, and drift—not just total usage.
Make fairness and accountability concrete
Bias testing must reflect the actual population and decision context. For Indian deployments, test differences across relevant languages, regions, gender, socioeconomic proxies, disability contexts, names, occupations, and educational backgrounds. Avoid using caste or other sensitive attributes casually; where legally and ethically justified for fairness testing, apply strict access controls and purpose limits.
Do not allow a generative model to make a high-impact decision without a defined human authority. Reviewers need enough evidence to challenge the output, not merely click “approve.” Tell users when they are interacting with AI where that disclosure matters, provide a route to a human, and preserve an appeal or correction mechanism.
A responsible governance group should include product, engineering, security, privacy/legal, risk, and domain specialists. For smaller companies, this can be a named cross-functional working group rather than a permanent board. Assign one accountable owner for each system and publish an escalation matrix.
A practical 90-day implementation plan
Days 1–30: discover and classify
- Inventory approved and unofficial AI use
- Classify data, impact, vendors, and affected users
- Freeze high-risk experiments that lack an owner or safeguards
- Publish an interim acceptable-use policy
Days 31–60: design controls
- Complete privacy, security, and vendor assessments
- Select model and hosting architecture
- Build redaction, access control, logging, retrieval, and human-review controls
- Create a representative evaluation and red-team suite
Days 61–90: pilot and operate
- Run a limited pilot with measurable success and failure thresholds
- Train employees on approved tools and sensitive-data handling
- Establish incident response, rollback, deletion, and change-management procedures
- Review metrics with leadership and decide whether to expand, redesign, or stop
What good governance looks like
By 2026, credible governance should be visible in delivery workflows: every production use case has an owner, risk tier, data map, model record, evaluation evidence, access policy, monitoring plan, and rollback path. Governance teams should track outcomes such as resolution quality, error rates, privacy incidents, review burden, cost per task, and user complaints.
The strongest Indian businesses will not compete by using the most powerful model everywhere. They will build a disciplined portfolio: narrow models for narrow jobs, grounded data for factual tasks, human judgment where stakes are high, and clear evidence that AI improves the service without shifting unacceptable risk to customers or employees.