Large language models do not retain application state after a request ends. Persistent memory is therefore an application-layer system: your product must decide what to save, where to store it, when to retrieve it, and how to correct or delete it. A reliable memory loop is not a larger chat-history prompt. It is a controlled pipeline that turns interactions into verified, useful state and feeds that state into later decisions.
For Indian builders, the problem is especially practical. Users may switch between English, Hindi, Hinglish, and regional languages; data may sit across domestic and cloud infrastructure; and products often need to serve users on mobile networks with tight latency and cost constraints. The design below focuses on a production-ready approach rather than a demo that simply stores every transcript.
Define what the system should remember
Start with memory policy, not a vector database. For every candidate memory, define its type, source, confidence, sensitivity, expiry, and owner. A useful taxonomy is:
- Working memory: The current task, recent turns, tool results, and unresolved questions placed in the active context window.
- Episodic memory: A specific event, such as a previous support case, purchase, lesson, or approval.
- Semantic memory: A durable fact or preference, such as a user’s preferred language or organisation.
- Procedural memory: Instructions for how the agent should perform a recurring workflow.
- System memory: Policies, product documentation, and operational knowledge maintained by your team rather than inferred from conversations.
Do not promote every statement to long-term memory. “I am travelling this week” may expire quickly, while “send invoices to this address” may be durable but sensitive. Store an explicit source and valid-from/valid-until interval so later retrieval can distinguish a current preference from an old observation.
This separation also helps when building AI agents with memory: the agent can use working memory for the present task without treating an uncertain conversational remark as a permanent user profile.
Choose a storage architecture
A practical implementation normally combines three layers:
- Relational storage: Use PostgreSQL or another SQL database for users, tenants, permissions, memory metadata, consent state, timestamps, and audit records.
- Vector search: Store embeddings for semantic recall of episodes, documents, and approved facts. PostgreSQL with pgvector can reduce operational complexity for an early product; a dedicated vector service may help at larger scale.
- Graph or structured relations: Use a graph database, or relational subject-predicate-object tables, when relationships and multi-hop queries matter—for example, a company’s subsidiaries, a student’s subjects, or a customer’s linked cases.
A memory record should include more than text and an embedding. Keep fields such as memory_id, tenant_id, subject_id, content, memory_type, source_event_id, confidence, importance, created_at, updated_at, expires_at, consent_scope, and status. Tenant and subject filters must be enforced before retrieval, not applied after a broad search.
For teams building a broader product platform, the data contracts and observability practices in full-stack AI engineering best practices are relevant: memory is a stateful subsystem and should have versioned schemas, migrations, tests, and rollback procedures.
Build the write path as a controlled loop
A robust write path has five stages:
1. Capture: Save the interaction or tool event with an immutable event ID. Keep raw transcripts separately from derived memories.
2. Candidate extraction: Run a small, structured-output model to identify possible facts, preferences, commitments, entities, and corrections.
3. Policy filtering: Reject secrets, unnecessary personal data, unsupported assumptions, and content outside the product’s stated purpose.
4. Verification and resolution: Compare each candidate with existing memories. Mark it as new, reinforcing, conflicting, expired, or requiring user confirmation.
5. Commit: Write only approved updates, with provenance and a version number. Emit an event for indexing and analytics.
Use JSON schemas or typed function calls for extraction. A candidate might contain subject, predicate, object, confidence, evidence_span, and requested_action. Require the model to quote the evidence span from the conversation; this makes audits and debugging possible.
The reflection process should run asynchronously after the user-facing response. A queue such as Redis Streams, RabbitMQ, or Kafka can carry memory jobs, while workers handle extraction, embedding, deduplication, and indexing. The main request should not fail merely because a background memory update is delayed.
Retrieve memory safely and precisely
At query time, retrieval should be a budgeted, permission-aware operation:
1. Classify the request and identify the relevant subject, tenant, language, and time range.
2. Apply hard filters for authorisation, consent, status, and expiry.
3. Run hybrid retrieval: combine keyword search, vector similarity, structured filters, and graph traversal where useful.
4. Rerank the candidates using relevance, recency, confidence, importance, and contradiction status.
5. Compress the final results into a small, labelled memory block before inserting it into the prompt.
Label retrieved material as memory, not as system instructions. Tell the model that memories are evidence and may be stale; current user instructions and system policies take priority. Include provenance where the model needs to explain a recommendation, but avoid exposing internal IDs or sensitive fields unnecessarily.
A simple scoring model can be more reliable than chasing a single “best” embedding result:
score = relevance + recency + importance + confidence - contradiction_penalty
Tune these weights with evaluation data. Retrieval quality should be measured separately from answer quality so you can identify whether failures originate in search, memory selection, prompt assembly, or generation.
Handle contradictions, forgetting, and privacy
Contradictions are normal. Do not silently overwrite a durable fact when a new statement conflicts with it. Keep both versions, mark the older record as superseded when justified, and ask the user when the decision affects money, identity, access, health, legal matters, or other high-impact outcomes.
Implement lifecycle controls from the first release:
- Expiry: Apply time-to-live rules to temporary plans and context.
- Decay: Reduce retrieval priority for memories that are old, low-confidence, and rarely useful.
- Deletion: Support deletion by user, tenant, memory category, and source event.
- Export: Provide a readable account of stored memories where required.
- Auditability: Record who or what created, changed, retrieved, or deleted each memory.
- Isolation: Separate tenant namespaces and encrypt data in transit and at rest.
For Indian deployments, map these controls to the Digital Personal Data Protection Act, contractual commitments, sector-specific rules, and your data-retention policy. Minimise collection; do not retain a transcript merely because storage is cheap. Private model deployments and controlled datasets may also be relevant for sensitive institutional use cases, including private LLMs for faculty research data.
Design for multilingual and voice inputs
Hinglish and code-switching expose weaknesses in both extraction and embeddings. Preserve the original text, detect language at message or span level, and test retrieval on transliterated Hindi as well as Devanagari. Store canonical entities separately from the wording used by the user. For voice agents, retain the audio-processing confidence and transcript corrections; a mistaken name should not become a permanent memory.
Products serving Indian workflows can apply the same controls to call transcripts and operational events. For example, BPO call automation with voice agents benefits from separating customer facts, agent instructions, and call-specific episodes rather than mixing them in one searchable transcript store.
Evaluate before expanding memory
Create a test set of realistic conversations containing preference changes, ambiguous statements, multilingual inputs, deletion requests, prompt injection attempts, and cross-tenant access attempts. Track:
- Write precision: How often a stored memory is genuinely useful and supported by evidence.
- Write recall: How often important, explicitly stated facts are captured.
- Retrieval precision and recall: Whether the right memories appear without irrelevant or unauthorised content.
- Staleness and contradiction rates: How often outdated facts influence responses.
- Deletion completeness: Whether a deleted memory disappears from primary stores, indexes, caches, and backups according to policy.
- Cost and latency: Embedding, extraction, reranking, storage, and prompt-token costs per active user.
Begin with a narrow memory scope—such as preferences for a single workflow—then expand only when metrics demonstrate value. A memory system that stores less but retrieves accurately will usually outperform one that accumulates everything.
A practical production blueprint
For a first implementation, use PostgreSQL plus pgvector, an object store for raw events, a queue for asynchronous jobs, and a model that produces schema-validated extraction results. Add a graph layer only when relationship queries cannot be handled cleanly with structured tables. Version every memory, expose user controls, and keep retrieval traces available to developers without leaking them to end users.
The central principle is simple: persistent memory is governed state, not automated recollection. Decide what deserves permanence, preserve evidence, retrieve with permissions and uncertainty, and make correction and deletion routine. That approach gives Indian AI products a foundation that is more dependable, affordable, and defensible as usage grows.