What Is AI Agent Memory?

Memory for an AI agent is anything the agent can retrieve about past work that is not already in its current context window. That definition is deliberately mechanical, because the mechanism is the whole point: a language model holds no state between calls, so everything it appears to remember was placed in the prompt by something outside the model.

This is why memory is an architecture question rather than a model question. A more capable model reasons better over what it is given. It does not change what it is given.

Conversation history is the record of what was said. It is the easiest to keep and the least useful to re-read: it grows without bound, and the information it contains is buried in the exchange that produced it.

Document retrieval is the record of what is written somewhere. A corpus is prepared in advance by humans, indexed, and searched at query time. It answers questions about a body of knowledge that exists independently of the agent.

Operational state is the record of what has been decided, done, and constrained. It is written by the agents themselves as work proceeds, it is small relative to the transcript that produced it, and it is the thing another agent needs in order to continue rather than restart.

Most systems that describe themselves as having memory have one of the first two. Chat history is kept and replayed, or a vector index is bolted on and consulted. Both are real capabilities, and neither answers the question an agent picking up someone else work has to ask: what has already been decided here, and what is still open?

That question is answered by operational state, and operational state is the kind that has to be produced deliberately. Nothing generates it as a side effect. An agent that is never asked to record a decision will not have recorded one, no matter how much of its transcript is retained.

RAG indexes documents and answers what information exists here. Agent memory keeps the state of work in progress and answers what has already been decided, and what is left to do. The distinction is not about the storage technology, which is often the same vector index in both cases. It is about who writes and when.

A RAG corpus is authored ahead of time and is stable between queries. A memory is authored by agents at runtime and changes as the work changes, which means it also has to represent supersession: a fact that was true last week and is not true now. A document index has no natural way to say that.

The two layers coexist. A system can retrieve a policy document from a corpus and, separately, recall that the team decided to deviate from that policy for one client three sessions ago.

Storing more is not the goal, and it is usually the failure mode. Every item an agent writes has to be re-read by some future agent under a token budget, so the useful memory is the small typed one, not the exhaustive one.

Venkai is a store for the third kind. It holds facts, decisions, constraints, preferences, events and relationships as typed objects, retrievable by relevance across sessions and across models, over REST, a Python SDK, or MCP. It orchestrates nothing and replaces no document index.