How to Give an LLM Agent Long-Term Memory

You give an LLM agent long-term memory by writing the durable parts of its work into an external store, then retrieving them by relevance at the start of each run instead of replaying the whole transcript. Everything else is detail on those two moves.

In practice it decomposes into four steps: decide what deserves to persist, type it so it can be queried, retrieve under a budget, and handle supersession. Each has a distinct failure mode, and skipping any one of them produces a system that looks like it has memory until it is used for a month.

The candidates are facts the agent established, decisions it took, constraints it discovered, and events that changed the state of the work. The non-candidates are the turns of dialogue in which those things happened.

The failure mode here is writing everything, on the reasoning that storage is cheap. Storage is cheap; retrieval under a token budget is not. A memory that contains everything forces every future read to rank a large pile of near-duplicates, and ranking is where the errors come from.

An untyped memory is a pile of strings, and a pile of strings can only be searched by similarity. Typing an entry as a decision rather than a fact lets a caller ask for the decisions on a project without competing against every sentence that happens to mention it.

Types also give you an honest place to put metadata that matters at retrieval time: which project, which agent wrote it, when, and what it supersedes. None of that is expressible in free text without the reader having to parse prose to recover it.

At the start of a run, the agent asks for what is relevant to the task it is about to do, and gets back a bounded set. The cap is not a performance optimisation, it is the mechanism that keeps the prompt from drifting back toward transcript size as the store grows.

The failure mode is an uncapped query that returns everything matching a project. It performs well on the first week of data and degrades continuously afterwards, which is the hardest kind of regression to notice because nothing ever breaks.

Work changes its own history. A decision taken in March is reversed in May, and a memory that returns both without ordering them hands the agent a contradiction to resolve on its own, which it will do inconsistently.

This is the step most home-built memories skip, and it is the one that makes the difference between a store that gets more useful over time and one that gets noisier. Versioning entries, or at minimum recording what an entry replaces, costs little at write time and removes an entire class of downstream confusion.

All four steps can be built on a database and an embedding index. Nothing here requires a product, and for a single agent on a single project the do-it-yourself version is often correct.

It stops being correct when several agents, sessions, or models have to read each other work, because then the typing and the supersession rules have to be shared rather than reimplemented per agent. Venkai implements this pattern as a service over REST, a Python SDK, and MCP, which is a few lines per agent rather than a subsystem to maintain.