Bigger Context Windows Are Not Memory

Context windows have grown from a few thousand tokens to hundreds of thousands, and in some models past a million. That growth genuinely removes a real constraint: a model can now read an entire codebase, a long contract, or a full support thread in one pass instead of needing it pre-summarized or chunked. That is useful, and it is also the entire benefit — it is a statement about how much can be read in a single request, not about what survives after the request ends.

Nothing about window size changes what happens when the session ends: the tokens are discarded, and the next session starts with an empty window regardless of how large that window is capable of being. A million-token window used once and then dropped has stored exactly as much between sessions as a four-thousand-token window used once and then dropped — zero. Persistence is a property of something outside the model that decides what to put back in the window next time, and window size says nothing about whether that something exists.

A larger window also does not solve selection. Handing a model more tokens than it needs does not make it better at knowing which of those tokens matter to the current question — it can even make performance worse, since more competing information gives the model more chances to weight the wrong part of it, or to treat a stale document sitting next to a current one as equally authoritative. A bigger window can hold an entire policy's revision history at once; it cannot on its own tell you which revision is the one still in force.

Persistence and relevance are supplied by something that sits outside any single inference call: a system that decides, per request, what this company currently knows that's relevant here, and keeps that judgment current as the business changes. That is an infrastructure problem — building and maintaining a structure of what's true now and how it relates to what came before — not a model-capacity problem, and it does not get smaller as context windows get larger. It is the specific gap a memory layer like Venkai is built to fill, independent of how much any given model can read in one sitting.