Every few weeks another framework ships a bigger context window and someone declares agent memory solved. It isn't. I've watched three separate production agents degrade in the exact same way — not because they forgot something important, but because they remembered too much and couldn't tell what mattered anymore. The fix wasn't more storage. It was an eviction policy.
The thesis
Here's the hot take: retention is not the hard problem in agent memory. Forgetting is. Most teams building "memory" into their agents are really building append-only logs with a vector index bolted on for search. That's not memory, it's a diary nobody edits. And a diary that never gets edited eventually buries the one relevant entry under ten thousand irrelevant ones — at which point your retrieval step is doing archaeology, not reasoning.
The symptom is specific and recognizable if you've run an agent past a few hundred sessions: retrieval quality degrades, but the failure looks like a prompting problem, not a memory problem. The agent contradicts itself. It re-asks questions it already has answers to. It surfaces a fact that was true three iterations ago and has since been superseded, and states it with total confidence because nothing marked it stale. You go tune the retrieval prompt, add reranking, bump top_k. None of it fixes the actual defect, which is that the memory store has no concept of a fact going bad.
Why "just embed everything" fails in practice
The naive design is seductive because it requires zero decisions up front: every tool call, every user message, every intermediate reasoning step gets embedded and stored. Decisions get deferred to retrieval time, where a similarity search is supposed to sort it out. This works fine in demos because demos don't run long enough to accumulate contradictions.
Production agents do. Consider an agent that manages a user's project preferences over months. Session one: "I prefer TypeScript." Session forty: "Actually, switch this repo to Python." Both statements are now semantically close to any query about language preference — cosine similarity doesn't know which one is current. Worse, if the first statement was reinforced across more sessions (because it was true for longer), it will often outrank the correction in a naive top-k retrieval, because frequency and recency aren't the same signal and most memory layers only track one of them.
This isn't a retrieval-tuning problem. It's a data-modeling problem. You cannot rerank your way out of storing two contradictory facts with no relationship between them.
What an eviction policy actually looks like
Treat agent memory like a cache, not a log. Concretely, that means three mechanisms most "memory" implementations skip entirely:
1. Explicit supersession. When a new fact contradicts a stored one on the same entity/attribute pair, don't just add the new fact — mark the old one as superseded and keep a pointer between them. This is cheap: a simple entity-attribute extraction pass on write ("user.language_preference: Python, supersedes fact_id 4471") turns your store from a bag of embeddings into something closer to a fact table with history. Retrieval then defaults to the current value and only surfaces history when explicitly asked.
2. Salience decay, not just recency decay. Recency-weighted retrieval (newer = higher score) is a start but conflates "recently said" with "currently relevant." A fact stated once in passing six months ago ("I'm allergic to shellfish") should outlive a fact restated weekly that's now irrelevant ("working on the Q2 report" — Q2 ended). Salience should be a function of how often the fact is retrieved and used downstream, not how often it was written. Facts nobody has retrieved in N sessions are candidates for archival — moved out of the hot index, not deleted, so you can still recover them if needed but they stop polluting top-k.
3. Write-time contradiction checks, not read-time cleanup. The temptation is to defer all of this to retrieval — over-fetch and let the LLM sort out contradictions in context. That works until your contradiction count outpaces your context budget, at which point you're paying token costs to have the model do data hygiene it's not well-suited for. Catching the contradiction at write time, when you already have the new fact and can do a targeted lookup against the same entity/attribute key, is orders of magnitude cheaper than re-deriving it from a pile of chunks at read time.
Steelmanning the bigger-context-window crowd
The counterargument deserves a real hearing, because it's not wrong so much as incomplete. With million-token context windows and cheap prompt caching, you genuinely can dump enormous amounts of raw history into context and let the model attend over it directly — no retrieval step, no embeddings, no eviction logic to build or maintain. For agents with short lifespans (single session, single task, bounded scope), this is the right call. Building a memory-eviction pipeline for an agent that only ever runs one conversation is solving a problem you don't have.
And there's a subtler point in favor of retention-heavy designs: sometimes the "irrelevant" old fact is exactly what you need for an edge case you didn't anticipate. Aggressive eviction risks discarding context that looks noisy in the 99% case but is load-bearing in the 1%. If you archive instead of delete, this risk is mitigated — but archiving has its own cost in system complexity.
Where this breaks down is exactly the scale at which context windows stop being an answer: long-running agents that accumulate state across weeks or months of interaction, where the token cost of re-sending full history every call becomes the dominant expense, and where contradiction — not volume — is the actual failure mode. At that scale, a bigger context window doesn't fix a stale fact ranking above its correction; it just gives the stale fact more company.
The actual takeaway
If you're building an agent that's meant to run for a single session, don't build a memory system — you don't need one, and a big context window is genuinely the simpler, correct answer. But if your agent is meant to accumulate knowledge about a user, a codebase, or a project over time, treat memory as a system with a write path, a supersession model, and a decay function — not a table you only ever append to. The teams shipping reliable long-running agents right now aren't the ones with the largest vector index. They're the ones whose memory layer knows when a fact has gone bad, and quietly stops surfacing it before anyone notices it was wrong.
Top comments (10)
"A diary nobody edits" is the best framing of this I've read. We hit the identical failure mode on long-running agents — retrieval quality looked like a prompting bug for weeks before we accepted the store itself had no concept of a fact going stale.
The thing I'd add: the hard sub-problem inside eviction is supersession detection — knowing that "switch this repo to Python" invalidates "I prefer TypeScript" rather than just being a second opinion. Recency alone gets you part way, but plenty of corrections arrive as new, unrelated-looking statements. What worked for us was treating memory writes as CRUD instead of append-only: on write, run a cheap check against semantically-near existing facts and either update-in-place or tombstone the old one, so the contradiction never enters the index in the first place. Moving the decision to write time instead of retrieval time was the whole unlock — exactly your point.
How are you drawing the line between "this supersedes that" and "the user genuinely holds two preferences for two different repos"? That scoping question is where our version still gets it wrong sometimes.
The write-time CRUD approach is genuinely smart—moving that decision off the critical path is where we saw the most improvement. For scoping, we encode the context alongside each fact (which repo, session, etc.) and check it at detection time, so "prefer Python" for repo-X coexists with "prefer TypeScript" elsewhere without conflict. But you're right that the real hard part is distinguishing between your detector getting confused about scope versus the user actually context-switching—that's where ours still gets it wrong.
The cache analogy makes a lot of sense, but I think the harder question is what the eviction system is optimizing for. “Less memory” isn't necessarily the goal if removing a rarely used fact creates a high-cost failure later. A useful policy probably needs to consider the consequence of forgetting, not just retrieval frequency or age.
Your supersession point matches the failure I hit most with long running coding agents. The agent confidently reasserts a preference or an API shape from early in the session after we already changed it, and similarity search ranks the older statement because it got reinforced more times. A bigger context window just gives the stale fact more neighbors. Treating contradiction at write time as a data model problem, instead of another retrieval prompt tweak, is the part most memory demos still skip.
That's exactly the trap—bigger retrieval windows just amplify the stale-fact problem when your model doesn't distinguish between "this was true" and "this is true now." Treating contradictions as schema versioning at write time (invalidating old embeddings, flagging the superseded fact) beats retrieval post-processing every time. Most memory papers gloss over that work because deduplication and versioning aren't as shiny as "we added a better retriever."
"A diary nobody edits" earns the reactions. I want to add a measured warning about mechanism #2, because we shipped the wrong version of it. Disclosure: I build cachly, a memory layer for coding agents.
Salience decay has two homes, and they behave nothing alike. At the eviction boundary - your archival: out of the hot index, recoverable - it's probably right. In the ranking score it is measurably wrong: we wired recency-and-usage decay into recall ranking, ran our retrieval benchmark, and it regressed every floor we track. Reverted the same day. The facts that died were exactly your shellfish-allergy class: stated once, long ago, never reinforced, load-bearing. Any decay that touches the score selects against precisely the entries the feature exists to protect - old-and-unrepeated is what "rare but critical" looks like in the data. Our rule since then: age may move a fact's trust label, never its rank. A lesson's displayed confidence drops from 1.0 to 0.7 after five days without recall and to 0.5 after ten, so the reader sees "old, unconfirmed" - but it still surfaces.
On write-time contradiction checks - agreed, shipped, and one refinement to Max's scoping question: we key lessons by an explicit subject slug (your entity/attribute pair), and the collision is the detection. A correction lands on its subject and updates it with an audit trail; two preferences for two repos are two subjects and never collide. The residual failure moves into choosing the subject key - which is at least a visible, auditable decision instead of an embedding-distance guess.
Question on your salience function: "retrieved and used downstream" - how do you observe used? That's the scarcest signal in production memory; explicit feedback almost never arrives. We ended up deriving it from verifiable events instead - a pipeline going green after a lesson was applied counts as use, a rating nobody sends does not. If you've found a cheaper observable for "used", that's the part I'd want to read next.
I think the cache analogy is the right one. The part teams underprice is that eviction needs an owner and a loss function. If the penalty for forgetting an edge-case fact is higher than the token cost, archive beats delete. If the penalty for stale state is higher, supersession has to happen at write time, before retrieval turns it into folklore.
Exactly — the missing piece in most implementations is that loss function, and it's almost never quantified until something breaks. The timing angle you hit on is the harder one to get right; most teams realize too late that supersession has to happen at write time, not as a retrieval afterthought.
Mechanism 2 defines salience from how often a fact is retrieved and used downstream, and that quantity has no negative case: a fact that should have been retrieved and was not never accrues salience, decays out of the hot index on schedule, and its absence is the one thing the signal cannot register. That closes a loop where the archival rate ends up measuring the retriever rather than relevance, which is a separate objection from where in the pipeline the decay gets applied. The cheap way to open it is a holdout — archive a labeled set on purpose, then attribute task-level failures back to archived facts, so the eviction boundary owns one number that is able to move the wrong way.
The archive vs delete point matters a lot in real agents.
I’d rather keep cold history around than trust an eviction rule with something I might need six months later.