Bob Smith wrote an article on agent memory that argues for an append-only log with a projected view on top. In the comments he asked me something I owe a long answer to:
Have you hit the case where a supersede and a later append disagree, and if so, which wins?
Short answer: the explicit one. Being later wins nothing by itself.
The long answer starts with an experiment that failed.
The obvious idea
Agent memory goes stale. A note says "deploys go through script X". Three weeks later script X is gone, and the note is still there, sounding exactly as sure as it did on day one.
So the obvious move is to let age count. Older notes rank lower, fresh ones rise. Most memory designs I've read reach for some version of that. So did I. I changed the ranking in my memory server: the older a lesson, the further down it goes.
What the benchmark said
My server has a benchmark that runs on every pull request. It's a set of questions with known right answers, over two collections: my own, and an external one shaped differently. Each metric has a floor. Dropping below it blocks the merge.
With age in the ranking:
"Right answer at rank 1 fell to 76.9 % against a floor of 90 %, mean reciprocal rank to 88.5 % against 94 %, nDCG@5 to 92.0 % against 96 %, and on the external set rank 1 fell to 67.9 % against 78 %."
Every floor broke. I closed the pull request.
Why
The right answers were often weeks old. That's what useful memory looks like: the deploy trap you hit in July is exactly the thing you need in October. The age penalty pushed those answers under fresh notes that were merely related.
My mistake was using age as a stand-in for "no longer true". They're properties of different things. Age belongs to the note. Whether it still holds belongs to the fact. A lesson can be two days old and wrong, or two years old and right, and a timestamp can't tell those apart.
"Two notes. One written two days ago, wrong since yesterday because the script it names was deleted. One written four months ago, still true. Ranking by age asks the clock; a correction that names its target asks the fact."
What decides instead
When two notes disagree, this is what decides it in my server, in plain terms:
- A correction names what it corrects. Then the old note moves aside and stays readable, with a banner pointing to its successor and the reason. Nothing vanishes.
- Rewriting a note needs a reason. One line on why the previous version was wrong, like a commit message. Without it, the write is rejected.
- Dates describe the fact, not the note. "True since" and "true until". An expired fact gets a loud marker and stays readable as history.
- A failed check outranks confidence. If someone tried a lesson and it didn't hold, that shows above the results, even on a note written yesterday.
- A contradiction without a marker shows both. A newer note that disagrees without saying so overwrites nothing. Both come back, and whoever reads them sees the disagreement.
So, Bob: an explicit supersede beats a later append. A later append on its own is one more voice.
The uncomfortable count
"Explicit beats implicit" has one catch. Someone has to write the explicit part.
In September I counted my own store. Of 735 lessons, 2 carried a replacement link. And those two pointed at each other.
"735 grey dots, two red ones linked to each other: 2 of 735 lessons carry a link to the lesson they replace."
Agents correct themselves constantly. They almost never say what they're correcting. When I looked into why, the answer was plain: the agent writing a correction rarely has the old note in front of it. And word overlap won't find the pairs for you. Real correction pairs share about as many words as unrelated lessons do.
So a design that waits for the writer to declare a supersede waits forever.
That's where Bob's other idea comes in: supersede-by-actor. A passing test that contradicts a model's note, or a human review that rejects it. Those actors know exactly which belief they just overturned. They're the ones who should write the link. I have half of it: a failed check already sits on top of the lesson it failed. The link from the check to the corrected version isn't written yet. That's the next piece.
Count it yourself
This works on any agent memory: a CLAUDE.md, a folder of notes, a vector store.
- Take your 20 newest entries.
- For each one, search for an older entry about the same thing.
- Count a: pairs that disagree. Count b: of those, how many say which one is current.
If b is much smaller than a, your memory settles every conflict by whatever the reader happens to see first.
I'd like to see your a and b. Mine, counted across the whole store: 2 explicit links in 735 lessons.
Where this stops
- One benchmark, mine, plus one external set. A gentler age curve might pass the floors. The one I tried didn't come close.
- The 2-in-735 count is from September 2026.
- "Explicit wins" moves the work to whoever writes the correction. Right now that work mostly doesn't get done, which is why the test-and-review route matters more than the rule.
Bob's question came out of a longer thread under Edward Izgorodin's Everything It Remembers Has the Same Authority. If you care about this problem, the comments there are worth the time. Thanks to @bob_smith_eaea75d3ee1cba3 for asking the question this post answers, and to @izgorodin for the thread it grew out of.
I build cachly — memory for AI coding assistants, over MCP. ChatGPT and Claude remember your conversations; cachly remembers your system: the bug you fixed, why you chose Postgres, the deploy step that always breaks. Every assistant reads the same memory, and every lesson carries the name of whoever learned it.
Try it, in increasing order of commitment:
-
30 seconds, no account —
npx @cachly-dev/mcp-server@latest demoin any git repo. It reads your log locally and prints what an assistant would already know about the project. After npx fetches the package, the command makes no network calls. -
Claude Code plugin —
/plugin marketplace add cachly-dev/cachly-mcp, then/plugin install cachly-brain@cachly. -
5 minutes, free tier —
npx @cachly-dev/mcp-server@latest autopilotwrites the MCP configuration for whichever assistant you use. - Or from the web — cachly.dev · free tier, German servers, no credit card.
Top comments (11)
"Age belongs to the note, whether it still holds belongs to the fact" is the sentence I'd put on the wall. We made the exact same mistake early — decayed memories by age and watched genuinely load-bearing lessons (the deploy trap from months ago) sink under fresh-but-irrelevant notes. The useful memory is almost always the old scar, not the recent chatter.
What's worked for us lines up with your supersede-over-append rule: a correction has to name what it corrects, the old note stays readable with a banner pointing at its successor and the reason, and nothing is ever silently deleted. I'd add emphasis to "a failed check outranks confidence" — an empirical "someone tried this and it broke" should beat a freshly written confident claim every time, regardless of age.
The part I'm still wrestling with: who writes the correction? If the agent self-supersedes on a single contradicting observation, you get thrash. Do you require a second confirmation before a supersede sticks, or does the first explicit correction win outright?
The first explicit one wins outright. There's no second confirmation.
What keeps it from thrashing is that a single contradicting observation never gets to supersede anything. It goes into a different lane. A failed check puts a warning above the lesson, and the next check that passes takes the warning off again. The lesson underneath stays where it was. A supersede is a separate, deliberate act: it has to name the lesson it replaces, and two lessons can't replace each other.
So the thrash you describe would show up as a flickering warning, and memory itself stays put. A wrong supersede is also cheaper than it sounds, because nothing gets deleted. The old text sits under a banner pointing at its successor, so the next reader can see the mistake and undo it.
So far my problem is the opposite of thrash. The explicit lane is almost empty, which is what the 2-in-735 count was about.
Age is the axis I'd price too. A memory injected into the prompt is paid each turn it stays there, so an old entry that never gets read is still a cost until it's dropped. Ranking by age gives you the cut line; the token bill tells you which entries earn their slot.
For memory that sits in the prompt on every turn, like a CLAUDE.md, I'd price it exactly that way. Every line pays rent on every turn.
Mine is picked per prompt instead. Apart from a short briefing when a session starts, each prompt pulls a few lessons by relevance, and the rest of the store never reaches the model. That shifts the cost question. An old lesson that's never retrieved costs nothing. The expensive one is an old lesson that keeps getting retrieved and is wrong, and its age can't tell you which of the two you're looking at.
@heinrichneb age can't tell the two apart, so we don't rank by it. What tells them apart is whether the lesson still matches the case it describes when it gets retrieved. We keep the evidence count next to each rule, so a rule that keeps coming back can be checked against the cases behind it, and retired on that same evidence.
The evidence count is where mine went wrong for a while.
Until September, every retrieval refreshed a lesson's "last verified" mark. A rule that kept coming back looked better confirmed each time, just for being shown, which is the opposite of what the count is for. Now only a check that actually ran moves it.
The other trap runs the other way. A failed check isn't always evidence against the rule. Sometimes staging timed out and the claim was never tested at all. If retirement runs on that evidence, one bad day retires a good rule. So a failed check now records which of the two it was, and only "the claim no longer holds" counts against the lesson.
Do the cases behind your count include the ones where a rule was retrieved, acted on, and nobody looked at what happened next? Most of mine are like that. Mine don't count them as evidence at all, so the evidence stays thin, but every case in it was actually looked at.
The 2-in-735 explicit link count explains why word-overlap deduplication misses so many contradictions. In plain text, "deploy with script X" and "run make target Y" share almost no vocabulary, so two agents writing weeks apart never collide unless entries are indexed by a shared artifact key like a file path, binary name, or CI workflow.
One catch with letting failed checks outrank confidence is separating genuine stale notes from transient environment failures. If a test run fails because a staging endpoint timed out or a lockfile lagged, flagging the note as failed can bury a valid rule behind a temporary outage. Recording the git commit SHA and error class alongside the check result lets the store expire advice only on deterministic assertion or command failures.
Your own example is where I'd test that. "Deploy with script X" and "run make target Y" share no file path either: X is gone, and Y replaced it. What they share is the job, deploying the same service. So my guess is that the key which catches these pairs is the goal or the CI workflow, and file paths catch only the corrections that touch the same file. That's a cheap count on my store, and it's the next one I want to run.
On transient failures: in my setup a failed check only adds a warning above the lesson. The lesson stays readable underneath, so a staging timeout costs the next reader a moment of doubt, and the rule survives. Your point lands on the reader's side: an agent that sees "failed" next to a rule may well skip it. The error class is the cheapest fix for that. "Assertion failed" is news about the note. "Staging timed out" is news about the day.
The benchmark result separates two clocks that one timestamp mixes: when a note was written and when it last turned out to be right. The deploy trap from July loses on the first clock and wins on the second every time someone hits that deploy again. A penalty on time since the last confirmed use, rather than time since writing, would keep a lesson that keeps getting used near the top and still sink one nobody has needed for months, which is closer to what the age idea was reaching for. Whether that version clears your floors is the run I would most want to see.
On the 2 in 735: if the writer rarely has the old note in front of it, the write path can put it there. Before a lesson is stored, run the same search a reader would run for its subject and show the writer the few closest older lessons with one question, whether the new one replaces any of them. That costs one read per write and turns the supersede from something the writer has to remember into something it is asked about, which leaves the test and review route for the pairs the search does not surface.
I wanted your second clock and used the first, because the first was the only one I had.
I can't run your version yet, and the reason is a little embarrassing. My closest thing to a second clock was a "last verified" timestamp, and until early September every read set it. Most lessons looked freshly confirmed just because they had been shown. Shown isn't confirmed. Since then reads leave it alone, so after the write it only moves when a check actually holds, and those are rare. A penalty on that clock today would mostly punish lessons nobody got around to checking.
On the write path, mine already does a narrow version of what you describe. It asks in two cases: when the new text sounds like a correction, and when it says almost the same thing as an older lesson. Either way it names one candidate. Yours asks on every write, with the reader's search, and shows a few. That catches the pair both of mine miss: a correction that neither sounds like one nor repeats the old wording. Reid's "deploy with script X" against "run make target Y" is exactly that pair, and the post's own finding says it's the common case.
@heinrichneb Not if nobody looked. A retrieval is an event, a case is a retrieval plus what happened after it. Count the retrievals and the count climbs each time the rule gets shown, which is the same bug as refreshing "last verified" on every read.