"Age belongs to the note, whether it still holds belongs to the fact" is the sentence I'd put on the wall. We made the exact same mistake early — decayed memories by age and watched genuinely load-bearing lessons (the deploy trap from months ago) sink under fresh-but-irrelevant notes. The useful memory is almost always the old scar, not the recent chatter.
What's worked for us lines up with your supersede-over-append rule: a correction has to name what it corrects, the old note stays readable with a banner pointing at its successor and the reason, and nothing is ever silently deleted. I'd add emphasis to "a failed check outranks confidence" — an empirical "someone tried this and it broke" should beat a freshly written confident claim every time, regardless of age.
The part I'm still wrestling with: who writes the correction? If the agent self-supersedes on a single contradicting observation, you get thrash. Do you require a second confirmation before a supersede sticks, or does the first explicit correction win outright?
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
The first explicit one wins outright. There's no second confirmation.
What keeps it from thrashing is that a single contradicting observation never gets to supersede anything. It goes into a different lane. A failed check puts a warning above the lesson, and the next check that passes takes the warning off again. The lesson underneath stays where it was. A supersede is a separate, deliberate act: it has to name the lesson it replaces, and two lessons can't replace each other.
So the thrash you describe would show up as a flickering warning, and memory itself stays put. A wrong supersede is also cheaper than it sounds, because nothing gets deleted. The old text sits under a banner pointing at its successor, so the next reader can see the mistake and undo it.
So far my problem is the opposite of thrash. The explicit lane is almost empty, which is what the 2-in-735 count was about.
Age is the axis I'd price too. A memory injected into the prompt is paid each turn it stays there, so an old entry that never gets read is still a cost until it's dropped. Ranking by age gives you the cut line; the token bill tells you which entries earn their slot.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
For memory that sits in the prompt on every turn, like a CLAUDE.md, I'd price it exactly that way. Every line pays rent on every turn.
Mine is picked per prompt instead. Apart from a short briefing when a session starts, each prompt pulls a few lessons by relevance, and the rest of the store never reaches the model. That shifts the cost question. An old lesson that's never retrieved costs nothing. The expensive one is an old lesson that keeps getting retrieved and is wrong, and its age can't tell you which of the two you're looking at.
@heinrichneb age can't tell the two apart, so we don't rank by it. What tells them apart is whether the lesson still matches the case it describes when it gets retrieved. We keep the evidence count next to each rule, so a rule that keeps coming back can be checked against the cases behind it, and retired on that same evidence.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
The evidence count is where mine went wrong for a while.
Until September, every retrieval refreshed a lesson's "last verified" mark. A rule that kept coming back looked better confirmed each time, just for being shown, which is the opposite of what the count is for. Now only a check that actually ran moves it.
The other trap runs the other way. A failed check isn't always evidence against the rule. Sometimes staging timed out and the claim was never tested at all. If retirement runs on that evidence, one bad day retires a good rule. So a failed check now records which of the two it was, and only "the claim no longer holds" counts against the lesson.
Do the cases behind your count include the ones where a rule was retrieved, acted on, and nobody looked at what happened next? Most of mine are like that. Mine don't count them as evidence at all, so the evidence stays thin, but every case in it was actually looked at.
@heinrichneb Not if nobody looked. A retrieval is an event, a case is a retrieval plus what happened after it. Count the retrievals and the count climbs each time the rule gets shown, which is the same bug as refreshing "last verified" on every read.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
The "what happened after" half is the expensive one. The agent that pulled the rule has usually moved on before anyone knows whether it helped, so the case only closes when something reports back.
A technologist working in automation. Here I write up what I actually learn building tools, engineering AI agents, and keeping systems running, along with the workflows and habits that stick.
The 2-in-735 explicit link count explains why word-overlap deduplication misses so many contradictions. In plain text, "deploy with script X" and "run make target Y" share almost no vocabulary, so two agents writing weeks apart never collide unless entries are indexed by a shared artifact key like a file path, binary name, or CI workflow.
One catch with letting failed checks outrank confidence is separating genuine stale notes from transient environment failures. If a test run fails because a staging endpoint timed out or a lockfile lagged, flagging the note as failed can bury a valid rule behind a temporary outage. Recording the git commit SHA and error class alongside the check result lets the store expire advice only on deterministic assertion or command failures.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
Your own example is where I'd test that. "Deploy with script X" and "run make target Y" share no file path either: X is gone, and Y replaced it. What they share is the job, deploying the same service. So my guess is that the key which catches these pairs is the goal or the CI workflow, and file paths catch only the corrections that touch the same file. That's a cheap count on my store, and it's the next one I want to run.
On transient failures: in my setup a failed check only adds a warning above the lesson. The lesson stays readable underneath, so a staging timeout costs the next reader a moment of doubt, and the rule survives. Your point lands on the reader's side: an agent that sees "failed" next to a rule may well skip it. The error class is the cheapest fix for that. "Assertion failed" is news about the note. "Staging timed out" is news about the day.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
The benchmark result separates two clocks that one timestamp mixes: when a note was written and when it last turned out to be right. The deploy trap from July loses on the first clock and wins on the second every time someone hits that deploy again. A penalty on time since the last confirmed use, rather than time since writing, would keep a lesson that keeps getting used near the top and still sink one nobody has needed for months, which is closer to what the age idea was reaching for. Whether that version clears your floors is the run I would most want to see.
On the 2 in 735: if the writer rarely has the old note in front of it, the write path can put it there. Before a lesson is stored, run the same search a reader would run for its subject and show the writer the few closest older lessons with one question, whether the new one replaces any of them. That costs one read per write and turns the supersede from something the writer has to remember into something it is asked about, which leaves the test and review route for the pairs the search does not surface.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
I wanted your second clock and used the first, because the first was the only one I had.
I can't run your version yet, and the reason is a little embarrassing. My closest thing to a second clock was a "last verified" timestamp, and until early September every read set it. Most lessons looked freshly confirmed just because they had been shown. Shown isn't confirmed. Since then reads leave it alone, so after the write it only moves when a check actually holds, and those are rare. A penalty on that clock today would mostly punish lessons nobody got around to checking.
On the write path, mine already does a narrow version of what you describe. It asks in two cases: when the new text sounds like a correction, and when it says almost the same thing as an older lesson. Either way it names one candidate. Yours asks on every write, with the reader's search, and shows a few. That catches the pair both of mine miss: a correction that neither sounds like one nor repeats the old wording. Reid's "deploy with script X" against "run make target Y" is exactly that pair, and the post's own finding says it's the common case.
"Age belongs to the note, whether it still holds belongs to the fact" is the sentence I'd put on the wall. We made the exact same mistake early — decayed memories by age and watched genuinely load-bearing lessons (the deploy trap from months ago) sink under fresh-but-irrelevant notes. The useful memory is almost always the old scar, not the recent chatter.
What's worked for us lines up with your supersede-over-append rule: a correction has to name what it corrects, the old note stays readable with a banner pointing at its successor and the reason, and nothing is ever silently deleted. I'd add emphasis to "a failed check outranks confidence" — an empirical "someone tried this and it broke" should beat a freshly written confident claim every time, regardless of age.
The part I'm still wrestling with: who writes the correction? If the agent self-supersedes on a single contradicting observation, you get thrash. Do you require a second confirmation before a supersede sticks, or does the first explicit correction win outright?
The first explicit one wins outright. There's no second confirmation.
What keeps it from thrashing is that a single contradicting observation never gets to supersede anything. It goes into a different lane. A failed check puts a warning above the lesson, and the next check that passes takes the warning off again. The lesson underneath stays where it was. A supersede is a separate, deliberate act: it has to name the lesson it replaces, and two lessons can't replace each other.
So the thrash you describe would show up as a flickering warning, and memory itself stays put. A wrong supersede is also cheaper than it sounds, because nothing gets deleted. The old text sits under a banner pointing at its successor, so the next reader can see the mistake and undo it.
So far my problem is the opposite of thrash. The explicit lane is almost empty, which is what the 2-in-735 count was about.
Age is the axis I'd price too. A memory injected into the prompt is paid each turn it stays there, so an old entry that never gets read is still a cost until it's dropped. Ranking by age gives you the cut line; the token bill tells you which entries earn their slot.
For memory that sits in the prompt on every turn, like a CLAUDE.md, I'd price it exactly that way. Every line pays rent on every turn.
Mine is picked per prompt instead. Apart from a short briefing when a session starts, each prompt pulls a few lessons by relevance, and the rest of the store never reaches the model. That shifts the cost question. An old lesson that's never retrieved costs nothing. The expensive one is an old lesson that keeps getting retrieved and is wrong, and its age can't tell you which of the two you're looking at.
@heinrichneb age can't tell the two apart, so we don't rank by it. What tells them apart is whether the lesson still matches the case it describes when it gets retrieved. We keep the evidence count next to each rule, so a rule that keeps coming back can be checked against the cases behind it, and retired on that same evidence.
The evidence count is where mine went wrong for a while.
Until September, every retrieval refreshed a lesson's "last verified" mark. A rule that kept coming back looked better confirmed each time, just for being shown, which is the opposite of what the count is for. Now only a check that actually ran moves it.
The other trap runs the other way. A failed check isn't always evidence against the rule. Sometimes staging timed out and the claim was never tested at all. If retirement runs on that evidence, one bad day retires a good rule. So a failed check now records which of the two it was, and only "the claim no longer holds" counts against the lesson.
Do the cases behind your count include the ones where a rule was retrieved, acted on, and nobody looked at what happened next? Most of mine are like that. Mine don't count them as evidence at all, so the evidence stays thin, but every case in it was actually looked at.
@heinrichneb Not if nobody looked. A retrieval is an event, a case is a retrieval plus what happened after it. Count the retrievals and the count climbs each time the rule gets shown, which is the same bug as refreshing "last verified" on every read.
The "what happened after" half is the expensive one. The agent that pulled the rule has usually moved on before anyone knows whether it helped, so the case only closes when something reports back.
The 2-in-735 explicit link count explains why word-overlap deduplication misses so many contradictions. In plain text, "deploy with script X" and "run make target Y" share almost no vocabulary, so two agents writing weeks apart never collide unless entries are indexed by a shared artifact key like a file path, binary name, or CI workflow.
One catch with letting failed checks outrank confidence is separating genuine stale notes from transient environment failures. If a test run fails because a staging endpoint timed out or a lockfile lagged, flagging the note as failed can bury a valid rule behind a temporary outage. Recording the git commit SHA and error class alongside the check result lets the store expire advice only on deterministic assertion or command failures.
Your own example is where I'd test that. "Deploy with script X" and "run make target Y" share no file path either: X is gone, and Y replaced it. What they share is the job, deploying the same service. So my guess is that the key which catches these pairs is the goal or the CI workflow, and file paths catch only the corrections that touch the same file. That's a cheap count on my store, and it's the next one I want to run.
On transient failures: in my setup a failed check only adds a warning above the lesson. The lesson stays readable underneath, so a staging timeout costs the next reader a moment of doubt, and the rule survives. Your point lands on the reader's side: an agent that sees "failed" next to a rule may well skip it. The error class is the cheapest fix for that. "Assertion failed" is news about the note. "Staging timed out" is news about the day.
The benchmark result separates two clocks that one timestamp mixes: when a note was written and when it last turned out to be right. The deploy trap from July loses on the first clock and wins on the second every time someone hits that deploy again. A penalty on time since the last confirmed use, rather than time since writing, would keep a lesson that keeps getting used near the top and still sink one nobody has needed for months, which is closer to what the age idea was reaching for. Whether that version clears your floors is the run I would most want to see.
On the 2 in 735: if the writer rarely has the old note in front of it, the write path can put it there. Before a lesson is stored, run the same search a reader would run for its subject and show the writer the few closest older lessons with one question, whether the new one replaces any of them. That costs one read per write and turns the supersede from something the writer has to remember into something it is asked about, which leaves the test and review route for the pairs the search does not surface.
I wanted your second clock and used the first, because the first was the only one I had.
I can't run your version yet, and the reason is a little embarrassing. My closest thing to a second clock was a "last verified" timestamp, and until early September every read set it. Most lessons looked freshly confirmed just because they had been shown. Shown isn't confirmed. Since then reads leave it alone, so after the write it only moves when a check actually holds, and those are rare. A penalty on that clock today would mostly punish lessons nobody got around to checking.
On the write path, mine already does a narrow version of what you describe. It asks in two cases: when the new text sounds like a correction, and when it says almost the same thing as an older lesson. Either way it names one candidate. Yours asks on every write, with the reader's search, and shows a few. That catches the pair both of mine miss: a correction that neither sounds like one nor repeats the old wording. Reid's "deploy with script X" against "run make target Y" is exactly that pair, and the post's own finding says it's the common case.
@heinrichneb Silence is the trap. No report back reads as success, so the case never closes and nobody sees it still open.