Leaking ephemeral container IDs into tool descriptions isn’t just an idempotency nitpick—it silently wrecks prompt caching at the inference layer. When tool schemas drift across runs, your KV cache hit rate drops to zero, turning what should be a low-latency warm invocation into a full prefix recomputation that inflates TTFT and inference spend. Runtime identifiers belong strictly in telemetry spans or transport headers, keeping the model-facing tool contract strictly deterministic.
that's an angle i hadn't measured. if the tool definitions sit in the prompt prefix, a different id on every new container would change it, and a prefix cache would miss. all we saw is the description changing between two runs in docker, nothing on the model side, so i can't say how much it costs in practice. it is a good reason to keep run-specific values out of descriptions though.
GMT strategy and custom B2B outreach lists on request: people and companies by ICP, domain, profile URL, email or name. 1B+ profiles. Claude Code plugins, API, MCP. [email protected]
With Claude Code the definitions are deferred, so a drifting description only costs from the turn it gets loaded. With eager loading your prefix point holds. I have not measured the cache miss itself.
Product leader and systems analyst in fintech and edtech. Founder of AnalystCraft. Writing about requirements, acceptance criteria, agent evaluation, and Test-Driven Product Development.
I'd keep the raw descriptions alongside the normalized hash. A hostname inside explanatory text can be incidental, but a hostname or path identifying the actual target can change what a tool does. A small paired test could change only the container ID in one case and the permitted target in another: normalization should suppress the first difference without hiding the second. Would a server-specific normalization rule, with a visible raw diff, fit your cards better than stripping all hostnames and paths?
Fair point. We do keep the raw hash: the normalized one only swaps 12-character hex strings (the container ID) and sits next to the raw one, it never replaces it. But stripping every hostname and path would hide a changed target, you're right, and I wrote that too loosely in the post. Your paired test is a nice idea, I haven't run it. Might try it with a dummy server.
@innokentyb ran the paired test. synthetic server with one tool, three variants: a base, one where only a container-style id changes in the description, and one where only the allowed target changes in the input schema. changing the id moved the description hash and the combined one. replacing 12-character hex strings made all three hashes identical again, and the schema hash never moved. changing the target left the descriptions alone and moved the schema hash, raw and normalized. so for this case the normalization removes the incidental difference without hiding the real one. it's a toy, one tool, nothing like a real server. the script and the raw comparisons will be in the public repo soon.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
The pass criterion has a protocol revision built into it. Answering initialize with a valid protocol version is how every session opens under the revisions up to 2025-11-25. The current revision, 2026-07-28, removes the initialize and notifications/initialized handshake: every request carries its protocol version, and servers must implement server/discover, which returns the versions and capabilities a server supports and its identity. A server built only for the new revision rejects initialize with an error, so a probe that starts with it would record a failure that belongs to the probe rather than to the server.
Sending server/discover first and falling back to initialize when it errors or times out is the order the spec gives for stdio clients that speak both eras, and it keeps a pass meaning what the post says it means whichever revision the server speaks: installed, started, answered with a valid protocol version and listed its tools. Recording on each card which of the two the server answered, and the version it returned, would also show how many of the ten passes would read the same to a client that speaks only the new revision.
you're right, and i checked the spec before answering. our probe only sends initialize, so a server built only for 2026-07-28 would show up as a failure that belongs to the probe, not the server. everything we recorded answered initialize, so those speak the older era or both. i've changed the probe to send server/discover first and fall back to initialize like the stdio section says, and to record which one answered and the version. so far i've only tried it on a toy server, i'll rerun the real ones soon and put the era on each card.
@izgorodin ran it. 27 servers: 12 answer server/discover, 10 error and then work with initialize, 4 give no answer in 20 s and work with initialize, and one failed on an npm error. for 11 of the 12 modern ones the tool list is identical either way. server-everything isn't: 16 tools with initialize, 22 with discover, same version. one run, one machine, so i wouldn't generalize.
Building persistent memory for AI agents at Mnemoverse.
I write about the unglamorous half: forgetting, contradiction, and the moment an agent confidently repeats a fact you retracted.
The server-everything split is the result worth putting on its card, because it makes the era part of what a hash means. Same version, 16 tools after initialize and 22 after server/discover: a stability hash over tools/list now has to say which handshake produced the list, or a client that moves to the new revision reads as a changed server while nothing in the package changed. Hashing per era and comparing like with like keeps the Desktop Commander lesson intact on the new axis.
The four that stayed silent for 20 seconds are the case the spec describes for older servers, which may answer server/discover with an error or not at all. Across a directory that is the slow path of every probe, and recording how long each server took to fall back would show whether a shorter wait would mislabel any of them.
agreed on the card. the 22 tool list sits next to the 16-tool one as its own record, with the era on each, so the hash says which handshake made it. it goes live with the next deploy. fair point on the 20 secns too: we only logged total time per run, not how long each server took to answer discover or fall back, so i can't tell yet if a shorter wait would mislabel any. i've added it to the probe.
yeah, that was my first reaction too. the other 24 descriptions and all the schemas were identical though, and the only difference was a 12-character container id in one description, so the rest of the run held up. still a good reason to look at the raw text before trusting a hash.
Leaking ephemeral container IDs into tool descriptions isn’t just an idempotency nitpick—it silently wrecks prompt caching at the inference layer. When tool schemas drift across runs, your KV cache hit rate drops to zero, turning what should be a low-latency warm invocation into a full prefix recomputation that inflates TTFT and inference spend. Runtime identifiers belong strictly in telemetry spans or transport headers, keeping the model-facing tool contract strictly deterministic.
Bohdan Hordii // bohdanhordii.dev
that's an angle i hadn't measured. if the tool definitions sit in the prompt prefix, a different id on every new container would change it, and a prefix cache would miss. all we saw is the description changing between two runs in docker, nothing on the model side, so i can't say how much it costs in practice. it is a good reason to keep run-specific values out of descriptions though.
With Claude Code the definitions are deferred, so a drifting description only costs from the turn it gets loaded. With eager loading your prefix point holds. I have not measured the cache miss itself.
I'd keep the raw descriptions alongside the normalized hash. A hostname inside explanatory text can be incidental, but a hostname or path identifying the actual target can change what a tool does. A small paired test could change only the container ID in one case and the permitted target in another: normalization should suppress the first difference without hiding the second. Would a server-specific normalization rule, with a visible raw diff, fit your cards better than stripping all hostnames and paths?
Fair point. We do keep the raw hash: the normalized one only swaps 12-character hex strings (the container ID) and sits next to the raw one, it never replaces it. But stripping every hostname and path would hide a changed target, you're right, and I wrote that too loosely in the post. Your paired test is a nice idea, I haven't run it. Might try it with a dummy server.
@innokentyb ran the paired test. synthetic server with one tool, three variants: a base, one where only a container-style id changes in the description, and one where only the allowed target changes in the input schema. changing the id moved the description hash and the combined one. replacing 12-character hex strings made all three hashes identical again, and the schema hash never moved. changing the target left the descriptions alone and moved the schema hash, raw and normalized. so for this case the normalization removes the incidental difference without hiding the real one. it's a toy, one tool, nothing like a real server. the script and the raw comparisons will be in the public repo soon.
The pass criterion has a protocol revision built into it. Answering initialize with a valid protocol version is how every session opens under the revisions up to 2025-11-25. The current revision, 2026-07-28, removes the initialize and notifications/initialized handshake: every request carries its protocol version, and servers must implement server/discover, which returns the versions and capabilities a server supports and its identity. A server built only for the new revision rejects initialize with an error, so a probe that starts with it would record a failure that belongs to the probe rather than to the server.
Sending server/discover first and falling back to initialize when it errors or times out is the order the spec gives for stdio clients that speak both eras, and it keeps a pass meaning what the post says it means whichever revision the server speaks: installed, started, answered with a valid protocol version and listed its tools. Recording on each card which of the two the server answered, and the version it returned, would also show how many of the ten passes would read the same to a client that speaks only the new revision.
you're right, and i checked the spec before answering. our probe only sends initialize, so a server built only for 2026-07-28 would show up as a failure that belongs to the probe, not the server. everything we recorded answered initialize, so those speak the older era or both. i've changed the probe to send server/discover first and fall back to initialize like the stdio section says, and to record which one answered and the version. so far i've only tried it on a toy server, i'll rerun the real ones soon and put the era on each card.
@izgorodin ran it. 27 servers: 12 answer server/discover, 10 error and then work with initialize, 4 give no answer in 20 s and work with initialize, and one failed on an npm error. for 11 of the 12 modern ones the tool list is identical either way. server-everything isn't: 16 tools with initialize, 22 with discover, same version. one run, one machine, so i wouldn't generalize.
The server-everything split is the result worth putting on its card, because it makes the era part of what a hash means. Same version, 16 tools after initialize and 22 after server/discover: a stability hash over tools/list now has to say which handshake produced the list, or a client that moves to the new revision reads as a changed server while nothing in the package changed. Hashing per era and comparing like with like keeps the Desktop Commander lesson intact on the new axis.
The four that stayed silent for 20 seconds are the case the spec describes for older servers, which may answer server/discover with an error or not at all. Across a directory that is the slow path of every probe, and recording how long each server took to fall back would show whether a shorter wait would mislabel any of them.
agreed on the card. the 22 tool list sits next to the 16-tool one as its own record, with the era on each, so the hash says which handshake made it. it goes live with the next deploy. fair point on the 20 secns too: we only logged total time per run, not how long each server took to answer discover or fall back, so i can't tell yet if a shorter wait would mislabel any. i've added it to the probe.
same version, different description is the kind of drift that makes you distrust the whole run.
yeah, that was my first reaction too. the other 24 descriptions and all the schemas were identical though, and the only difference was a 12-character container id in one description, so the rest of the run held up. still a good reason to look at the raw text before trusting a hash.
that 12-character id detail is what saves the run, hash alone would have called the whole thing drift.