Idempotency keys for AI agents: why hashing the arguments doesn't work

The pending-row ground is covered upthread, so here is the case where the provider's key fights yours.

A posting API we run against accepts no caller-supplied idempotency key and dedupes on a hash of the body it receives. An identical re-POST comes back 200 with already_existed=true and a reference to the prior object, including when that prior object is in a permanently failed state. A legitimate retry gets swallowed and reported as success. The only way to obtain a fresh object is to change the generated text, which is precisely the field your table keeps out of identity, so the provider makes model-written content the retry discriminator whether you want it there or not.

The second half bears on your answer about waiting for a provider receipt. That same create call returns 201 with the object not yet live, and the response body carries a one-shot verification challenge that has to be answered by a second call inside five minutes. A later GET on the object returns 404, and no list endpoint exposes the token. Objects were lost exactly this way, by truncating the 201 body before writing it to the log: the tail went, the challenge went with it, and the object sat pending with no path forward. Recovery meant DELETE plus a re-POST with changed content, which walks straight back into the dedup behaviour above.

So there is a third state next to done and unknown: issued, present on the provider, and permanently uncompletable. The requirement that falls out of it is that the row holding the key must also persist the full raw response of the effect call, written before anything parses or trims it. Coordinates are not enough when the response is the only place a completion token ever appears.

Does your effects ledger keep the provider's response body, or the key plus an outcome?

Good post, and since you asked for the proposal to be torn apart: I think "hashing the arguments doesn't work" is two claims wearing one coat, and they need different instruments.

I measured the first one. Five representations of the same tool call — same recipient, same subject text, same amount, same note — hashed with sha256 over json.dumps:

• default dumps: 5 distinct hashes over 5 representations
• sort_keys=True: 4 over 5 (key order collapses, the rest survive)
• sort_keys + canonical numbers + NFC + rstrip: 1 over 5

The survivors are ordinary: 0.1+0.2 serialises as 0.30000000000000004 while 0.3 serialises as 0.3; a note stored as NFD and the same note stored as NFC are different byte strings; a trailing newline is a different subject. None of that is the LLM's fault and none of it is a property of hashing. It is a bug class with a known fix, and one test — N representations of one call must yield one key — catches it.

The second claim is the one that ends the naive design, and canonicalisation cannot touch it: four drafts of one intent gave four distinct hashes, which is correct behaviour, because they are four different strings. A hash of generated content is a hash of a different object every time the model runs, by definition.

That sharpens your rule rather than disagreeing with it. The key cannot come from generated content at all — not because hashes are brittle, but because there is nothing stable in there to hash. So the authored part is exactly what your TL;DR names: intent, declared by the caller.

What I would add is that the shape of the effect stays hashable even when the prose does not: tool name, declared target, amount, idempotency scope. Exclude the generated body from the digest by construction rather than by convention, and then the property becomes testable — N drafts of one intent must produce one key while all N bodies differ.

One trap worth writing into the design. Whatever canonical form you pick becomes part of your key derivation forever. If a dependency changes how it serialises floats or normalises text, every in-flight key changes silently and replays stop deduping, which fails in the direction that emails the customer twice. Pinning the canonicaliser behind a version prefix in the key itself makes such a change a new key space instead of a silent one.

"The key answers 'have I already done this thing, for this reason, to this target?'" — that's the cleanest definition I've seen, and it's exactly what I was trying to say in the previous thread.

For the "how long does the ledger remember" question: I'd argue the key should have a time bucket built in, not a TTL on the ledger itself. Instead of (event, tool, recipient), use (event, tool, recipient, day). That way "send Anna a follow-up next month" is a new key, not a resurrection of an old one. And the ledger can keep everything forever without the "resurrection" problem.

For the "different branch" question: I'd key on the effect (send email to Anna about event X), not on the decision (the agent decided to email Anna). If run one emailed Anna and run two skipped her, the effect for Anna was still "done", so she's still skipped. The branch changed, but the effect didn't.

Does that match what you're converging on, or am I overcomplicating it?

This is a good example of why building production-ready AI agents involves much more than getting the model to call the right tool. At IT Path Solutions, this is the kind of engineering challenge worth addressing early: what happens when a workflow partially succeeds, retries, or encounters an uncertain outcome? Designing those failure paths upfront can make agentic systems far more dependable in production.

Your first bullet, the GET that is not free to repeat, is the everyday case for a paid search tool: a replay bills again and hands back rows the client already holds, and a hash of the arguments catches neither, because the model rewrites the filter the way it rewrites Anna's subject line. What held for us is your rule applied to a read: the key is the record identity, never the query. The caller passes the ids of the result sets it already received, the tool skips every record present in them and bills only what is new, so a replay with a reworded title filter still delivers each person once. And the ledger lesson transfers whole: a record missing from the re-pull is unseen, not gone, so nothing downstream gets marked as having left.

Great piece! Your distinction between external identity and LLM-generated content for idempotency keys completely hits the nail on the head.

A few thoughts on your open questions:

Multiple emails per recipient/event: Add a logical action tag or step type to the key (e.g., (event_id, recipient, "acknowledgment") vs (event_id, recipient, "summary")) to separate distinct intents.

Branch divergence on replay: If the agent drops a recipient on the second run, they simply aren't emailed again, which is usually fine. For the ones targeted twice, deterministic keys successfully block duplicates. Post-execution reconciliation helps catch any missed critical actions.

Ledger retention: Instead of a blind time-based TTL, tie cleanup to the parent business object's lifecycle (e.g., active campaign or ticket status) combined with a rolling 30-day window.

The gap before ledger write: For APIs lacking native idempotency, an Outbox pattern (PENDING state before the call, confirmed after) paired with an async reconciliation loop is the safest bet.

Two things on the open questions.

On drift between runs: the ledger as described keys on (event, tool, target), which handles overlap but not the drop. I'd freeze the intended target set as a plan row before the first effect — a list of (target, key) written when the plan is approved — and let a replay reconcile against the union of plan and ledger. Then "emailed in run one, dropped in run two" becomes a visible diff (approved but never sent) instead of a silent skip, and the metric you want changes from "did the run finish" to "did every approved recipient get exactly one effect".

On retention: a time TTL feels wrong for this. A provider replay window like Stripe's 24h is a clock, but "the same email next month is legitimate" is a business rule. Since the key already scopes to the event, the event id is the natural expiry — a new event id never collides, so the row can live as long as you keep the business record of the event.

One question: for a tool whose provider has no idempotency support and whose effect is irreversible, do you treat the ledger write as the only barrier, or do you require a provider-side receipt you can look up later before the row counts as done?

You nailed the hard part: the key answers "have I done this, for this reason, to this target." We run Service Bus duplicate detection on a few flows, and the dedup key always comes from the business identifier, never the payload, for exactly this reason. On the ledger TTL question, scoping retention to the window where replays actually happen feels right. A duplicate from six months ago is usually a different request.

The rule that the key comes from stable intent and never from generated content is the part I would pin above every agent tool. The email campaign that dies at 40 of 100 shows why both obvious answers fail: refuse the replay and 60 people get nothing, force it and 40 get the email twice.

Max Kaya's idea of freezing the intended target set as plan rows before the first effect fits well with what you already have. Once a human approves the campaign, that approval is the stable intent: a fixed list of recipients, each keyed on the event id and the resolved record, with nothing the model wrote. A replay then only has to ask which rows are still pending.

We use the same idea on client builds outside agents too. Write down what done looks like for each target before anything irreversible runs, and recovery becomes a lookup instead of a judgment call. Will the revoked state be visible to the person approving a forced replay?

The pending row upthread covers the crash case. The same insert also covers one nobody has raised: parallelism inside a single turn. The Messages API can return several tool_use blocks in one assistant response, and the docs leave execution order to the caller, with a note that tools with side effects or shared state may be better run sequentially. Run them with Promise.all and two calls carrying the same (event, tool, recipient) key can both read an empty ledger before either writes. A select, then send, leaves that window open for the length of the request. The fix is the one your webhook section already uses: a unique index that rejects the duplicate insert, moved off (subscription_id, resource_id) and onto the effect key, with the pending row written before the call. The loser of the insert skips instead of asking. So: does that recipient loop live inside one tool call, or does the model get one tool_use block per recipient? Inside one call the loop is yours to serialise. One block per recipient hands the ordering to the runtime, and a batch call instead hides the race behind a key covering many targets, where a partial failure gets hard to attribute.