Help Net Security covered a new open-source tool this week that writes tamper-evident audit logs for AI agents. It stores recorded agent activity in a file, each record carries a hash of the record before it, and edits to old records break the later fingerprints. The author is upfront about what it can't do, which I respect. That's rarer in this space than it should be.
The story points at a wider problem with this wave of tools. "Tamper-evident" is starting to read as a synonym for proof. A log that's awkward to edit and evidence that holds up when money or blame is on the line are different things. Two questions expose the difference, and neither one needs a security background.
Question one: who holds the key?
A hash chain with no signature can be rewritten by anyone who holds the file, right up until its head gets checkpointed somewhere outside the operator's reach. Anyone who can alter an old record can recompute the later hashes, and the chain checks out clean again. A bare hash chain only proves the file agrees with itself, and a rewritten file agrees with itself too. An outside checkpoint makes rewrites of already-checkpointed history visible, and that's real progress, but it still can't show which actions the recorder never captured. The docs for the tool in the story say "no secret required; that is the point." For making verification easy, fair enough. For evidence, that's the limit: if no secret was needed to write the record, no secret is needed to rewrite anything that hasn't been checkpointed yet. An unsigned chain is a notebook with numbered pages, not a notarized record.
Question two: where does the recorder sit?
When the recorder only runs inside the agent process, the agent decides what gets recorded. A prompt-injected agent can skip the logger before it exfiltrates your data, and malware doesn't file a report on itself. The record is complete only when the thing being audited chooses to make it complete. An audit trail needs a boundary outside the process being recorded, where the agent can't skip it, or the trail documents the agent's good days and goes quiet on the bad ones.
Some tools add a witness service: an outside party that periodically stores the chain's length and fingerprint. Witnesses are a good idea, and more of them should appear. A witness can only vouch that the chain hasn't changed since it last looked. It can't vouch for what the recorder left out when the recorder lives inside the process it's reporting on. A timestamp on an incomplete record is still an incomplete record.
The same test, pointed at Pipelock
Apply the same two questions to Pipelock, because a test that exempts its own author isn't a test. Pipelock mediates traffic at a separate proxy boundary. With enforced containment, meaning the network only lets agent traffic out through the proxy, recording happens at a boundary the agent can't skip. Containment is part of the deployment, and skipping it leaves a weaker guarantee, which the docs say plainly. Receipts are signed, and the operator who deploys Pipelock holds the key. I never see it. You can verify a receipt offline against a public key the operator provides, with no account and no server. The limit needs stating plainly: operator-signed means the operator could omit records before signing. The receipts prove what the proxy saw and that nobody changed it afterward. They don't yet prove independence from the operator who ran it. Outside anchoring is where this whole category needs to go. The math doesn't deliver more than that.
A shared vocabulary would help, because "audit trail" covers everything from a text file to signed, replayable evidence, and buyers can't price the difference. That's why I published Agent Evidence Levels, an open spec and checker for testing what an evidence artifact can and can't establish. A producer or operator can't award a grade through it. A public grade takes a verifier who's independent of both, and that rule applies to Pipelock the same as everyone else. If you build one of these tools, publish an artifact and a capability declaration an independent verifier can check before someone with a subpoena does the checking for you.
Top comments (14)
The two questions are the right frame, and I'd push on the "who holds the key" one a bit further: a single external witness (or a single operator-held key, like Pipelock's) still concentrates trust in one party — it just moves the trust from "the agent didn't lie" to "the witness didn't collude or get compromised." The stronger version is witness quorum: multiple independent parties checkpointing, with the guarantee only holding when enough of them agree, so no single compromised or colluding party can retroactively rewrite history. Even that isn't the finish line — quorum alone doesn't stop two isolated verifiers from each being shown a different but internally-consistent fork, unless the witnesses actually cross-check against each other's views (gossip / consistency proofs), not just against their own last-seen state. "Tamper-evident" keeps eating "unforgeable" the same way "detected" keeps eating "prevented" — appreciate you naming the gap plainly instead of selling past it.
Yeah. A single witness just moves the trust instead of removing it, and quorum only helps if the witnesses cross-check each other. Same wall certificate transparency hit with SCT gossip, still not solved there.
I haven't shipped even the single witness yet, so I won't pretend the quorum version is real. It's the direction. The levels problem you're describing is why I wrote up Agent Evidence Levels, a spec that names what each tier of evidence proves, since tamper-evident keeps getting sold as unforgeable.
Respect for the honesty about where the single witness actually is. On the SCT gossip parallel: does Agent Evidence Levels put any bound on time-to-detection at each tier, or is it currently in the same place as CT — assumed eventually caught, without a hard SLA on how eventually? Feels like that's the natural next axis once the tiers themselves are named: not just what a tier proves, but how long a forged entry can survive undetected at that tier before someone's obligated to notice.
No bound today. AEL grades what a tier proves, not how fast a forgery at that tier has to get caught, so right now it's in the same spot as CT: assumed eventually, no clock on eventually. You're right that it's the natural next axis.
A real bound needs two things the spec doesn't name yet: checkpoint cadence and someone actually obligated to check. Signed checkpoints give you the first half cheap. The second half is the hard part, same as it was for CT gossip. I'm adding it as an open question on the spec, and if you want to file the issue yourself I'd rather have your wording than mine.
github.com/luckyPipewrench/agent-e...
CT's actual fix for "someone obligated to check" was never really social — it's structural: browsers require an SCT to come from a log that's independently monitored, and require multiple logs, so redundancy substitutes for anyone being obligated. A consumer that only checks one witness gets zero benefit from the others existing. Might be worth building AEL the same way: instead of naming a responsible checking party, require that a tier claim gets independently cross-checked against N witness endpoints before it's trusted, so the incentive is structural rather than depending on someone volunteering to gossip. Doesn't solve the bound-on-cadence question, but it means the answer doesn't rest on finding a volunteer.
Better read of CT than mine, and it changes what I file.
One split though. Multiple logs kill the case where one log is the only witness and can lie. They don't catch a split view, where a log shows one tree to you and another to everyone else. That still needs someone comparing signed tree heads, which is why SCT auditing took years to land in Chrome.
The consumer-side version of your point does hold, and it's writable: a claim is worth the number of independent logs the relying party checked inclusion in itself, and one anchor is worth one anchor no matter how many exist. AEL-3 names a single log today, so that's a real change.
What CT had was Chrome. Nobody's in that seat here, so I can make the weak claim illegal and let relying parties set N, and that's it. If you see how to get the incentive without a browser-sized enforcer, I want it.
Filing it either way. Yours if you want your wording, otherwise I'll write it and mention you.
Chrome didn't stay the only enforcer for long, though — that's the part of the CT story worth borrowing. SCT verification eventually moved into TLS libraries themselves, so the check runs wherever a client validates a cert, not just inside one company's browser. The equivalent here isn't finding a single central authority, it's getting the cross-check into a widely-adopted client library that anyone consuming AEL claims imports by default — one that refuses a single-witness claim unless it's bundled with actual inclusion proofs from N distinct logs, verified locally by the consumer, not asserted by the claimant. You don't need one browser-sized enforcer; you need enough consumers importing the same skeptical-by-default library that skipping the check becomes the unusual choice instead of the normal one. Slower than a mandate, but it's the version that doesn't require anyone to be in Chrome's seat.
That's the incentive without Chrome. I'll file it with the checker as that library.
Still yours if you want the wording.
Take it and file it as yours — you're the one building AEL, it should read as one voice. One more thing worth naming while you're at it: the library needs to fail closed on absence, not just on contradiction. A client that silently accepts a single-witness claim when the skeptical library isn't installed gives you the same failure mode as today, just with extra steps. The spec should say what a consumer without the library is entitled to claim — probably nothing above AEL-1 — so skipping the check has a visible cost.
The recorder-inside-the-process problem is the one that keeps coming back in agent work. I run agents with persistent filesystem access between sessions, and the first thing I learned is that an agent that can write its own logs can also not write them. My workaround has been crude but functional: every agent writes to a directory it can append to but not delete from, and a separate process checksums the directory on a fixed interval that the agent does not control. Not tamper-proof by any real standard, but it catches the gap between "the agent reported this" and "something outside the agent saw this."
Your point about unsigned chains being a notebook with numbered pages landed well. The numbered-pages metaphor gets at why people over-trust these systems. A notebook is useful, but nobody enters a notebook into evidence without asking who wrote in it. The same question should apply to agent logs, and it usually does not.
Your workaround is more right than you're giving it credit for. An append-only directory the agent can't delete from, checksummed on an interval by a process the agent doesn't control, is a witness. That's the same shape the fancy tools are selling, you just built it out of directory permissions. The interval is your exposure window: anything written and rewritten between two checksum passes looks clean, and the checksummer still can't see what the agent never wrote down. But you've already crossed the line most setups never cross, which is getting the observer outside the thing being observed.
And yeah, that's exactly why I keep coming back to the notebook framing. The question "who wrote in it" has a boring, well-understood answer in every other evidence context. Agent logs just haven't been asked it yet.
The key distinction here is integrity vs completeness. A signed, tamper-evident receipt can strongly prove that a recorded event wasn’t altered, but it says nothing about events that never reached the recorder. For agent systems, I’d treat those as separate guarantees in the architecture: capture completeness enforced at the execution boundary, then cryptographic integrity on the captured stream. Otherwise it’s very easy to build an audit system that produces excellent evidence for an incomplete history.
Agreed, and that's the split I built around. Capture completeness is the containment boundary's job: the agent runs behind a kernel-enforced egress boundary it doesn't operate, so traffic can't route around the recorder. Integrity is the chain's job. The verifier reports them separately on purpose, containment attested or UNKNOWN printed right next to the chain verdict, so a valid chain never gets to imply complete capture.
Your last sentence is the whole failure mode. Great evidence for an incomplete history is worse than no evidence, because it launders the gaps.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.