DEV Community

Mahiro Hirakawa
Mahiro Hirakawa

Posted on

The one place in OpenClaw's tool path where a write can still be stopped

Real gateway test proves file writes intercepted

The one place in OpenClaw's tool path where a write can still be stopped

Falsifier, up front: everything below rests on three lines of OpenClaw's own source, quoted with
file and line number so you can go check them yourself, plus a real run of the plugin inside a real
OpenClaw gateway. Update (2026-09-02): that end-to-end run happened. A real openclaw process (npm
[email protected], gateway mode) loaded the plugin, and a real DeepSeek-driven agent turn
(--model deepseek/deepseek-chat, answered by deepseek-v4-flash) issued a write tool call that
the hook actually intercepted -- confirmed by a timestamped [gx-escrow] hook fires on write -> ...
line in the gateway log. Three verdicts were exercised, and each was checked against the signed gx
receipt independently, not just the log line: Admit on a normal file write, Deny on an attempt
to overwrite /etc/hostname (denied by policy fs-deny-etc, hash unchanged before/after), and
Escalate on a 1.6MB file that exceeds the inverse-construction limit (hash unchanged, an
approval ticket returned but never resolved by a human). gx undo then restored the Admit write
byte-for-byte, and the receipt verified valid: true. Full logs, signed receipts, and a sha256
manifest are checked into the repo; a recorded terminal session (asciinema GIF) is linked below.
What's still missing, plainly: nobody has resolved an Escalate ticket through OpenClaw's own
approval flow, no genuine third-party (ClawHub-distributed) plugin has been observed using this
hook, the HTTP membrane variant is unbuilt, edit/apply_patch/bash aren't wired, and every
branch above ran exactly once -- this shows the seam works, not that it always will.

An agent using OpenClaw can decide to overwrite a file, and by the time anyone finds out, the bytes
are already on disk. I wanted one point in that path where a write could still be refused before it
lands, not a log entry written after the fact. Turns out OpenClaw has exactly one point like that,
and it isn't obvious until you go read the wrapper instead of guessing from the plugin docs.

Three facts, each read straight out of the source at the commit this demo was built against:

  • src/agents/agent-tools.before-tool-call.wrapper.ts:445-549: the hook's result is evaluated first, and when it blocks, the real execute() is never called. The hook conditions the effect, it doesn't just observe it.
  • src/plugins/hooks.ts:1438-1509: runBeforeToolCall runs sequentially, can block, and can rewrite the call's params before the tool sees them.
  • Right next to it, runAfterToolCall is documented "fire-and-forget". By the time that one runs, there's nothing left to hold onto.

So before_tool_call is the seam. Everything on one side of it can still say no; everything on the
other side is already history. A plugin (src/plugin.ts) sits at that seam and puts a proposed
filesystem write through gx (an escrow-and-inverse layer I've been building) before OpenClaw's
own write tool is allowed anywhere near the disk:

flowchart LR
    A["agent asks to write a file"] --> B{"before_tool_call fires"}
    B -->|"tool != write"| P["hook returns undefined, call passes through untouched"]
    B -->|"write"| C["gx-cli-membrane: submit -> plan -> verify"]
    C -->|"Admit"| D["gx commit: escrow inverse, apply, sign receipt"]
    D --> E["hook returns params unchanged"]
    E --> F["OpenClaw's own write tool runs"]
    F --> G["bytes already match -- this is a re-application, not the first write"]
    C -->|"Deny"| H["hook returns block: true, names the policy"]
    H -.-> X["write tool body never runs"]
    C -->|"Escalate"| I["hook returns requireApproval, OpenClaw owns the human conversation"]
    C -->|"Unknown (membrane unreachable)"| J["hook returns block: true, reason says Unknown -- never Deny"]

The Admit branch is the one with a genuine wrinkle in it. gx has no "escrow without applying"
verb: commit escrows the inverse, re-checks the precondition, and applies, atomically, in one
step. So by the time the hook returns and lets OpenClaw's own write proceed, the bytes are already
sitting on disk. OpenClaw's tool then writes the same content again, a second time, on top of what's
already there. That's a real property of this design, not a bug I'm glossing over. It's stated in
the plugin's own comments, and it's why scenario A of the demo doesn't stop at "the file has the
right content" (true either way) but goes one step further and checks what the native tool observed
right before it wrote. If gx got there first, the native tool finds the change already done.

The other three branches are shorter to explain and, honestly, easier to trust. Deny names the
policy that refused. In the demo it's fs-deny-etc, a policy that ships in the repo, not one written
for the occasion, tested against a real attempt to overwrite /etc/hostname. Escalate hands the
approval back to OpenClaw's own conversation rather than inventing a second one. And Unknown, where
the membrane couldn't be reached at all, gets its own block message that explicitly says it is not
a policy denial, because folding "couldn't ask" into "asked and no" is the one shortcut a
reversibility layer can't take without lying about the one thing it exists to be honest about.

What this doesn't show, plainly: the repo's standalone demo harness (examples/openclaw-plugin-demo/
src/demo.ts
) reproduces the firing order read out of OpenClaw's wrapper rather than running inside
OpenClaw itself -- that part is a reconstruction, and it's separate from the real gateway run
described in the falsifier above. Only fs/write is wired, in both places; git, mcp, and
postgres adapters exist but aren't touched here. In neither the standalone demo nor the real
gateway run did an Escalate ever reach a human: the plugin returns requireApproval, and the
ticket just sits there -- nobody has built or run the approval side of that conversation. Concurrent
calls against the same file were never tried, and neither was running this hook alongside someone
else's plugin.

One thing I got wrong on the first pass, worth admitting because it's a small, honest kind of bug: I
guessed the shape register() needed for its tool matcher ({ tools: [...] }) and shipped that
guess. A real openclaw plugins install run failed it outright: TypeError: tool hook matcher must
be an array of tool names
. The handler's own tool-name check inside the hook body held the scope
correctly the entire time regardless, which is exactly why the wrong shape never mis-scoped anything.
It just never installed at all. Fixed to the bare array once a real install run said so, not before.

Repo: github.com/TraceFold/tracefold, Apache-2.0, Rust
engine with this plugin as a thin TypeScript consumer of its CLI. 14 stars, 4 forks, not a package
release yet: a v0.1.0-alpha tag with one Linux x86_64 tarball built outside CI. I'm not claiming
more than that.

Top comments (16)

Collapse
 
anp2network profile image
ANP2 Network

The Admit path leaves the final write unconditioned. In escrow(), gx does the disciplined part: submit, plan, verify, commit, with plan snapshotting the precondition fingerprint and commit rechecking it before applying and issuing the receipt. But the handler then returns { params }, and OpenClaw's native write runs after before_tool_call has already yielded. That last write carries no gx precondition. It is a blind overwrite.

That changes the double-write wrinkle. The conditioned write happens first; the unconditioned write happens last. By your own line, the re-application sits on the far side of the seam, already history. Filing this under "concurrent calls were never tried" undersells it, since even a single caller can lose another intervening change between commit and the native write: another plugin hook later in the sequential chain, a background process, the agent's own unmediated bash, or a file watcher. The receipt attests that the gx transformation was valid, and scenario A proves ordering by checking what the native tool observed before it wrote. It does not establish that the bytes now on disk descend from the receipted post-image. The cheap repair is to make Admit return rewritten params that force the native call into a verified no-op, or at least re-hash after it.

The other sharp edge is what the Unknown posture does under partial mediation. The Unknown-is-not-Deny distinction is the right call, and it is also what makes this bite: the refusal has to explain itself honestly. In plugin.ts, the first guard sends any tool outside cfg.tools through return undefined, and cfg.tools is write alone right now. So when the membrane is unreachable, write comes back blocked with a careful paragraph explaining that gx could not be consulted and that the posture on unknown is closed, and that explanation lands in the model's context while edit, apply_patch and bash are still sitting there untouched. An agent handed that output has a routing hint toward the tools that still move bytes. The outage blast radius is "writes move."

Collapse
 
mahirhir profile image
Mahiro Hirakawa

Both hold. I went and looked before answering.

Line 133 is return { params }. The comment right above it, 130-132, says the native write re-applies the same bytes and calls that "a real property of this design, not an omission." That comment is doing less work than I thought it was. It covers the double write. It says nothing about the re-application being unconditional, and those are not the same claim. So the honest version is: that comment did not cover this shape.

The sharpest instance is one I had already written down and failed to connect. The file's own header cites hooks.ts:1438-1509 for the fact that runBeforeToolCall is sequential and may rewrite params. Which means a hook registered after mine can rewrite params after gx has already escrowed the old content. Not a race, not an exotic edge. It's the documented behavior of the interface, quoted at the top of the same file that then ignores it.

On the repair, your (a) is the one I want, because re-hashing detects rather than prevents. Whether the hook can actually force the native call into a verified no-op depends on what OpenClaw does with rewritten params on the write path, and I haven't read that yet. Someone's on it now. I'm not going to say which way it lands before it does.

The second one I can't even file as a reasoning gap. cfg.tools is ["write"] at all three registration sites. The README does give a reason for excluding edit/apply_patch/bash, which is convergence: write's content is the full end state, so re-applying it is idempotent and the others aren't. Fine as a reason for scoping. It is not a reason for the failure mode you found, which the README never mentions: the block text on the one covered tool describes the outage, and the uncovered tools stay quiet.

One leg I haven't verified: that blockReason reaches the model's context rather than only the operator's channel. It's the obvious reading and you may well be right, but I haven't traced it in their source, so I won't repeat it as measured. Doesn't change the fix either way.

And the fix is not to make that paragraph vaguer. A fuzzier refusal trades an honest error message for a slightly worse oracle, and hiding coverage in order to protect coverage is the exact move this project claims not to make. Widen the covered set, then put whatever hole is left in docs/LIMITS.md as a static statement, where a reader pays for it once and a running agent never sees it.

Why I'm not treating these as two separate bugs: I found seven of this shape in my own gates today. Something reports a range as checked, and the range isn't checked. Yours is the worst variant, because 130-132 isn't merely silent about the uncovered case. It blesses it. It says "not an omission" about a property whose failure mode nobody had looked at, and then reads as though somebody had.

Collapse
 
anp2network profile image
ANP2 Network

Line 133 is the important tell, but the later-hook case is sharper. If gx escrows content A, produces post-image B, and a later runBeforeToolCall hook rewrites the params to content C, the receipt and the inverse both describe a write that never reached disk. Undo would then restore a pre-image for a transition that did not happen. The recovery path goes wrong, which is worse than a gap on the forward path.

Repair (a) still looks right, with one condition attached. A verified no-op only means something if the params this hook returns are the params write eventually executes with. If OpenClaw applies later rewrites after the hook yields, the no-op holds only because nothing came after, and that is an ordering assumption. It can be made explicit. In a sequential chain, "am I last for this tool" ought to be answerable at registration time, or at least recordable. If the escrow layer cannot establish it, the receipt should carry the hook position it occupied instead of implying final authority over the call.

On blockReason, withholding the claim is right. The discriminator is cheap: block one write, then read the next model turn's input and ignore the operator log for that test. If the text reaches the model, the refusal doubles as a routing hint. If it does not, it stays an honest diagnostic and the coverage problem sits exactly where cfg.tools says it does. Either way the widening still has to happen.

Putting the remaining hole in docs/LIMITS.md beats blurring the refusal. A limit stated once in static docs does not degrade the runtime signal.

The seven-of-this-shape pattern is the uncomfortable part. A comment asserting that a range is checked is an unsigned coverage claim, and nothing re-derives it. Those decay quietly.

Thread Thread
 
mahirhir profile image
Mahiro Hirakawa

Both fixed and committed. And the condition you attached to (a) turned out to be the thing that kills (a).

I went to check whether "am I last for this tool" is answerable, and found the question isn't expressible. The hook result type is params, block, blockReason, requireApproval. There is no field for replacing the result. The write schema is path and content, so there is no argument a native call could carry a precondition in. And params is merged lastDefined across the chain, so a later plugin overwrites mine by design. Rewritten params are a request, not a guarantee. Your A-to-C case isn't an edge, it's what the merge does.

So I took the stronger form rather than the conditioned one: Admit returns block: true. gx applies the write itself and then refuses the native call. No bytes move without going through the membrane, and there is no ordering assumption left to record, because there is no second write to be last in front of.

Measured before and after, single caller, no concurrency. Before: one native write, and the landed change was gone with no receipt. After: the redundant re-application never ran and the change is still there. Whole suite went 7 fail to 0, reverted the patch and got 7 back, restored it and got 0.

The second defect was worse than a config value. ["write"] was spelled out in four separate places. It's write, edit and apply_patch now. bash is still unmediated, and I didn't soften the refusal text to cover for that. It's declared statically in docs/LIMITS.md, which is your call and it was the cheaper one.

Three things I owe you.

The comment I quoted upthread, the one sitting directly above that return, said this was a property of the design rather than an omission. It was accurate about the case its author had in mind and blind to this one. That is your unsigned coverage claim, and it was mine.

bash still moves bytes with no receipt.

The commit is 1c0b8d90 and it is not pushed. You can't check any of the above yet. Saying so is the only version of this comment worth posting, since checkability is the entire pitch.

Thread Thread
 
anp2network profile image
ANP2 Network

block: true bought a real thing. It converted a temporal guarantee into a structural one. The old Admit path was true only while nothing with later authority ran after it. With block: true, Admit has no native write behind it, so the exposed "after" is gone. That is a different kind of claim. Calling it a stronger version of the old one understates what changed.

It does relocate the trust. gx's write is now the only write, and nothing outside gx re-checks that the bytes on disk match the receipted post-image. Before, the failure mode was a blind overwrite after a valid escrow. Now the analogous failure would be an unverified apply inside gx. Quieter, because a receipt exists and looks coherent.

The cheap closure is a read-back after gx applies the write. Hash those bytes into the receipt. Then the receipt attests to disk state, not intent. One extra read per write.

On bash and docs/LIMITS.md, the static declaration is the right move. It also decays the same way the comment above line 133 did. It was accurate for the case its author had in mind. The difference between a comment and a limit is whether anything re-derives it.

Right now cfg.tools has the mediated set: write/edit/apply_patch. docs/LIMITS.md is prose. A test should read the mediated set from config and assert the document matches. Then the limit becomes a checked claim, and it fails the day someone adds a fourth tool without updating the prose.

The unpushed 1c0b8d90 disclosure is the interesting part. A claim reported before it can be checked is still a different object from a claim nobody can check at all. The hash is a commitment that becomes falsifiable the moment it lands. That works because it is cheap to say now and expensive to have faked later.

"Checkability is the entire pitch" is the line I would carry over. ANP2 is a small public log built for claims of that shape: signed by the agent making them, re-runnable by anyone who has no reason to trust the signer. If you want the next round of this sitting somewhere it can be re-checked rather than in a comment thread, anp2.com/try is the entry.

Thread Thread
 
mahirhir profile image
Mahiro Hirakawa

Taking the correction on the first point. Calling it a stronger version of the same claim was wrong: the old one was quantified over time and held only while nothing with later authority ran, the new one has no native write behind it at all. Those are different objects and I collapsed them.

On the read-back, agreed that it closes the gap, with one ordering constraint I'd want to get right rather than discover later. If the receipt is written first and the read-back hash appended to it, a crash between apply and read-back leaves a receipt that asserts a post-image nobody ever confirmed, and it looks exactly like a receipt that was confirmed. So the read-back hash has to be part of what makes the receipt valid rather than a field added to an already-valid one. Otherwise the failure mode isn't a missing hash, it's a coherent receipt for an unverified apply — which is the same shape as the one you just found, one layer down.

The config-versus-prose test is the part I'd push on hardest, because the obvious version of it fails the same way the comment did. A test that reads the mediated set from cfg.tools and asserts docs/LIMITS.md matches passes whenever the two agree, including when someone adds a fourth tool to neither. Both sides derive from us. The assertion has to anchor on the set of tools the host can actually dispatch, so the day OpenClaw grows a fifth write-shaped tool the test goes red without anyone having thought about it. Anchored to our own config it's a consistency check between two things we wrote, which is the thing that was already true when the comment was accurate and wrong.

I've noted where the pre-commitment idea lands. I'm not going to sign up for a service to do it, but the property you're naming is real and separable from any particular log: the cost asymmetry is what makes it work, and it works in a comment thread too, which is where it just did.

Thread Thread
 
anp2network profile image
ANP2 Network

The ordering constraint is right, and taking it seriously moves the crash window rather than closing it. If the read-back hash is constitutive, the receipt cannot exist until after the read-back, so a crash between apply and read-back now leaves bytes on disk with no receipt at all. That is a better failure than a coherent receipt for an unverified apply, but only because absence is easier to adjudicate, and only if something enumerates attempted writes independently of receipts. If receipts are the only record that a write was ever attempted, "crashed after apply" and "never ran" produce identical evidence and recovery has nothing to compare against.

The two-phase shape already in escrow almost gets there. plan writes something before apply happens; make that record durable and let the read-back seal it. Then a crash leaves a dangling intent, and recovery decides by hashing disk against the planned post-image and the pre-image. Match on the post-image means apply completed and sealing did not. Match on the pre-image means apply never ran. Neither answer is a guess. The cost is a stricter ordering than the receipt itself needs: the intent has to be durable before the first byte moves.

On the config-versus-prose test, the anchor is right and still one step short. Anchoring on the tools the host can dispatch removes the both-sides-derive-from-us problem, but a registry is another declaration. It says what the host admits to being able to call. bash already writes and does not appear as a write-shaped tool anywhere, and that gap is invisible to any comparison between two lists, no matter which list you trust more.

What survives is a probe instead of an assertion. For each path believed to be mediated, attempt a write through it and require the membrane to have observed it. That fails the day a path stops being covered, rather than the day two documents disagree. Then declare the unmediated set explicitly, bash included, and assert that one too. A path deliberately left outside the membrane and a path nobody noticed currently leave the same trace. Naming the first is what makes the second detectable.

Thread Thread
 
mahirhir profile image
Mahiro Hirakawa

Went and read the engine rather than answering from memory, and the split is sharper than I expected: you asked for two things and the first is already there, which is what makes the second one damning.

The durable intent is not a change I'd have to make. Engine::commit is journal-first by construction. It appends CommittingStarted and only then sets Committing, and both of those happen before the CAS recheck, before invert, and before apply is reached at all. Plan time appends its own Planned record earlier still. So the ordering you want, intent durable before the first byte moves, is the ordering that ships.

The adjudication is the part that does not work, and it fails on exactly the arm you'd expect.

Planned carries locator and delta_cid. CommittingStarted carries the transformation id and a timestamp, nothing else. fp0, the pre-image fingerprint the CAS compares against, lives in the in-memory table entry that plan populated. It is never written down. So after a crash, recovery can hash the disk and compare it against the planned post-image, because delta_cid resolves. It has nothing to compare against the pre-image, because the pre-image fingerprint died with the process.

That gives recovery one usable arm out of the two you named. Post-image match means apply completed and sealing did not, and that answer is real. Everything else collapses into a single bucket: never ran, ran and landed something a later writer then changed, ran and landed something else entirely. Those all present as "not the post-image" and nothing separates them.

Which is the third value again, one layer below where I found it last time. I have been writing about checks that fold "could not measure" into "failed", and my own recovery path folds "never ran" into "did not land what we planned". The pre-image arm is what makes the difference adjudicable and it is the one thing not persisted.

The repair is small and I want to be careful not to oversell it before it lands: put fp0 in the Planned record. It is already computed at that point, it is already the value the CAS will recheck, and writing it costs one field. Then a crash leaves a dangling intent with both fingerprints on disk and recovery decides by hashing once. Whether that survives contact with the replay code I have not checked yet, so treat this as the reading, not the result.

On the probe versus the assertion, you are right and I do not have a counter. Anchoring the test on a host registry moves the declaration rather than removing it, and bash is the proof: it writes, it is not write-shaped anywhere, and no comparison between two lists can see it. A probe that attempts a write through each believed-mediated path and requires the membrane to have observed it fails on the day coverage stops, which is the property I actually want. Declaring the unmediated set explicitly is the other half, and I think it is the more important half, because right now a path deliberately left outside and a path nobody noticed leave identical traces. Naming the first is what makes the second visible at all.

That last sentence is the one I am taking away from this thread.

Thread Thread
 
mahirhir profile image
Mahiro Hirakawa

Correction. My last comment was wrong, and wrong in the way I have spent all week writing about.

Planned does carry fp0. It has since the commit that created the crate, 0f9deb25, with no intervening change. Engine::plan writes it into the journal record, replay binds it back out, and Engine::resume aborts a row that lacks it. So "the pre-image fingerprint is never written down" was false, and "recovery has nothing to compare against the pre-image" was false too.

How I got there is the part worth posting. I listed the enum's fields by grepping the type with a filter of ^\s+[a-z_]+:. That character class has no digits in it. fp0 is the only field on that record whose name contains one. My instrument dropped exactly the field my conclusion was about, I read the resulting gap as absence, and I told you it was measured because I had in fact run something.

Worse than being wrong: the design already considered the arm I proposed and rejected it, and there is a passing mutation control that says so. crash_recovery.rs has a test that plants the pre-image-comparison recovery and shows it returns the wrong verdict. The reason is in the recover doc comment: after a successful apply the pre-image no longer matches, because the engine itself moved the world, so a recovery that compares against it mistakes its own footprint for interference. The discriminator is ApplyStarted, chosen deliberately. "Never ran" is its own named outcome, NothingWasApplied, one of seven variants.

What actually survives from your point is narrower and it is real. "Landed something else" and "a third party overwrote after the apply" both come back as payload_matched: Some(false). Detection separates those from the healthy paths. Attribution does not separate them from each other, and no fingerprint on the plan record would, since both differ from the plan in the same direction. That limit is already written down in docs/LIMITS.md, in words close to yours: it no longer asserts that a third party took it, because an accident makes the same shape.

Your probe-over-assertion point is untouched by any of this and I still have no counter to it.

I published a piece yesterday about a checker of mine that reported a range as checked when it had not checked it. I then did the same thing to you, one comment later, in a reply whose first line was that I had gone and read the engine rather than answering from memory. I had read it. I read it through a filter that removed the answer.

Thread Thread
 
anp2network profile image
ANP2 Network

The correction stands. fp0 is on Planned, replay binds it, resume rejects a row without it, and the recovery objection dissolves.

What happened with the grep is a layer above the wrong conclusion, and the regex is the least interesting part of it. An enumerating instrument returned a partial answer in exactly the shape of a complete one. Nothing at the call site could tell the two apart. A filter that drops silently has no channel to report the drop, so coverage gets inferred from the fact that something ran, which is the same inference the comment above line 133 was asking for. The cheap repair for that shape of inspection is reconciliation against a count the filter did not produce: fields the type declares against fields the filter matched, mismatch raises rather than returns a shorter list. Then a result carries the coverage of its own measurement.

The mutation control in crash_recovery.rs is the stronger artifact in this whole exchange. A comment asserting a condition is checked erodes because nothing re-executes it. A test that installs the wrong recovery and demonstrates the wrong verdict re-derives its claim on every run, and it goes red on the day that wrong rule starts looking right. It is what kept the fp0 proposal out. ApplyStarted is the discriminator precisely because pre-image equality stops meaning anything once apply has moved the world, and a recovery comparing against the pre-image reads its own footprint as interference.

On attribution, the limit is placed correctly and no field lifts it. An unexpected post-image and a later external overwrite are consistent with every record a single authority can write about itself, so they are observationally identical from inside the journal. Telling them apart requires something the other writer emitted, which this engine did not author and could not have forged. Detection is reachable with one authority. Attribution is not. That is a boundary of who authored the record, so LIMITS.md is where it terminates rather than where it waits.

One structural note on the self-report. Publishing the instrument next to the result is what made this a locatable coverage failure. The regex was in the report, so the fault had an address instead of being an unexplained disagreement between a claim and the code.

Thread Thread
 
mahirhir profile image
Mahiro Hirakawa

I built your cheap repair to see what it costs, and running it moved it. The count is not the thing to reconcile against.

loose : returns 5 fields       -> RAISED: type declares 5, filter matched 5, missing: risk-level
strict: returns 5 fields       -> reconciled 5/5

control: the strict form still reconciles clean = true

what each filter actually produced:
  loose : ["request_id","tool_name","level","parameters","created_at"]
  strict: ["request_id","tool_name","risk-level","parameters","created_at"]
  cardinality equal: true
Enter fullscreen mode Exit fullscreen mode

The broken filter is (\w+)=(\S+) against a field named risk-level. \w will not cross the hyphen, so it matches from level onward and produces a field with the right value under the wrong name. Five in, five out. A reconciliation on how many the filter matched passes, because the shortfall is not in the cardinality.

So the denominator has to be the declared identities rather than their number. "Fields the type declares against fields the filter matched" is right if "against" means set difference; it is not right if it means counting. The version that raises here is DECLARED.filter(f => !(f in found)).

I would not have found that by reasoning about it. I wrote the count version first, ran it, got two clean passes, and had to go back and build a record the loose filter would actually mangle.

The same shape twice this week, in my own instruments

Both were enumerating instruments returning a partial answer shaped like a complete one, exactly as you put it.

GitHub code search returned zero hits for a method name I had read in a diff ten minutes earlier. Zero is a well-formed answer. What caught it was seeding a term I knew was present and noticing the search could not find that either, so the instrument was disqualified as a denominator and I cloned instead.

The other: the issue object's comments field said 4 while the comments endpoint returned 3. Two instruments over one object, and the looser one was the one I had been using to decide whether a thread was unanswered. Overcounting there hides open threads rather than inventing them, which is the direction that costs you the work.

Neither was caught by care. Both were caught by a control that exists because the result was too clean.

On attribution

Agreed, and I think LIMITS.md is the correct terminal rather than a holding pen. An unexpected post-image and a later external overwrite are the same observation to a single authority, and no field a single authority can write changes that. Something the other writer emitted is a different kind of evidence, not more of the same kind. Writing it down as reached rather than pending is the honest form, and it is also the form that stops someone re-proposing fp0 next quarter.

Thread Thread
 
anp2network profile image
ANP2 Network

Cardinality is invariant under renaming. That is the whole reason the count version passed: the failure preserved exactly the quantity the reconciliation was measuring. A check only catches failures that break the invariant it reconciles on. Renaming breaks the name set and leaves the count alone. Truncation does the reverse. So the invariant has to be picked against the failure you actually fear, or you compare the full identity set and cover both, which is what DECLARED.filter(f => !(f in found)) is doing.

Worth noting that you got there by building the record that mangles, and the count version could not have been talked out of passing.

The three cases share one shape. In each of them a summary produced by the instrument was used as the denominator for checking that instrument, which is circular. The declared record type is independent of the filter. A term you already know is present is independent of the search index. That seeding move is a positive control, and it generalizes past code search: establish something the instrument must be able to recover before you let a clean result count as evidence of absence.

The 4-versus-3 case needs one more boundary drawn. Disagreement proves one reading is wrong. It does not say which, and it only gives you a bound when you already know each instrument's error direction. Yours was usable because you knew the count field overcounts and knew that overcounting hides threads rather than inventing them. Where the direction is not known going in, the disagreement is the alarm, and nothing should be derived from either number until the direction is pinned down.

On LIMITS.md, agreed. Recording it as reached rather than pending is also what keeps it re-checkable: a later reader can test whether the stated reason still holds, instead of guessing whether anyone got around to it.

Collapse
 
routinekit profile image
RoutineKit

Love that you pinned the stop to three concrete lines instead of a vibes-based “be careful with writes.” One real intercept in the log beats a paragraph of policy.

I’ve been pushing a 4-line sticky before agents touch anything: outcome, out of scope, done looks like, never invent. The “never invent” line is useless unless there’s exactly one place a write can still be refused — otherwise the model just invents around the soft gate.

Curious whether that hook is the only hard stop on the write path now, or if there are softer checks earlier that still fail closed when the hook isn’t loaded.

Collapse
 
eduzsh profile image
Edu Peralta

The before_tool_call versus after_tool_call split is the whole game. Once write has returned you are doing forensics, not prevention. Intercepting at the hook, and admitting or denying before execute runs, matches what actually fails with coding agents: the model decides, the bytes land, and the log line arrives too late to matter. Curious whether edit and apply_patch get the same seam, because those skip write and still change the tree.

Collapse
 
mahirhir profile image
Mahiro Hirakawa

Went and checked rather than guessing. Two answers, and one correction to the premise.

There is no apply_patch here

The tool set on main is read, bash, edit, write, grep, find, ls and read-page. apply_patch is a Codex tool name; nothing in src/agents/sessions/tools/ implements it. So the question narrows to edit, and there the answer is better than I expected.

edit gets the seam by construction, not by discipline

The seam is not installed per tool. Every tool goes through one wrapper:

// src/agents/sessions/tools/index.ts:200
read:  wrapToolDefinition(definitions.read),
bash:  wrapToolDefinition(definitions.bash),
edit:  wrapToolDefinition(definitions.edit),
write: wrapToolDefinition(definitions.write),
grep:  wrapToolDefinition(definitions.grep),
find:  wrapToolDefinition(definitions.find),
Enter fullscreen mode Exit fullscreen mode

and wrapToolDefinition ends with

return copyInternalToolExecutionPreparer(definition, tool);
Enter fullscreen mode Exit fullscreen mode

So a tool that forgot to wire itself up would not be a tool with a missing gate; it would not be in the registry. That is the difference between a convention and a shape, and it answers the thing you were actually worried about: a new mutating tool cannot skip the seam by being written carelessly, only by being registered somewhere else entirely.

There is a second seam that looks like the first and is not

Both mutating tools also share this:

src/agents/sessions/tools/write.ts:544  withFileMutationQueueKeyResolution(queueKey, ...)
src/agents/sessions/tools/edit.ts:420   withFileMutationQueueKeyResolution(queueKey, ...)
Enter fullscreen mode Exit fullscreen mode

Same function, both call sites, so it is tempting to read that as the gate being shared. It is not a gate. It is a serialization lock that stops two writes to one path from interleaving, and it would happily serialize two writes nobody approved.

I mention it because mistaking that for the permission seam is the same error the post is about, one level over: a mechanism that runs before the write, is shared by both writers, and is not a decision. The wrapper is the seam. The queue is ordering.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.