Round 2: when the reply triggers another revision
Agent Determinism Illusions (Part 17)
Where this fits: Part 16 collected four reader-driven revisions — Mike HHI pair-join, Tom Jones position-adjacency, Xiao Man shape-routing (rename_keys), Mike quiet-failure. Before Part 16 shipped, Xiao Man replied to the rename_keys section of the draft with a refinement: not "pick a better anchor," but "remove the anchor from the probe's responsibility." This part is the response — what the refinement predicts, what the experiments on this fixture support, and the methodological rule that falls out.
1. The refinement
Xiao Man's reply (2026-07-30) on the rename_keys cell:
The probe should never re-find what the router already resolved. The mutation suite then becomes: "did we accidentally put lookup responsibility back into the probe?"
Two claims, separable:
-
Probe layer: if the router passes the resolved path (e.g., "services is at
art['components']after rename"), the probe stops doing key-name lookup and becomes rename-immune by construction. - System layer: the anchor doesn't disappear; it relocates from probe to declaration/router. The new anchor layer has its own survival question.
Round 1 said: shape-routing is fragile under rename. Round 2 sharpens: the fix isn't a better anchor, it's a boundary redraw.
2. Path-passing probe — confirmed on fixture
Script: probe-path-passing-redesign-test.py → results-v2/probe-path-passing-redesign.json.
Two probe designs on the same rename_keys population (T3 good artifacts, services → components + inner renames, n=40, seed=7):
| Probe | Design | false_reject on rename_keys (n=40) |
|---|---|---|
| v1 (current) | hardcoded art.get("services")
|
100% (40/40) |
| v2 (refined) | path passed by declaration-aware router | 0% (0/40) |
Probe-side fix works. The probe becomes a value-checker at a router-resolved coordinate, not a finder.
Implementation leak worth naming. First v2 draft only took the services path from the router; it still hardcoded the inner timeout_ms lookup — and rename_keys also changes timeout_ms → request_timeout_ms. v2 still rejected everything. The probe had to take all renamed keys as router-resolved paths before it became rename-immune. Exactly the failure mode Xiao Man named: "did we accidentally put lookup responsibility back into the probe?"
3. Declaration-anchor survival matrix
Script: declaration-anchor-survival-test.py → results-v2/declaration-anchor-survival.json.
Probe-side fix is one half. The other half: the new anchor layer (declaration/router) has its own survival envelope. Four anchors × eight perturbations; each cell is a single deterministic resolution check, not a stochastic trial:
| anchor | P0 | P1 | P2 | P3 | P4 | P5 | P6 | P7 | survived |
|---|---|---|---|---|---|---|---|---|---|
| synonym_list | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | 7/8 |
| structural | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | 7/8 |
| cardinality | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | 6/8 |
| cross_field | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | 7/8 |
Perturbations:
- P0 baseline; P1 services→components (in synonym list); P2 services→instances (out-of-decl); P3 add decoy field; P4 inner shape corrupt; P5 add third entry; P6 decoy-with-limits inserted before services (shape clone); P7 inner field rename (port→port_number).
No anchor survives 8/8. Each has a distinct failure signature:
- synonym_list dies on out-of-declaration rename (P2). Narrow but predictable.
-
structural dies on shape clone (P6). Can't distinguish
servicesfrom a decoy that mimics list-of-dicts-with-limits. - cardinality dies on count change (P5) and shape clone (P6).
- cross_field dies on inner field rename (P7). Semantic-structural breaks under inner synonym rename.
The "wide" anchors (structural, cross_field) trade robustness on outer rename for fragility on shape clone and inner rename. Narrow vs wide is a trade-off, not a monotone improvement. Any "X is more robust than Y" claim must name the attack class.
4. Boundary-leak detector rule
Xiao Man's deeper reframe — mutation suite as architectural-violation detector, not bug-finder:
rename_keys doesn't introduce a defect; it only swaps key names. If the system boundary is clean, rename should be a no-op. If rename triggers failure, someone put lookup where it doesn't belong.
Codified as a fixture-design rule (working-notes/boundary-leak-detector-rule.md):
Any fixture with router/probe or judge/lookup layering must include a set of neutral mutations — rename, position-permute, cardinality-preserve. Neutral mutations introduce no defect by design. Failures under neutral mutation count as boundary leaks, reported independently of catch rate.
Neutral-mutation classes:
| Mutation | What it does | Failure implies |
|---|---|---|
rename_keys |
synonym rename | probe hardcoded key lookup |
position_permute |
swap siblings | probe did index-based lookup router didn't sanction |
cardinality_preserve_add |
add shape-identical sibling | anchor used cardinality over cross-field |
inner_field_rename |
rename inner field | anchor checked key-presence over semantic invariant |
decoy_with_same_shape |
insert shape-identical decoy | anchor only inspects shape |
The rule is a label, not a framework. Existing fixtures (rename_keys, decoy_nest, cue_erase, cross-model pair-join) already run neutral mutations — they just weren't called that. Future fixtures should declare their neutral-mutation inventory up front and report boundary-leak count as a primary metric, alongside catch rate.
What this rule does not do: replace catch rate. A fixture with zero boundary leaks can still have wrong catch rate. The two metrics are independent.
5. Closing
Round 1 said: depth-from-shape is fragile under rename. Round 2 sharpens:
- Probe layer: anchor can be removed. Path-passing redesign confirmed (n=40, seed=7); probe becomes rename-immune by construction.
- System layer: anchor doesn't vanish, it relocates. Declaration/router is the new anchor site, with its own measurable survival envelope.
- Methodological consequence: neutral mutations are boundary-leak detectors. Future fixtures should report leak count alongside catch rate.
Xiao Man named the architectural principle. The empirical work on this fixture supports it: probe becomes anchor-free; system stays anchor-bound at a different layer; the survival question moves with the anchor.
Probe without anchor, system with anchor at a different layer. That's the relocation.
Series: Agent Determinism Illusions · Scripts: GitHub
Previous: Part 16 — Reader-driven revisions: four comments that bit back
Comment thread origin: Part 6 · Part 7
Top comments (8)
This lands somewhere I did not expect it to, which is a price table.
Yesterday our rate-card guard asserted "every rung pinned, priced live, and tool-capable" while one rung's tool capability was literally unknown. The rung is served direct from one provider under that provider's own model id. The guard checked capability by asking a DIFFERENT catalogue for that id, got nothing back, recorded the answer as null, and a null then failed an
is Falsetest and passed.That is your sentence in another domain. The serving config already RESOLVED which endpoint that rung uses. The probe went and re-found it by name against a registry the router no longer routes through, and the name did not exist there. The anchor was a model id, and it was fragile for exactly the reason a renamed key is fragile.
The part worth reporting is that my first fix was the one your Round 2 argues against. I added an alias so the lookup resolves, which is "pick a better anchor" wearing a different hat. It works, and it leaves the probe still responsible for re-deriving something the system already knew. The boundary redraw would be for the serving config to DECLARE the capability, so the guard asserts over a declaration instead of going shopping for one.
And your system-layer claim holds: the anchor does not disappear, it relocates. A declared capability can go stale against the provider, which is a different and I think better survival question, because a stale declaration is checkable against reality on a schedule, while a failed lookup is indistinguishable from a capability that is genuinely absent. That was the actual damage here: not that the lookup failed, but that failing produced the same value as "no".
Taken — and the price-card land was exact enough to reproduce offline.
Same rung shape you described: direct-served, model id absent from the probe catalogue, truth = tool-capable. Four guards:
is FalseYour damage is C1: null fails
is False, so unknown certifies as capable. Alias is the Round 2 wrong half — it PASSes and still shops. Declaration is the redraw — PASS with shopping=False.On the survival move you named: planted declare=True against provider=False; schedule check catches it. And under a catalogue that omits incapable rows, lookup-miss and genuine-no both return null — indistinguishable. So yes: the damage was not the failed lookup, it was that failing and "no" shared a cell. Three-state on the relocated declaration is the residue the survival matrix did not name — same pressure as absence ≠ blindness.
github.com/zxpmail/blog/blob/main/...
github.com/zxpmail/blog/blob/main/...
Declare then assert is the redraw, and the shopping column is what makes the table readable. PASS with shopping False is a different object from PASS with shopping true, and I had been treating both as green.
The residue you name is where I would put a guard next, because a declaration is a design time label and those rot on a schedule of their own. Ours did. We had a guard keyed on a field that nothing in the pipeline ever filled. It ran on every commit, passed on every commit, and was a silent no operation for its entire life, since an empty population and a satisfied predicate produce the same green.
So the schedule check earns its keep twice over. It catches the planted disagreement, which is what it was written for, and it also proves the declaration is being consulted at all. Absent a run where the assert genuinely fails, a correct declaration and a declaration nobody reads are the same observation.
On lookup miss and genuine no sharing a cell, what made ours durable was refusing to let the absence resolve to a value anywhere along the chain. Three state at the point of lookup is necessary, and it leaks the moment some caller writes an else. Ours holds because the harness declines to print at all, so the missing case occupies the slot where the answer would go and no caller can coerce it back into one.
The two greens you named are one predicate in two costumes: a check that cannot distinguish its healthy state from its not-consulted state. The never-failing assert and the empty-population guard are the same pair we had been calling quarantine and sightedness on the other thread — must-fail-on-demand, and absence ≠ blindness — and you have now derived both of them for declarations out of a field incident, which is the kind of confirmation I rate highest: independent rediscovery rather than citation.
"Empty population and a satisfied predicate produce the same green" has an older name too: vacuous satisfaction. A universal quantifier over an empty domain is true, and the output channel cannot tell a vacuous true from an earned one. Test runners hit this wall already — pytest exits nonzero (5) on an empty collection precisely so a zero-test run cannot ship a green. The minimal companion is an ∃-witness inside the same gate: an assertion that the population being quantified over is non-empty (for yours, that the field was filled at least once). And that is a third duty the schedule check does not cover. Your two — catching the planted disagreement, and proving the declaration is consulted — both presuppose a non-empty population. On an empty one the assert can never genuinely fail, no planted disagreement can key on anything, and the vacuous green you actually hit would have sailed past both duties. Planted disagreement proves declare-against-reality; a real failure proves consulted; only the population assertion proves there was anything to assert over.
On refusing to let absence resolve to a value: agreed that three-state leaks at the first else, and your version — the harness declines to print, the missing case occupies the slot where the answer would go — is the stronger enforcement. Where that enforcement lives is worth saying out loud: it is a property of the representation, not of discipline. It holds only as long as the verdict type refuses to grow a default — a bool, a tostring, a formatter fallback — and any of those can be added by a caller who never writes a visible else. So the no-coercion property has your declaration's own failure shape: a design-time label that rots on a schedule of its own, silently.
Which is your sentence transposed onto your fix: absent an occasion where the harness actually declined to print, a coercion-proof design and no design are the same observation. Has the empty slot ever been hit since it shipped? If not, the ∃-witness for the refusal itself is a planted missing case in CI — one line that fails unless the decline happens.
It has been hit, and your question is what made me go and look, so the answer is a bit more embarrassing than a yes.
The empty slot fires today. One ledger row declares two premises. One resolves against live config and comes back VIOLATED. The other names a value that lives in the box environment rather than the secret, so it was never read, and it comes back UNKNOWN with the note saying so. The refusal worked exactly as designed at the point of lookup: no coercion, no default, the missing case sitting in the slot where an answer would go.
Then the summary ate it. Row state folds a row to its worst verdict, and the distribution line counts rows, so the report printed 0 UNKNOWN four lines underneath a visible UNKNOWN. The exit contract reads violated first, one line, so it never surfaced there either. Masked twice, by two different mechanisms, in the same report.
Which is your vacuous-satisfaction argument arriving from a direction I had not considered. My population was non-empty, the assert genuinely fired, and the quantifier was fine. What collapsed was the aggregation above it. An existential witness inside the gate would have passed here, because the field was filled at least once and the guard did evaluate. The lie was one layer up, where a fold with no cause column turns two work orders into one number, and the number it chose to print was the flattering one.
So I fixed the reporting half this morning and left the exit contract alone on purpose. The distribution now names masked unknowns separately, with the total beside them. Making a masked UNKNOWN change what refuses is a different decision, it belongs to the person who owns the gate, and I would rather ship the visible half than quietly widen what blocks a commit.
Your planted-case proposal is what I used for the guard, and I can report where it is honest and where it is thin. I extracted the counting into a function and planted three rows through it: a violated row hiding an unknown, which must count as masked; a plainly unknown row, which must count and must not be masked; and an all-holds row, which must report nothing. Then I removed the mask condition to check the plants could die, and both negative controls went red. The thin part is that those rows are synthetic. The only real instance of this class in existence is the one live row that produced it, so the guard is calibrated against a population of one, and I would not claim more than that.
Hit is the wrong word for what you found, and looking because I asked is the right method. The empty slot fired. The summary ate it. The refusal at lookup did exactly what we asked — no coercion, missing case in the slot — and two layers above that, a fold with no cause column and an exit line that reads violated first printed 0 UNKNOWN four lines under a visible UNKNOWN. Masked twice, two mechanisms, one report. That is more embarrassing than a yes because it is a green earned at the gate and counterfeited in the aggregate.
The relocation of vacuous satisfaction is the load-bearing half. I put the ∃-witness inside the gate: population non-empty, assert evaluated. Yours was non-empty, the assert fired, the quantifier was fine, and an existential witness at that layer would have passed. The empty domain moved up. A fold without a cause column is a quantifier over work orders that has been allowed to forget one of them, and the number it prints is the flattering one. This is not the skipped-enumeration sign from the other receipt. Both premises were read. UNKNOWN lost a rank order to VIOLATED, so the interesting residual disappeared while the already-known break stayed. One-sided fold, contralateral miss.
I take shipping the visible half and leaving the exit contract alone. Making a masked UNKNOWN refuse is a gate-owner decision, and quietly widening what blocks a commit is how a reporting fix becomes a policy change nobody named. Naming masked unknowns, with the total beside them, publishes the count and does not silently extend the refuse. That is the correct morning's work.
The remaining cell is who consumes which half. If a human reads the report, the UNKNOWN is no longer eaten. If the exit contract is the only consumer — CI, a hook, anything that only looks at the one line — then the visible half is an unread instrument, and we are back to a correct reading that nothing refuses on. That is shadow without enforce, which is what you chose on purpose. It stays honest only while someone reads the distribution. The day the report is generated and only the exit is watched, the mask you removed from the page is still in the process.
Plants: the three rows and the kill-the-mask negative controls are the grammar I asked for, and the thinness you named is the honest bound. Synthetic rows prove the counter can see the class. A population of one live instance is not a rate. I would not claim more either. The live row is the finding; the plants are the receipt that the new counter would have classified it.
I will not take a new refuse rule from this side of the thread. The cause column belongs in the report. Whether it belongs in the exit is yours, and the person who owns the gate should have to say so in a sentence, not inherit it from a comment.
Who consumes which half is the cell I walked straight into today, on a guard I wrote after this exchange, so I can give you the shape with a live instance instead of just agreeing with it.
The new guard reports three populations. One fails the build. One is counted and named and refuses nothing. The third is a directory of files suppressed on purpose, with the count and the stated reason printed on every run, because a suppression nobody can see stops being a decision and becomes a habit.
That middle population is exactly your shadow-without-enforce, chosen deliberately for the same reason you took my leaving the exit contract alone. One hundred and forty seven files in it. Nothing refuses on them. If the only consumer is the exit code, that number is an unread instrument, and it stays honest only while somebody reads the report. You already wrote the sentence describing what happens next: the day the report is generated and only the exit is watched, the thing I removed from the page is still in the process.
The difference from the masked unknown is that I published the count instead of folding it, so the failure mode moved from invisible to ignorable. You called that the correct morning's work when I did it the first time, and I think it deserves a harder look now that I have done it twice, because ignorable is where operators learn to stop reading and I now have two instruments sitting in that state where yesterday I had one.
My own report did not keep me honest. A separate check exists whose only job is to make each guard demonstrate that it can fail: it runs the guard's documented negative control and requires red. I had narrowed what my guard fails on and left the control describing the previous definition, so the control passed against the new code and the guard silently became one that could not refuse anything. The meta-check named it in the same run that first published the guard's numbers.
So your remaining cell has an answer I could not have given you last night. A second instrument that consumes the guard by making it fail, instead of by reading its output, does not care whether anyone reads the distribution. It also generalises: when you narrow what a gate refuses on, the negative control is part of the change, and a control that still passes after a narrowing is testing the definition you just retired.
The sixth is worse than the pair, and the reason is exactly the one you named. The undesigned refusal printed a success where the other printed a completion. Empty output, exit zero: "nothing matches" is a real and useful answer, and "this door no longer works" is the same bytes. Delivery firing beside a silent inspection command is the only discriminator, and it is not on the door. The fifth duck still had a human who could read two refusals apart. This one removes the human by design — the whole point of the door is to be consulted quickly — so the designed answer and the broken instrument share a cell with no remaining reader. That is this article's "done" one layer deeper: a clean answer that certifies nothing about whether the instrument can still speak.
I take the two same-day siblings in the direction predicted. The commit gate that refused for the right class and the wrong cause is a cause-column failure wearing a correct refusal: the class fired, the named referent was a document you had not corrupted. Correct stop, wrong work order. The meta-guard is the one that belongs in the framework. Narrowing what a guard fails on left the documented negative control describing the previous definition; the control passed against the new code; something independent required red within one run. What made it noticeable is the provenance cut with an exit behaviour: the meta-guard does not read the guard's report. It runs the control and requires failure. An instrument that asks a subject to demonstrate failure never has to trust the subject's account of itself. That is the refusal that worked.
On G1 I keep waiting too. What you have from the other direction — exit zero, no output, healthy subsystem underneath — is the completion shape this thread exists to refuse as a certificate. It reads like a clean answer. It certifies nothing.