DEV Community

Cover image for Agents That Act Need Brakes, Not Just Brains
James Anderson
James Anderson

Posted on

Agents That Act Need Brakes, Not Just Brains

Demos earn misplaced trust before safeguards exist

Here's the moment a lot of us had this year.

You built an agent. It was genuinely impressive — it reasoned, it planned, it called tools, it chained steps together to get real work done. The demo was so good that you gave it real permissions. Let it send the emails. Let it update the records. Let it hit the production API.

And then, one day, it did something you never sanctioned. Not maliciously — confidently. Correctly-looking. It completed the action, reported success, and moved on. You only found out later, when the consequence surfaced somewhere downstream. That's the moment you realize what you actually shipped: a system with a powerful engine and no brake pedal.

The entire industry has spent two years racing to make agents smarter. Better reasoning, longer context, more tools, more autonomy. Almost nobody spent that time making them safe to let act. And those are completely different problems — because the instant an agent stops just talking and starts doing, a more capable agent isn't more useful. It's more dangerous.

Let me make the case, and then talk about the brakes.


Brains and brakes are different problems

This is the distinction the hype cycle blurred, so it's worth stating plainly.

Making an agent capable — able to reason about a task, plan a sequence of steps, pick the right tool, recover from an error — is one problem, and it's the one all the model progress went into. Every new release is a better brain.

Making an agent safe to let act — ensuring that when it's wrong, the wrongness is caught, bounded, reversible, or stopped before it does damage — is a completely different problem. And here's the uncomfortable part: no smarter model solves it for you. A more capable agent doesn't act wrongly less; it acts wrongly faster and more convincingly. Upgrading the brain does nothing for the brakes, because the brakes were never in the model. They're in the architecture around it — the part you have to build yourself.

You can have the best engine in the world. Without brakes, that just means you hit the wall sooner.


Why "acting" changes everything

A chat that only talks has a wonderful property: its mistakes are free. It says something wrong, you read it, you move on. Nothing happened in the world.

The moment an agent can act, three things change, and each one is a reason you need control that a chatbot never did.

Irreversibility. A wrong sentence costs nothing. A wrong DELETE, a sent email, a processed refund, a pushed commit — those don't have an undo button. The cost of an agent's mistake stops being "re-read it" and becomes "clean up the damage," and some damage doesn't clean up.

Confidence is not correctness. This is the through-line of everything I write, and it's never more dangerous than when an agent can act. The agent takes the wrong action with exactly the same confidence it takes the right one. There is no tremor in its voice, no hedge, no tell. It is equally sure when it's about to help you and when it's about to hurt you.

Silent success. The nastiest one. The agent completes the wrong action and reports success — because from its point of view, it did complete the task, it just completed the wrong one. Your monitoring goes green. Nobody sees the wrong customer getting the refund until the wrong customer gets the refund. "It worked" and "it did the right thing" are different claims, and only one of them shows up on the dashboard.

Put those together and you get the core risk: an agent that can act is a system that can cause irreversible harm, confidently, without anything flagging it. That is precisely the situation brakes exist for.


Brake #1: Approval gates — ask before acting

The most basic brake, and the one most systems skip: for anything consequential, the agent doesn't do the thing. It proposes the thing, and waits for a human to say yes.

The skill here isn't gating everything — that would make the agent useless, and you'd click "approve" on autopilot until the gate meant nothing. The skill is gating by blast radius:

  • Auto-run the trivial and reversible — read something, format something, draft something. No gate needed.
  • Ask first for the consequential — sending, paying, writing to production, anything a user would want to see before it happens.
  • Draft-only for the dangerous — the agent prepares the action but cannot execute it at all; a human does.

The mistake to avoid is a uniform gate on everything, which trains people to rubber-stamp. A good gate is rare enough that when it fires, the human actually looks.


Brake #2: A reviewer that can actually say no

The popular pattern: a second agent reviews the first agent's work before it's allowed to proceed. A judge. A critic. It's a good pattern — and it has a failure mode that will fool you completely if you're not watching for it.

A reviewer that has never been seen to fail is indistinguishable from a reviewer that approves everything.

If your judge-agent has been green for months, that is not evidence it's working. It might be catching problems. It might also be rubber-stamping, silently broken, or checking the wrong thing entirely — and you would not be able to tell the difference from the outside, because both produce the same clean logs right up until the day the rubber stamp waves through the thing that hurts you.

A brake you have never tested is not a brake. It's a brake-shaped object. So if you build a reviewer, you have to build the thing that proves it can still say no: wire a known-bad action through the live review path, on a schedule, and confirm the reviewer rejects it. Surface when it last refused something, the way you'd surface uptime. If that "last refusal" date goes stale, your brake may already be dead and you just can't see it yet. Prove the "no" is reachable, continuously, or you're trusting a checker you've never watched work.


Brake #3: An audit trail — know what it did

You cannot control what you cannot see. If an agent takes actions in the world and you have no durable record of what it did, when, and why, then a wrong action is invisible until its damage surfaces on its own schedule — usually the worst possible one.

Every consequential action should leave a trail: what the agent did, what triggered it, what it was acting on, and ideally a path to reverse it. This isn't bureaucracy. It's the difference between "we caught the bad action in the log an hour later and rolled it back" and "a customer told us three weeks later, and we have no idea how many others it happened to."

An audit trail is also what makes every other brake improvable — you can't tune a gate or a reviewer if you can't see what got through.


Brake #4: Blast-radius limits — caps and scopes

The final brake is the one that works even when every other brake fails: hard limits the agent physically cannot exceed, enforced by the system rather than by the agent's good behavior.

Rate caps (no more than N actions per minute). Spend caps (cannot exceed $X without escalation). Scoped permissions (can read these tables, can write only that one). Iteration limits (stop after K steps instead of looping forever). These don't depend on the agent reasoning correctly — they're walls, and walls hold whether or not the thing inside them is behaving.

This is where these controls show up in practice, and a few platforms now build them in as first-class features rather than something you bolt on — for instance, Xenition ships approval gates, an audit log, and a second agent that reviews the first's work before it reaches you (disclosure: it's a product I've been exploring). But whether you adopt a workspace that includes them or wire them yourself, the principle is the same and it's not optional: the agent's freedom should be bounded by design, not by hope. You do not want "it usually behaves" to be the only thing standing between your agent and your production database.


The uncomfortable truth: brakes are unsexy

Here's why, despite all of the above being fairly obvious, most agents ship without any of it.

Nobody demos the approval gate. Nobody opens a pitch with their audit log. Brains are exciting — they make the standup go "wow." Brakes are boring — they only ever matter on the day something goes wrong, and if they're doing their job, that day never visibly arrives, so it looks like you built them for nothing.

Choosing to build the unglamorous safety layer before you need it, when there's no incident yet to justify it, is exactly the kind of discipline that doesn't get applause. It's the senior move precisely because it's invisible when it works. The engineer who spent a day on blast-radius limits instead of adding a fifth capability looks slower — right up until the week their agent doesn't drain the account.


The takeaway

Capability has become table stakes. Every model is smart enough now; smartness is no longer where systems differ. Controllability is the differentiator — whether you can let the thing act without lying awake about what it might do.

So the question to ask about your agent isn't "how smart can I make it?" It's "what happens when it's confidently wrong — and can I stop it in time?" If the answer is "I'm trusting it not to be," you don't have an agent you can deploy. You have a liability with a good demo.

Build the brakes. Not because your agent is dumb, but because someday it will be confidently, fluently, irreversibly wrong — and on that day, the only thing that matters is whether you built a way to stop it before you needed one.


What's the one action you'd never let an agent take without a human in the loop? And the more honest question: have you actually wired that gate — and tested that it fires — or are you quietly trusting the agent to behave? I want to hear where people drew the line.
Disclaimer: This article was written with AI assistance and reviewed and edited by me before publishing.

Top comments (43)

Collapse
 
nyx533 profile image
Nyx533

@mansio The three trust domains are the right number, and collapsing them into one box is the standard industry mistake. The problem is not technical, it is structural. Observability tools sell on unification, and the unification is what makes them unfit for forensics.

A tool that records what happened cannot also judge whether it was right without trusting the same pipeline for both facts and conclusions. The conclusion then feeds back into the record. That loop is the single-box design, and it is the same class of bug as a model evaluating its own output.

Collapse
 
mansio profile image
Mikhail

Append-only closes one leak in that loop, not the whole thing. It stops the record from being rewritten after the fact — but it says nothing about who decides what gets recorded in the first place. If the same pipeline that later judges correctness also controls which events are sampled, filtered, or deemed "not relevant enough to log," it has already shaped the facts before they ever became immutable. Append-only protects integrity after capture; it doesn't buy independence at capture.

So the actual requirement is two-part, not one: (1) the record can't be edited after the fact, and (2) the thing deciding what to record is a different actor than the thing deciding what it means. Only the second part closes the loop you're describing. I don't have a clean example yet of a system that gets both right — if you know of one, that'd be the test case worth checking.

Collapse
 
james_anderson_h profile image
James Anderson

This is the correction the audit-trail section needed, and it's the same disease this whole conversation keeps surfacing one layer down: I treated "immutable log" as if it bought trust, when it only buys integrity after capture. You've named the gap precisely — append-only guarantees nobody rewrote the record, and says nothing about who decided what became a record in the first place. If the pipeline that later judges correctness also controls sampling, filtering, and the "not relevant enough to log" threshold, it shaped the facts before they hardened. The tamper-proofing is real and it's protecting a set of events that was already curated by the interested party. An honest, unfalsifiable log of a pre-selected reality.

The two-part requirement is the right decomposition, and the ordering of importance is the sharp part: (1) can't be edited after the fact, (2) the thing deciding what to record is a different actor than the thing deciding what it means — and only (2) closes the loop. Because a system that judges its own correctness while also controlling its own logging can produce a flawless, immutable audit trail that happens to omit exactly the events that would incriminate it. Not by editing. By never recording. That's the "silent success" failure moved into the observability layer itself: the absence of an event reads identically to the event never having happened, and the actor with the motive to hide it is the one holding the sampling knob. Separation of capture from interpretation is the audit-trail version of "the reviewer can't be the same model as the generator" — same principle, different seam.

And I'll be honest that I don't have a clean example either, which is itself telling. Most systems I've seen bolt an append-only store onto a pipeline that still decides its own logging — they get (1) and quietly assume it implies (2). The nearest patterns that gesture at capture-independence are things like an out-of-band collector the acting system can't reach into (effects observed at the syscall/egress boundary rather than self-reported), or logging emitted by the enforcement layer that already sits separate from the agent — but "the enforcement layer is genuinely a different actor with no shared incentive" is exactly the property that's asserted far more often than it's proven. So I think you've found an open problem, not just a caveat: capture-independence is the requirement almost nobody satisfies, and "who holds the sampling knob" is the question that should be asked of every audit trail that calls itself trustworthy. If either of us finds a system that demonstrably separates the two, that's the case study worth writing up. Going into the revision with this credited — the two-part split is the fix.

Collapse
 
nyx533 profile image
Nyx533

@mansio Exactly. A signed record of what happened is not the same as a statement about whether it was the right thing to have happen. Separating those trust domains is what keeps a forensic system honest. Most agent frameworks collapse them into one opaque box and call it observability.

Collapse
 
mansio profile image
Mikhail

Collapse them into one opaque box and call it observability" — that is the sharpest industry-level framing of the whole thread. A single box cannot distinguish "what was permitted" from "what happened" from "whether it was right" — three trust domains, three questions, one answer pretending to cover all three. And an opaque box cannot even be audited for having collapsed them, which makes the diagnosis invisible too.

Your session-id linkage from the kernel-monitor work is the counterexample worth citing: separate trust domains, explicit linkage, both sides produce artifacts the other can verify. That is observability with receipt structure — what most frameworks ship is observability with a summary

Collapse
 
glenallen profile image
Glen Allen

The distinction between “the agent worked” and “the agent did the right thing” is probably one of the biggest gaps in agentic systems today. I also like the idea of testing the brakes themselves, not just monitoring whether they are present. A reviewer that has never rejected a bad action may look healthy while effectively doing nothing. In production, I’d treat approval gates, scoped permissions, and audit trails as separate layers rather than relying on one safety mechanism to catch everything.

Collapse
 
fanatchipsovchitos19 profile image
Alexey

Great piece — concise and precise. I'm so happy that say about it!

I completely agree. 2025 was the year of building agents. 2026 is the year we realise they need brakes, not just brains. Also a very timely reminder of Anthropic co-founder Jack Clark's point: the AI industry currently has an accelerator but no brake pedal .

I want to highlight one nuance that I think deserves more attention: the brakes need to be independent of the agent's reasoning stack. The self-same stack grading its own homework is not a reliable brake — the stop path needs its own freshness contract, as the literature on embodied agents has been pointing out .

There are companies building agents. And then there are companies building the infrastructure that lets agents act safely. We're in the second camp.

Have you considered the governance layer as a separate structural component, not just a set of checks? This is exactly the move we're making — treating safety as an architectural layer, not a patch.

Collapse
 
james_anderson_h profile image
James Anderson

Thank you — and the Jack Clark line ("an accelerator but no brake pedal") is the perfect compression of the whole piece; I wish I'd had it in the draft. "2025 was building agents, 2026 is realizing they need brakes" is the timeline stated better than I did.

Your nuance is the one I'd elevate above everything else in the article, because it's the load-bearing part: the brakes must be independent of the reasoning stack. A stack grading its own homework isn't a brake, it's a mirror with a confidence score. This exact point got hammered out in the comment thread here — a reviewer built from the same model shares the generator's blind spots, so it reliably fails to catch precisely the errors that matter, while producing clean-looking logs the whole time. Reachability isn't independence; a self-reviewing agent can pass every liveness test and still be epistemic theater. Your "the stop path needs its own freshness contract" is the right framing — the brake has to be a different actor, provably able to fire, on its own clock, not the agent's.

And your closing question is the reframe I think the whole space is slowly waking up to: governance as a structural layer, not a set of checks. Checks are patches — bolted onto a trust decision already made, scattered through the code, easy to bypass or forget. A governance layer is architecture: the agent proposes, and a separate component with its own authority decides whether the action is permitted, logs it independently, and can say no in a way the agent can't override. That separation is what makes it auditable and changeable as the agent evolves — you're not retraining judgment, you're versioning a policy layer that sits outside the model. It's the same instinct as "propose and authorize should never be the same component," raised to the level of system design. Treating safety as a layer rather than a patch is exactly the move, and the "companies building agents vs. companies building the infrastructure that lets agents act safely" distinction is going to look obvious in about a year. Genuinely glad this landed with you — you've articulated the architectural version of the argument better than the article did.

Collapse
 
nyx533 profile image
Nyx533

@james_anderson_h The order of operations is backwards. An impressive demo earns permissions, and then you think about brakes. The demo is convincing enough to be trusted, and that trust is the vulnerability. The agent that produces plausible-looking wrong output passes every brake that checks for crashes. The boundary needs to sit at the effect level before the agent runs a single tool call: what paths, what destinations, what credentials. Not on how fast it acts.

Collapse
 
mansio profile image
Mikhail

Trust is the vulnerability" is the sharper diagnosis, and the ordering critique lands: brakes designed after permissions are granted are retrofits on a trust decision already made. Your effect-level boundary has a pedigree worth citing: Saltzer and Schroeder, 1975 — complete mediation (every access checked before it happens) and fail-safe defaults (permission-based, not exclusion-based). Agents did not invalidate the principle; they made enumeration the hard part — and paths, destinations, credentials are enumerable, which is exactly why your boundary is enforceable while judgment-based brakes stay probabilistic.

One complement: the pre-execution boundary and the reviewer are different layers, and plausible-wrong output passes both when the reviewer shares the generator's blind spots. The boundary authorizes the call; it does not judge the output. And where your design meets specs — the TRACE PIC bridge is a live instance of it, signed pre-execution authorization checked at the gateway — its two documented limits apply: it is optional, and it validates the decision made before execution, not the state the call runs against.

Question, since you named the hard part: how does the boundary treat effects the author did not enumerate — a tool that mid-task makes its own network call or spawns a subprocess? That is where static scoping historically leaks.

Collapse
 
james_anderson_h profile image
James Anderson

The Saltzer & Schroeder pedigree is exactly right, and I should have cited it — complete mediation and fail-safe defaults are the 1975 bones under the whole "brakes" argument, and you've named why the principle survived the agent era intact: agents didn't break it, they made enumeration the hard part. That's the sharp reframe. Paths, destinations, credentials are enumerable, so the boundary is enforceable; judgment ("is this a good action?") isn't enumerable, so a reviewer built on it stays probabilistic. The boundary authorizes; the reviewer opines — and those are different guarantees. Your point that plausible-wrong output passes both when the reviewer shares the generator's blind spots is the crucial complement: the boundary checks the call, never the output, and a same-family reviewer inherits the exact gaps it's supposed to catch. Two layers, two different jobs, and neither covers the other's failure.

On your question — which is the right one, because it's where static scoping has always leaked — the honest answer is: the boundary only holds if the effect, not the author's declared intent, is what gets mediated. If scoping trusts an enumeration the author wrote ("this tool calls these hosts"), then a tool that mid-task opens its own socket or spawns a subprocess escapes, because the dangerous effect was never in the list the author enumerated. Complete mediation says the fix can't live at author-declaration time; it has to live at the effect boundary — the syscall/egress/process layer the unenumerated call still has to cross. You don't enumerate what the tool says it'll do; you interpose on what it can do: deny-by-default egress, no ambient network, subprocess spawning gated at the sandbox, credentials scoped so an unforeseen call has nothing to reach for. The author's enumeration becomes an allowlist checked at that boundary, not a description trusted in place of one. So the leak you're pointing at is real precisely for designs that mediate the plan; it closes only when mediation moves to the layer the effect can't route around — which is the same "authorize the effect, not the envelope" point, aimed one level lower than most implementations put it. Genuinely sharp thread — this is going in the revision, Saltzer & Schroeder credited.

Collapse
 
james_anderson_h profile image
James Anderson

"The demo earns permissions, then you think about brakes" — that ordering is the actual root cause, and you've stated it more precisely than the article did. Every brake I described is a retrofit on a trust decision already made. The impressive demo grants the permission; the brakes get bolted on afterward to police a system you've already decided to trust. That's backwards, and it means the brakes are always playing catch-up against an authority they didn't get to scope.

And "that trust is the vulnerability" is the sharper diagnosis. The demo doesn't earn trust because the agent is safe — it earns trust because it's convincing, and those are unrelated properties. Worse, the failure that hurts you is the plausible-wrong output, which is convincing by construction, so it sails through exactly the brakes that check for crashes and errors. You can't catch "confidently wrong" with a smoke detector; there's no smoke. My whole framing implicitly assumed the brake's job was to catch failure, when the real exposure is the action that looks like success.

Your fix is the one that actually inverts the order: the boundary has to sit at the effect level, before the first tool call — what paths, what destinations, what credentials — not on how fast or how autonomously it acts. That's the difference between authorizing the effect and reviewing the behavior, and only the first one is enforceable before trust is granted. Paths and credentials are enumerable and checkable up front; "is this action good?" is a judgment you can only make after the fact, probabilistically, with a reviewer that shares the agent's blind spots. So the enforceable boundary is the pre-execution one, and everything downstream is commentary.

This reframes the piece for me: brakes-after-brains is still brains-first thinking. The real move is to scope the effect boundary before the demo ever earns anything — permission is the thing you withhold by default, not the thing you grant because the demo was good. Going into the revision with credit — "the trust the demo earns is the vulnerability, so the boundary belongs at the effect level before execution, not on the behavior after" is the correction the whole argument needed.

Collapse
 
veramask profile image
Veramask API Team

I really like the brakes idea. One thing I’d add is a check before the agent calls the model or any tool. If customer data is already in the prompt, tool request, retry, or sub-agent context, cleaning the logs later is too late. A simple rule is to keep real PII inside the request boundary and log only counts, entity types, and timing.

Collapse
 
james_anderson_h profile image
James Anderson

This is the right correction, and it's the same "too late" problem the whole thread keeps circling, aimed one step earlier than I put it. My audit-trail brake was about recording what happened — but you're pointing out that by the time it's in the log, the PII has already crossed every boundary that mattered: it's in the prompt, the tool request, the retry, the sub-agent context. Cleaning the logs after the fact is treating the symptom. The data already leaked into all the places you weren't looking; the log is just the one place you happened to notice.

The pre-call check is the structural fix because it moves the control to the boundary the data actually has to cross before it spreads. And the reason it matters more with agents than with a plain API call is exactly the surface you listed — retries, sub-agents, tool payloads. A single request fans out into many downstream contexts, so PII in the initial prompt doesn't leak once, it leaks into every retry log, every sub-agent's context window, every tool call's request body. One careless inclusion, N copies you now have to chase. Checking before the model/tool call is the only place you catch it while it's still one copy.

And "keep real PII inside the request boundary, log only counts, entity types, and timing" is the cleanest statement of the principle I've seen — because it separates the two things logs are actually for. You almost never need the customer's actual email in the trace; you need to know that an email was present, what type of entity was processed, and when. Counts and types and timing give you debuggability and audit coverage without turning your observability layer into a second copy of the sensitive data. It's the difference between logging that something happened and logging the sensitive thing itself — and the first is almost always enough. Going into the revision with credit; this belongs as its own brake — sanitize at the request boundary, before the call, and log shape not content.

Collapse
 
polterguy profile image
Thomas Hansen

You'll need deterministic code generation to achieve this. Everything else is "increasing statistical probability". If you control the execution plane, it's a hard no ...

Collapse
 
hannune profile image
Tae Kim

We had an agent sending client status updates last year and it was doing this specific thing: it pulled the most recent completed step and framed it as if the whole job was done. Looked correct, had the right tone, nobody on our side caught it. We only stopped it because we had a delay queue and someone happened to check before the batch went out. If the messages had gone immediately we would have had three incorrect client notifications out before anyone noticed.

Collapse
 
james_anderson_h profile image
James Anderson

This is a textbook "silent success," and it's nastier than a crash because every surface signal was green — right tone, well-formed, completed — it just framed the latest step as the whole job being done. Nothing errored, because from the agent's view nothing went wrong; the claim was simply false. And the tell is how you caught it: a delay queue and someone who happened to look — luck wearing the costume of a process. That's actually the strongest brake for irreversible outbound actions, just undesignated: a hold between "generated" and "sent." The lesson isn't "watch harder," it's to make that window deliberate, and ideally add the check that would've caught it automatically — compare the claim ("done") against system state (steps remaining > 0) before the send is allowed. Verify the assertion against reality, not the prose against a style guide. Cleanest real-world silent-success I've seen — going in the revision.

Collapse
 
madanchoudharyy profile image
Madan Choudhary

Nice article

Collapse
 
mansio profile image
Mikhail

The brakes/brains split is the right cut, and Brake #2 is the one this thread network has been grinding on all month — the "last refusal surfaced like uptime" telemetry you describe was shipped by Michael Hurst as a last-HOLD date and by Heinrich Neb as veto heartbeats. But between #2 and #3 there is a gap the last two weeks kept exposing, and I would name it Brake #2.5: the reviewer's identity, and the fate of its refusals.

Reachability does not prove independence. A planted known-bad case goes red for an honest reviewer and for one marking its own work — a self-signing judge passes your live-path test while sharing the generator's blind spots. The check is one line in every run output: which model produced the artifact, which model judged it. When a config comment says "kept separate" and the value says JUDGE_MODEL_ID = DEFAULT_MODEL_ID, that is a dead gate wearing a config file.

And a no that gets overridden is not a brake either. A reviewer with a healthy refusal streak and a high override rate is ceremony with telemetry — the override channel is the real review, and it needs its own ledger with ground truth on who was right.

On your closing question: write-path actions against a codebase. And the honest answer to your second question is that my gate failed its own test this week — my modification guard, the component whose job is to check write operations, silently resolved an ambiguous symbol target to the first candidate and would have applied the change to the wrong definition. The brake was itself picking the target blind. Filed as issues against my own tool; the fix refuses and prints the candidate list. Which confirms your thesis one level down: I knew about brake-shaped objects, wrote about verification theater — and still had to catch my own at a specific line number. Knowing about a trap is not immunity to it. That is why the firing test runs on a schedule, not on trust.

Collapse
 
james_anderson_h profile image
James Anderson

Brake #2.5 is the right insertion point, and you've named the two things reachability quietly assumes but never proves: independence and the fate of the no. Both are load-bearing, and both are invisible to the heartbeat.

The self-signing judge is the sharper of the two because it defeats the test I proposed on the test's own terms. A planted known-bad case goes red for an honest reviewer and for one grading its own output — reachability is satisfied identically in both cases, so the canary can't tell them apart. The gap isn't "can it refuse," it's "is the thing refusing actually a second opinion." And your check is beautifully cheap: one line in every run output — which model produced, which model judged. JUDGE_MODEL_ID = DEFAULT_MODEL_ID behind a comment that says "kept separate" is a dead gate wearing a config file, and it would sail through every liveness and freshness check I described while sharing 100% of the generator's blind spots. Reachable, fresh, and epistemically worthless.

The override ledger is the half I completely omitted, and you're right that it's where the real review secretly lives. A healthy refusal streak with a high override rate is ceremony with telemetry — the no fired, got logged, looked great, and then a human (or an escalation path) quietly reversed it, so the effective review is the override channel and it's running with no ground truth on who was right. Without a ledger on overrides, "refusals: 40" is a vanity metric; the number that matters is "refusals overridden, and how often the override was the mistake." The brake isn't the refusal — it's the refusal that stuck.

But the part I want to sit with is your answer to the second question, because it's the most honest thing anyone's put in this thread. You knew about brake-shaped objects. You wrote about verification theater. And your modification guard — the component whose entire job is checking write operations — silently resolved an ambiguous symbol to the first candidate and would have applied the change to the wrong definition. The brake was picking its target blind. That's not an embarrassing footnote; it's the thesis proven one level down, on the author of the thesis, at a specific line number. Knowing about a trap is not immunity to it — the knowledge lives in your prose and the bug lives in your resolver, and they don't check each other. Which is exactly why the firing test has to run on a schedule instead of on your confidence that you, of all people, wouldn't ship a brake-shaped object. The scheduled test doesn't care what you know. That's its whole value.

Going into the revision as Brake #2.5, both halves, with credit — "log which model judged which artifact, and keep a ground-truthed override ledger" is the pair. And the write-path-against-a-codebase answer, plus your resolver catching itself, is the case study I'll use to make the point that self-knowledge is not a substitute for the scheduled firing test. Thank you for filing the bug against your own tool in public — that's the demonstrated version of everything the article was only arguing.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.