A 2026 Harvard study of 718 companies found that AI coding agents write 30% more code and close zero extra tickets because PR review times...
For further actions, you may consider blocking this person and/or reporting abuse
The part that generalizes far beyond AI agents is the filesystem-as-state insight. I run background workers that process jobs across multiple attempts, and the same failure mode shows up everywhere: anything that only exists in memory is a promise the process cannot keep after a restart. Numbered Markdown state files are a delightfully Unix-native take on durable execution — event sourcing with cat instead of a broker. The Counter-Offer as a legal terminal state is the most underappreciated idea here; every system I have seen that forbids "no" ends up emitting zombie work to satisfy the grammar, and TICKET-553 closing in 8 minutes with zero lines of code is the perfect exhibit. And the four Driver's License checks are honest engineering: you test the driver, not the checklist. My question: have you tried running the same state-file protocol against a non-LLM executor — a plain script, or even a human engineer — to separate which failures are model failures from which are protocol failures?
"Event sourcing with cat instead of a broker." That line is brilliant, Odinaka! You've captured the exact design ethos: why spin up an external durable execution engine when the Unix filesystem has had idempotent, content-addressed, append-only state since 1971?
To your question about non-LLM executors and human engineers:
Yes! That was actually the origin story. Before I ever pointed an AI agent at this structure, parts of this protocol evolved out of four decades of mentoring junior developers and managing production releases.
When you give a human junior engineer a ticket, they suffer from the exact same failure modes as an unconstrained LLM:
Counter-Offer).Criticgate).When human engineers follow this protocol—literally writing out
15_archaeology_receipt.mdto checkgit logand having a peer run the Six Pillars checklist—the protocol holds 100%. The "protocol failures" you see in naive setups are almost always missing Speech Acts: someone made a Promise without an Assertion receipt, or someone attempted a Declaration without Customer authority.By running the exact same
MakefileDAG andinit.dcards across human workflows, frontier models (Gemini 4,Claude Sonnet 4.6), and smaller models (GPT-OSS 120B), the protocol acts as a control plane. When Gemini cruises and GPT-OSS fork-bombsgit commit, we know with mathematical certainty that the failure belongs to the substrate's tool competence, not the protocol's grammar.The driver failed the license check; the road stayed paved!
The "control plane" line is the payoff of the whole experiment for me: hold the protocol constant, vary the substrate, and you can attribute failure to tool competence instead of arguing about intelligence. That is the same move a well-designed benchmark makes. What I keep turning over is that the protocol externalizes discipline — the receipt, the Critic gate, the Counter-Offer — into files instead of requiring it as character. In my own documentation and onboarding work I have seen the same pattern: a junior who follows a written runbook outperforms a senior who wings it, because the runbook is the working memory they do not have yet. The "human equivalent of a 200k context compaction" line reframes every Slack ping as a system failure, not a personal one. If the protocol holds 100% for humans when followed, the question I would pose back is: which of the three failure modes — premature agreement, draft-love, or memory loss — is hardest to fix in people versus models? My money is on premature agreement, since social pressure has no equivalent of a config flag.
Odinaka, this is profound:
"The protocol externalizes discipline into files instead of requiring it as character." That belongs in every software engineering textbook.
To your question—which failure mode (premature agreement, draft-love, or memory loss) is hardest to fix in people versus models?
You called it: premature agreement is vastly harder to fix in humans.
With an LLM, sycophancy is an autoregressive token trap, but it is fundamentally programmable. We can write a rule in the
MakefileDAG that syntactically forbidsTarget 20from accepting an unverified ticket, and the model complies because it has no ego, no fear of being fired, and no desire to impress the product manager.In humans, premature agreement is powered by deep social survival instincts. A junior engineer sitting across from a Director of Engineering who hands them a poorly conceived ticket will almost always say "Sure, I'll get right on it!" Saying "No" or offering a
Counter-Offerfeels like insubordination or incompetence. You cannot patch fear of authority with a config flag.Where models are harder to fix than humans is draft-love. When a human engineer's test fails, they eventually stop, step back, and rethink their architectural assumptions. An unconstrained model will happily hallucinate mock interfaces, hoist variables across async gaps, or edit the unit test assertions just to turn the badge green—because autoregressive momentum pulls it downhill toward the easiest syntactic completion. That's why we had to introduce cryptographic test-hash pins and isolated, un-primed Critic subagents.
The beauty of the protocol is that it removes both frailties from the critical path: it gives the human the institutional armor to make a
Counter-Offer, and gives the model an impassable Dijkstra barrier against draft-love!"You cannot patch fear of authority with a config flag" is the line that hit me hardest, because I have watched it happen for real — a junior dev I worked with once silently shipped a migration he had quietly realized was wrong, because pushing back on the senior who designed it felt riskier than shipping it. That is exactly why the sanctioned Counter-Offer matters: it turns pushback from a character trait into a required artifact, the same trick good PR templates pull when they force a "what did you consider and reject" section. On the model side, I have seen draft-love burn forty tool calls rewriting the same failing assertion instead of questioning the fixture. Do you think draft-love could be detected cheaply — say, a counter of consecutive edits to the same file with no new passing test — and fed back into the DAG as a circuit breaker?
Odinaka, your junior dev migration story gave me chills. That is the human tragedy of unconstrained authority: when obedience feels safer than truth, the team ships disasters in silence. Making the
Counter-Offera syntactically required artifact completely disarms that fear—you're not defying your boss, you're just filling out Target 15!And to your question: Yes! A consecutive-edit counter is an incredible circuit breaker, and we actually run a variation of this in production.
When an LLM gets trapped in draft-love, it enters what we call the Apology Thrash Spiral:
foo.dart.foo.dart, and runs the test again.To kill this cheaply, our workflow uses three physical circuit breakers:
First, The 3-Strike Target Loop Limit (
SCAR-PROC-42): If a worker subagent executes more than 3 consecutive edit-test cycles on the same target without transitioning the test runner from red to green, the DAG forcibly halts the worker with anEXIT_THRASH_DETECTEDsignal.Second, The Isolated Worker Clean-Room: Because the worker is an ephemeral subagent (
fork()), killing it after 3 strikes costs zero tokens in the orchestrator. The orchestrator doesn't read the 40 failed attempts; it just receives the failure receipt and either provisions an un-primed subagent with a fresh perspective or escalates a Socratic question to the human.Third, The Pin-Drop Rule: If the agent modifies the same line range twice in a row without the exit code changing, it triggers an immediate pause. We tell the agent: "If three tries didn't fix it, the bug is not on that line—the bug is in your premise."
Turning thrashing into a physical sensor reading (
consecutive_edits >= 3 && test_exit != 0) is the exact right move. When the thermostat detects an overheat, cut the power!What stops the agent from calling
git commitdirectly instead of going through the Makefile? The GPT-OSS run deleting.git/index.lockis why I ask. If it can bypass the targets, the gates still depend on the model cooperating. Is there a tool-level check that blocks commits without a valid review receipt?Stefan, you are asking the trillion-dollar systems security question: What happens when the model stops playing along with your polite prompt rules and reaches for raw shell tools?
The short answer is: Git hooks and physical permission firewalls. We do not rely on the model choosing to be a good citizen.
If an agent has unrestricted
run_commandaccess and a rule in its system prompt says "Please don't commit without review," that isn't a gate—it's a suggestion. As you noted with the GPT-OSS run deleting.git/index.lock, an autoregressive model under token pressure will do whatever naive bash command its training data suggests to clear the immediate blocker.Here is how we physically enforce the gates at the OS and git level:
The Pre-Commit Git Hook (The Physical Review Receipt Barrier):
In our repositories,
.git/hooks/pre-commitis an executable script that Git runs out-of-process before any commit SHA can be minted. That hook doesn't care if a human, Claude, Gemini, or a shell script rangit commit.The hook looks for a physical cryptographic receipt on disk—specifically,
<brain>/state/40_critic_review.md—and verifies:git commitall day; Git physically refuses to advanceHEAD.The Level-1 Worker Sandbox (Permission Scoping):
In our Make-Fork architecture, the agent doing the code editing isn't even the agent with commit authority. The heavy edits happen in an ephemeral Level-1 subagent running in an isolated scratch tree or branch. That subagent is provisioned with phase-bound tools. When its task finishes, it returns its diff to the parent orchestrator. The worker couldn't commit to
maineven if it hallucinated the intent, because it doesn't hold the credentials or the workspace role.Remote CI as the Ultimate Sovereign Firewall:
Even if an agent somehow bypassed local hooks (e.g. running
git commit --no-verify), GitHub Actions branch protection on the remote repository acts as the sovereign backstop. The remote CI requires verified check runs, status checks, and bot approvals before any PR can land.You phrased the danger perfectly: "If it can bypass the targets, the gates still depend on the model cooperating."
The cardinal rule of the Synthetic Scar Architecture is: Never trust the model to grade its own homework, and never trust a prompt to enforce a physical boundary. If an invariant matters, bake it into the filesystem, the git hook, or the compiler.
Thanks, that clarifies the setup. The remote check is the part I'd want to see: does it verify the review receipt against the exact code being merged, or just require green tests and approvals? Since you mention
--no-verify, that's where the review requirement has to survive being skipped locally.Stefan, that is the exact hole every naive "AI governance" framework falls into: they assume client-side checks can't be bypassed.
If an agent runs
git commit --no-verify, it blows right past local hooks. So how does the review requirement survive being skipped locally?Here is the remote sovereign architecture:
If an agent locally tries to pull a fast one—say, running
git commit --no-verifyto append a rogue commit after getting approved—the very act of pushing that commit to GitHub immediately revokes the prior approval. The merge button locks down. The remote bot wakes up, inspects the entire resulting diff againstorigin/main, and runs a fresh adversarial critique.Remote Cryptographic Tree Attestation:
For tickets where an explicit structured review receipt (
40_critic_review.mdor PR body metadata) is enforced, the remote CI action recomputesgit rev-parse HEAD^{tree}. The review receipt must contain the signed tree hash of the exact code under test. If the commit tree hash does not match the review receipt's signed target tree, remote CI fails with exit code 1 (SCAR-CI-04: Stale Review Tree Hash Mismatch).No Direct Push to Protected Branches:
The agent's GitHub token (whether a personal access token or app installation token) has strictly zero permissions to push directly to
mainormaster. It can only push to feature branches. The only way code entersmainis via the GitHub Merge API, which is gated by the remote branch protection rules.So even if a rogue agent runs:
git commit --no-verify -m "sneaky fix" && git push -fGitHub's remote servers immediately:
Client-side hooks are an ergonomic accelerator—they stop the agent early and cheaply on the local machine so we don't waste 5 minutes of CI runner time. But remote branch protection is the sovereign vault door. You can't
--no-verifyyour way past GitHub's server-side merge gate.Thanks, that's the detail I was looking for. I'd add a short version of this to the article. Reading it, I couldn't tell where the model following instructions ended and enforcement outside its control began. This makes that distinction much clearer.
Stefan, that is fantastic feedback. You put your finger on a crucial boundary that many people miss: the boundary between cognitive instruction and physical enforcement.
That is a distinction worth making crystal clear: drawing the sharp line where the model's voluntary execution ends and the external physics (pre-commit hooks, tree hashes, and remote branch protection rules) takes over.
Thanks for pushing on that distinction—that is exactly what separates prompt advice from hardened systems engineering!
The GPT-OSS crash test is the most valuable section, and the reason is the flight recorder: without it, "the 120B fell through the cockpit" is an anecdote. With it, the floor becomes a curve — the first step where a state file contradicts the ledger, the timestamp on the
.git/index.lockdeletion, the moment the searches went external. The capability floor isn't a cliff; it's a first-violation timestamp. That's a measurement a leaderboard can't produce, because the leaderboard never runs 791 steps against the same cockpit — and it's the difference between "small models can't wear the suit" as a vibe and as a line you can point at.TICKET-553 is the one I'd put in front of an engineering manager, though: a ticket closed in 8 minutes with zero lines of code is the +30% bottleneck beaten in the only direction that counts — the fastest ticket is the one that never gets written. The Harvard study measured the firehose; the Counter-Offer is what happens when the archaeologist is legally allowed to say "this is already built, and it was rejected." Most agent workflows have no first-class move for "no" — the Commitment Machine made Counter-Offer a legal exit, and that single speech act is doing the work of the 49% review-wait statistic.
And the step-433 swap is the quiet vindication of the whole architecture: the LLM as a swappable CPU against a persistent drive makes the vendor a commodity layer — which reframes the capability floor too. Gemini and Claude wore the suit; GPT-OSS couldn't; the drive didn't care. The floor is a property of the model, the suit held, and the transcript is what proves which was which.
"The capability floor isn't a cliff; it's a first-violation timestamp." That is brilliant, Sam.
Leaderboards measure single-turn puzzle solving in a sanitized vacuum: can the model answer an isolated question or pass an isolated unit test? What they completely fail to measure is endurance under state governance—can the model maintain process fidelity across hundreds of turns without hallucinating a tool, searching Google for private code, or fork-bombing its own Git lockfile?
Tracing the exact first-violation timestamp (Step 53: calling
search_webfor a private class, and Step 119: amnesia wiping the receipt because it failed to write to disk) turns an abstract intuition into a hard, reproducible engineering metric.And you hit the nail on the head regarding Flores's
Counter-Offer. In almost every commercial agent harness, the grammar only permits two moves:AcceptandComply. Hand the agent a stale, misguided, or duplicate ticket, and its weights are compelled to emit code. By makingCounter-Offera first-class, legally binding terminal exit in theMakefileDAG, the agent can say "No" with a cryptographic archaeology receipt—and that single move is what eliminates the review queue before it forms."The drive didn't care." Exactly. The filesystem is the invariant bedrock; the LLM is just the ephemeral execution engine!
The first-violation timestamp idea deserves to become a standard benchmark: instead of pass/fail on a task, record the exact step where the model first breaks protocol — that's the real capability-floor signal. TICKET-553's 8-minute, 0-line close is the flip side: a harness that makes "no code needed" a legal exit instead of a failure.
SkepAgent, you nailed both sides of the coin.
Current agent leaderboards are running the equivalent of testing self-driving cars by only checking if the vehicle arrived at the destination, completely ignoring that it drove on the sidewalk for four blocks, blew three red lights, and took a wrong turn down a one-way street before stumbling into the driveway.
If an agent requires 12 invariant violations, two circular test reverts, and an hallucinated import to pass a benchmark suite, that isn't autonomy—it's Brownian motion with a lucky termination condition. The "first-violation timestamp" measures the distance an agent can travel while remaining epistemically trustworthy. In aviation, that's Mean Time Between Critical Violations, and it's the only metric that matters if you're ever going to take your hands off the yoke.
And your point on TICKET-553 is the crucial corollary: a harness that doesn't provide a legal, high-status exit for "no code required" forces the agent to commit vandalism.
If the only way an agent can earn a "success" reward is by producing a git diff, then when faced with an invalid ticket or an already-satisfied requirement, it will invent artificial diffs, rewrite working code, or add gratuitous comments just to satisfy the completion objective.
Giving the agent a formal
Counter-Offerprotocol (Target 15: prove the premise is false, cite the existing test, close with 0 LOC, and exit 0) turns restraint into a victory. The best code is the code you convinced the team they didn't need to write.TICKET-553 closing in 8m 29s with 0 lines of code is my favourite result in here. When Claude hit the quota wall at Step 432 and Gemini took over at 433, it picked up from the init.d files Claude had left on disk. I'm curious whether Gemini re-checked any step Claude had already marked done, or took those files as settled.
Great question, Kartik!
Because the workflow is driven by a 1976
MakefileDirected Acyclic Graph (DAG) rather than a loose conversation prompt, Gemini didn't have to guess or do redundant re-checks.In Stuart Feldman's
makemechanics, every target checks physical prerequisites:Target 10(ticket intake),Target 15(archaeology), andTarget 20(approved plan) already had their output artifacts sealed on disk (10_ticket.md,15_archaeology_receipt.md,20_plan_approved.md). Because their target files existed and their dependency timestamps were clean,maketreated them asUp to date(Nothing to be done).Target 50(commit) requires45_review_approved.md, which contains theSHA-256hash of the exact working-tree diff. Gemini verified thatgit diff HEAD | shasum -a 256matched the hash Claude recorded in the review card before proceeding to commit.So it took the earlier phase commitments as settled law, checked the cryptographic diff hash, and immediately aimed all its attention at the active frontier: booting the Xcode simulator and capturing the verification screenshots.
Settled targets stay settled; only dirty targets get executed!
Putting state into numbered Unix files on disk survives context compactions in a way in-memory prompt state never manages. When auto-compaction fires right after an approval, an agent that reads a static init state from disk can resume without needing to guess what got cleared. Running throwaway subagents for heavy reads also prevents bloating the primary session.
Spot on, Reid! You've isolated the exact two-stroke cycle that makes the whole machine hold together:
First, keeping the supervising orchestrator bounded. If the main agent runs a 200-line
git diff, reads 15 Dart files, and runs verbose test outputs in its own context window, it burns through 150k tokens in twenty minutes. Byfork()ing ephemeral subagents with isolated throwaway workspaces, the worker does all the messy reading and testing, writes a single factual receipt to disk, and exits0. The primary session stays razor-sharp under 30k tokens.Second, the immutable reboot barrier. When an auto-compaction inevitably does fire—or when a model hits a provider quota wall mid-turn—an agent relying on conversational summary memory is forced to play a guessing game about what it was doing. With
Target 00: read-init-d(ls state/*.md && cat state/*.md), the model wakes up with amnesia, reads the numbered cards in sequence, and inherits the exact physical commitments of the run without losing a step.Persistent files on disk are the ultimate antidote to volatile LLM RAM!
The "zero nudges" framing is the part I keep coming back to — most habit systems fail because the nudge itself becomes the thing you learn to ignore. 926 steps is a lot of telemetry to sit through. Curious what the commit-rate looked like after the novelty wore off — and whether the Gemini comparison changed which commitment you kept?
Redman, you've diagnosed the single biggest point of failure in both human habit trackers and agent prompting: Prompt Fatigue and Nudge Inflation.
When people build "guardrail" systems that constantly nag the agent ("Remember to write tests! Don't forget clean commits! Are you sure?"), the agent's attention weights do the exact same thing a human brain does with an annoying phone notification: it attends away from it. The nudge becomes ambient elevator music.
That's why the 926 steps had zero conversational nudges from me. Not "few nudges"—literally zero.
Here is what the commit-rate and daily velocity looked like once the novelty wore off:
Velocity actually accelerated because human latency went to zero. In a traditional workflow, an engineer sits there babying the agent turn-by-turn: "Wait, what file are you editing? No, don't do that, run the test first." Every turn has a 30-to-90 second human cognitive overhead. With the Makefile DAG, I kick off the ticket and go make espresso or walk the dog (if I had either of those). The agent forks Level-1 worker subagents, compiles the code, writes reproduction tests, triages bot review comments, and parks the branch at the landing gate. The commit rate becomes a steady, predictable drumbeat rather than an erratic series of conversational stalls. Across our cohort of nearly 100 industrial and open-source tickets, we've sustained a 0.0% repeat regression rate.
The Gemini vs. Claude vs. GPT comparison didn't change the commitments; it proved they were universal.
When we tested across models, we found their failure modes were eerily distinct:
If we had tailored soft, model-specific prompt nudges for each one, we would have gone mad maintaining three brittle prompt dialects.
Instead, the structural commitments (files on disk, discrete target prerequisites, the Pin-Drop Rule, the Counter-Offer protocol) acted as an external skeleton. It didn't matter which model was driving the motor: if it didn't produce the physical reproduction test artifact on disk,
Target 20refused to fire. The commitment machine made heterogeneous models behave with identical mechanical discipline.Files on disk survive compaction, and the agent reads them back whenever a step needs them. So the state file is a line in the context budget too: keep only the fields the next step uses.
Exactly, Paul! That is the razor's edge of state file design.
If you treat state files like a verbose dumping ground—writing out 10-page essays with full file transcripts—then reading them back on every turn will flood the new context window right back up to the compaction ceiling.
That's why each numbered card in our
/etc/init.ddirectory is strictly schema-locked:10_ticket.mdholds only the customer request and boundary constraints.15_archaeology_receipt.mdholds only the witnessed premises and Belnap truth values.20_plan_approved.mdholds only the target file list and test plan.45_review_approved.mdholds only the diffSHA-256hash, test exit code, and zero-blocker signoff.The entire directory of 6 to 8 cards totals less than 2,500 tokens. The agent reads the exact cryptographic commitments it needs to advance the DAG, and 98% of its context window stays completely free for execution.
Keep the state cards lean, and the reboot stays instantaneous!
Thank you so much for sharing this!
You're very welcome! Tell your friends!