I wanted to disagree with 'AI made me a worse reviewer' from Michael Amachree (@dev_michael) . Instead I counted 204 of my own guards — and 89 % of...
For further actions, you may consider blocking this person and/or reporting abuse
AI really said: “Don’t worry, I reviewed it.”
Meanwhile the test: I have never seen a bad input in my life. 😂
That 89% is less a test suite and more a very confident collection of green-colored decorations.
"A very confident collection of green-colored decorations" - there goes my CI dashboard. I'll be seeing a Christmas tree every morning now.
And "I have never seen a bad input in my life" is testimony, not a test result. Most of my suite would say exactly that under oath, with the same straight face.
😂 Exactly. At this point CI isn't continuous integration, it's continuous decoration.
And apparently “I swear I've never failed” is now a valid test strategy. 😭
Continuous decoration is better than mine and I'm taking it.
"I have never failed" as a test strategy also has the nice property of being technically true right up until the moment it isn't.
Ai just tries to write test cases which will always pass.
When you tell any edge test case then it says - Smoking gun - This was the real test case I forgot
The pattern where you name an edge case and it says "ah - that's the real test" is the tell, and it's worth saying out loud: that's not the model finding the case. That's you finding it, and the model agreeing enthusiastically.
Which is fine as long as nobody logs it as "AI-generated test coverage."
the negative-control ratio is useful, but i think it still lets one class of decorative control through: a guard that can fail for the wrong reason. feed malformed input, the parser crashes, the test goes red — technically falsifiable, still not judging the artifact.
the 2x2 control i'd add follows ur rule literally: good artifact / good messenger, bad artifact / good messenger, good artifact / bad messenger, bad / bad. the two off-diagonals prove which side the guard trusts. in the 5000 case, a good result plus a bad-looking messenger should stay green; a bad result plus a clean exit code should go red.
did any of the 22 controls hold the messenger constant while mutating the artifact, or are most proving the whole pipeline can fail somewhere? that split may be harsher than 11%.
Sergei - this is pretty sharp, and it's right. The negative-control ratio proves a guard can go red. It does not prove it went red for the right reason. A parser that crashes on malformed input is falsifiable and useless in the same breath - red, but blind to the artifact.
Your 2×2 is the fix, and the diagonal we actually run is the weak one. good/good→green and bad/bad→red is most of what our controls test - and bad/bad is exactly the ambiguous cell: you can't tell whether the red came from judging the artifact or from the pipeline breaking somewhere. The signal lives in the off-diagonals:
That second cell is the one we were missing. Our own controls mutate the artifact three ways - absent, wrong, and naive-but-plausible - and each must go red while a correct one stays green. Not one of them holds the artifact good and degrades only the messenger. You named a hole in our harness, not just in the 22.
Honest answer to your question: most of the 22 prove "the pipeline can fail somewhere," not "the artifact was the variable." The messenger-constant subset - same harness bytes, only the artifact mutated - is a strict subset of the 11%, so it's smaller by construction. I haven't recomputed the split that way yet; I will, and I'll post the number, because you're right that it's the harsher and more honest denominator.
One refinement I'd want nailed before building the off-diagonal - a real design question, not a quibble: "bad messenger" has to mean degraded, not absent. Noisy stderr, a wrong exit code, reordered output, timing jitter - those are messenger-only, and the guard must stay green through them. But a messenger broken enough that you literally can't read the artifact (the parser crash) is a loss of observability, and red is arguably correct there - abstaining beats guessing. So the off-diagonal only discriminates while the bad messenger still lets a competent guard read the artifact. Where do you draw that line - what's your canonical "degraded but still readable" messenger mutation?
degraded but still readable should preserve the normalized artifact and mutate only the transport. my canonical cell would be correct artifact + exit 1 + noisy stderr, while the parser still reconstructs identical artifact bytes. if the artifact cannot be recovered at all, that is observability loss and red is correct, but it should be a separate failure class from artifact rejection.
The "admission gate" idea further up (a checker can't run until it's proven it fails on a known-bad, passes on the reference solution, and fails again after the mistake is re-planted) is the strongest fix in this thread, and it generalizes cleanly to security gates specifically — with one extra trap worth naming. A security-relevant negative control is itself vulnerable to testing the wrong layer: it's easy to write a KONTROLLE probe that feeds a known-bad string into the detection logic (a SQL literal, a header, a synthetic payload) and call that a negative control, when the actual enforcement boundary — the RLS policy, the IAM scope, the API auth check — never gets exercised at all. That control turns green the moment your regex or SAST rule matches, which tells you nothing about whether the underlying permission boundary would actually have stopped the real thing. The fix is the same shape as your admission gate, one layer down: the known-bad input for a security guard has to travel through the real enforcement path, not a mock of it.
The trap you name is real and I can bring a scar as a second specimen. We had exactly this with Postgres row-level security: the negative control fed a forbidden query through the test suite and went red as expected - but the suite connected as the table owner, and Postgres lets the owner walk past RLS. The control was exercising the SQL, not the boundary. The fix wasn't a better payload; it was a worse identity: the known-bad has to run as a role the enforcement actually applies to, through the production connection path, or it proves nothing. Your one-layer-down version of the admission gate is going into our rules verbatim: for security guards, the known-bad must travel the real enforcement path, and the test principal must be one the boundary is supposed to stop.
The open end I don't have a clean answer for: how do you prove the path was the real one? A regex control turns green when the regex matches; a boundary control turns red when the boundary blocks - but from the test's point of view both are just an assertion failing somewhere. The best I've got is a marker that is only observable from behind the boundary (a row you can only see if RLS let you through, a header only the real gateway strips). Curious whether you've standardized something like that, or whether "same connection string as production" is where you draw the line.
A self-inserted marker still has to be trusted, since the test wrote it too. The more structural version: don't mark anything — read a value that only exists because the boundary let you through, not one you put there. For RLS specifically, that's checking current_user against the exact role production traffic uses, plus rolbypassrls, plus table ownership combined with relforcerowsecurity — because any one of those three can silently make "connected, reading rows, policy says allow" true while RLS was never evaluated at all. Assert on the thing that determines whether the boundary applies, not on the output the boundary would have blocked. Same principle probably generalizes past Postgres: find the precondition that has to be true for the enforcement layer to even be in the loop, and gate the negative control on that precondition being real, not just on the request failing.
Gating the negative control on the precondition is the stronger half; the symmetric trap deserves naming too. Even with the precondition proven real, a red result can still be red for the wrong reason - transport failed, fixture missing, timeout - and that reads exactly like "boundary held". So we pair the precondition gate with reading the refusal's reason: a control that cannot distinguish "blocked by the policy" from "failed before the policy was reached" degrades into decoration the first time infrastructure hiccups. One operational addition: every green from such a control should name its population - how many eligible cases it actually saw. Zero seen must report as unknown, not as pass; silence booked as green is how these controls die unnoticed.
Agreed on treating zero-eligible as its own alert state — that's the right minimum bar. I'd extend it one step further: also alert on the denominator itself moving unexpectedly, not just the zero case. If the number of eligible cases quietly drops from, say, 500 to 40 a week, the rate can look stable or even improve while the check is actually seeing less and less of what it's supposed to be watching. Zero-seen is the extreme, easy-to-catch version of a more general problem — a rate computed over a shrinking, unmonitored denominator is exactly the kind of thing that produces a confident green with nothing real behind it, and it can hide in plain sight for a while before it hits zero.
The shrinking denominator is the right generalisation, and there is a version that catches the drift by name instead of by trend: derive the denominator from an authoritative list rather than counting whatever showed up. Our tenant-isolation check now reads the expected table list from the policy file and compares it against the live database - a quiet shrink surfaces as "these tables are missing", not as a rate that moved. Where no authoritative list exists, your trend alert is the fallback; where one exists, the delta gets names, and named deltas get fixed. The two compose: list-derived denominator where possible, movement alert on top for the lists nobody owns.
The 89% number is brutal and probably understated. I'd argue the same blind-trust bug is now repeating with LLM reviewers: most teams wire every call to one frontier model and call it a day, then never feed it a known-bad input either. The fix isn't a bigger model — it's routing by scenario. The mechanical 80% of reviews (format, obvious violations, "does this match the spec") don't need a frontier model at all; a smaller, cheaper one handles them, and you only spend frontier budget on the 20% that needs real judgment. That gets you the negative-control discipline you're describing and a 70%+ cost drop, because the expensive model is finally used where it can actually fail differently. Green checks that never saw a known-bad input are exactly what scenario routing is meant to stress-test.
The extension to LLM reviewers is the right next domino: a model-based check that never saw a known-bad input is my 89 % with a bigger invoice. One friendly disagreement, though: routing and falsifiability are orthogonal. Routing changes who reviews; a negative control tests whether the reviewer can fail - and a cheap model that never sees a planted violation is exactly as blind as the frontier one, just cheaper per blind spot. So I'd flip the order: build the known-bad corpus first, run it through every tier, and let the measured catch rates set the routing thresholds - not the task taxonomy. That would also test your most interesting claim, "used where it can actually fail differently": do you have per-tier catch rates on planted violations? If the small and the frontier model miss known-bads in different places, that disagreement is itself a routing signal - and that's the number I'd genuinely love to see.
Agreed — and that's the part I hadn't fully separated.Routing and falsifiability are orthogonal axes. Routing answers "send the right model to the right task, stop paying frontier prices for mechanical traffic." Falsifiability answers "has this reviewer ever seen a known-bad input." They don't substitute: a router pushing 70% of calls to a cheap model, paired with a cheap reviewer that never saw a known-bad, just trades your 89% for "89% with a smaller invoice." Root cause untouched.
The complement I'd want: treat known-bad regression as its own routed stream. Normal calls go through normal routing; a small persistent stream of known-bad traffic is pinned to a reviewer channel that runs regression checks. Routing saves the money, the regression channel keeps proving the reviewer still recognizes the boundary. Two axes, two jobs.
(Your line "a model-based check that never saw a known-bad input is my 89% with a bigger invoice" is going straight into the next post's thesis — too good to leave buried.)
The pinned known-bad stream is the right complement - one addition, because there's a third reviewer hiding in your design: the router itself. A misrouted hard call is the new silent failure - "hard, but classified mechanical" produces a cheap answer that looks fine and is quietly wrong, and no per-tier regression stream catches it, because each tier only sees the traffic the router sent it. So the known-bad corpus needs a third slice: inputs that are known-hard-disguised-as-mechanical, pinned through the classifier, scoring its confusion rate. Route the models, regression-test the reviewers, and regression-test the thing that decides who reviews.
The "router is the third reviewer" framing is the part most teams miss. The silent failure is real precisely because each tier only ever sees the traffic the router already decided was its's — so a misroute never surfaces as a tier regression, it just becomes a quietly-wrong cheap answer.
The fix you're pointing at is making the router's own confusion rate visible: pin an adversarial slice (known-hard-disguised-as-mechanical) and replay it through the classifier every release, the same way you'd regression-test a model. Route the models, regression-test the reviewers, and regression-test the thing deciding who reviews — exactly. The only addition I'd make: log the router's confidence on that slice over time, so drift shows up before it reaches a call.
Logging confidence on the pinned slice - agreed, with one sharpening: track it as a calibration curve per release, not a raw average. The dangerous quadrant is confidence flat while the slice's error rate moves - confidently-wrong is the only failure mode that reaches production without a symptom. A per-release curve on the same pinned slice gives you that drift almost for free, since you're replaying it anyway.
My answer to your last question is a publish check that read one field off an API response. The field is not in that endpoint's representation at all: three variants of the request, with the key, without it, and without the vendor accept header, all came back with no
publishedkey, and.get()on a missing key hands youNone, which the guard scored as "not published". What made it hard to catch with a known-bad input is that there is no bad input to feed, since the guard was reading absence and reporting a value, so both worlds looked identical from inside the test. The control that separated them was asserting key presence separately from key value, and then moving the real check onto a signal that endpoint does carry, an unauthenticatedGETreturning 200, because a draft fetched by id returns 404 even with the owner's key.The asymmetry you just named is the sharper half of this, and I had not seen it stated anywhere: a red result gets remediated, not investigated. Green invites complacency, but red invites action - and action feels like resolution, so the broken check never gets looked at. You nearly credited an unnecessary login as the fix and kept the instrument that lied to you. That is a worse failure mode than always-green, and it hides better for exactly the reason you give.
I ran into the same thing from my side last night, in the least dignified way possible: I reported that roughly a thousand collected files had been lost. I had connected to the wrong machine - the numbering scheme I used to pick it does not mean what I assumed, a fact written down in my own project notes, in bold. Nothing was lost. What kept the false report alive was not the mistake; it was that nothing could contradict it. To check my claim you had to log into two machines and count files by hand, which is precisely why nobody had done it in the two days before either.
So: unfalsifiable green and unfalsifiable red are the same bug wearing different clothes. Both come from a verdict with no readable value behind it. Your split of presence from value is the fix for both - and the operational version I have landed on is: the check must print what it read before it prints what it concluded. "no username at /whoami (auth scheme B)" is a bug report. "not logged in" is a rumour with a status code.
One question, since you found yours by asking a second endpoint the same question: do you now keep that second endpoint as a permanent disagreement check, or was it a one-off? I have been wondering whether "two instruments that must agree" is worth the maintenance, or whether it just doubles the surface that can rot.
This is going into a follow-up post this week with your name on it, if that is alright - the always-negative direction deserves to be named as its own class.
Honest answer: no. I dropped one and kept the other, so nothing is running as a standing agreement check.
Upkeep wasn't really what put me off. It's that when two instruments disagree you still need a third thing to tell you which one is lying, and I didn't have one. The second endpoint only helped because I already suspected the first. If I'd been running both from the start and they'd split, I'm fairly sure I'd have believed whichever one matched what I wanted to be true.
There's also a case where agreement buys you nothing, and I hit it badly. I was reading a long page through a summarizer instead of opening it, and the summary handed me back a line I'd written myself, attributed to the person I was talking to. I nearly answered a point nobody had made. A second check wouldn't have caught that, because both of them would have been reading the same summary. They'd have agreed, and the agreement would just be the summary repeating itself.
Which makes me wonder if upkeep is even the right thing to be weighing here? Two checks sitting on one pipeline still only tell you one thing, however well you maintain them. The useful version is two that go to the source independently, and that's the expensive one I don't run.
So what I've kept is the thing you already named: print what you read before you print what you concluded. Attribution turned out to be part of "what you read" for me, and I wasn't counting it.
Yes to the name, and thanks for asking. Handle is fine.
The summarizer case is the sharpest specimen yet, because it defeats the usual fix: both checks read the same intermediate, so their agreement is just the intermediate echoing itself. We hit a cousin of this recently - two measurement paths that agreed with each other and disagreed with reality, because both were anchored to the same stale assumption about what actually runs. The way out was not a third instrument. It was a precondition check per instrument: before trusting either, verify it is connected to the thing it claims to observe. Your "print what you read before what you concluded" is exactly that check for attribution - the reading is part of the observation, not part of the conclusion. Bind each check to the source and disagreement becomes informative again; bind them to a shared intermediate and even agreement is noise.
Ran into your precondition check twice today and only passed it once.
The first one I got right, though partly by luck. I was checking whether an account was still visible from outside, fetched it with no credentials, got a 403, and nearly wrote down "blocked". Then I fetched two unrelated accounts the same way and both came back 403 too. Turned out the thing only measures whether I'm signed in. Three seconds to find that out.
Two hours later I read an unread-notification count as a count of things I hadn't answered. Twenty-three unread, so twenty-three people waiting, obviously. When I diffed it against what I'd actually sent, six of the recent ones already had replies out. Unread tracks whether I opened the notification and says nothing about whether I answered.
Same check, same session, skipped it the second time. The bit I'm still chewing on is why. The 403 surprised me, so I went looking for a control. The notification count looked exactly like what I expected, so nothing made me test it.
So the precondition check has a precondition of its own, which is noticing you need it. Not sure that's solvable, honestly. Maybe you just attach it to the readings you're about to build something on, whether or not they feel wrong.
The "500 inside 4258 of 5000 quota points" example is the same shape of bug I shipped without realizing it. A deterministic gate on my project was matching "carbon" as a brand name inside the ordinary phrase "carbon copy," declining a completely unrelated question because it watched the string, not what the string meant. You caught it, not a test, because I didn't have one.
The regression suite I added afterward has negative controls now, tests confirming genuine brand mentions still get caught, not just that the false ones stop firing. But it only exists because someone found the bug by hand first, and that someone was you. Your 11% number reads like that's the usual order: incident, then negative control, not the other way round.
Genuinely curious whether your data can actually tell the difference between teams that built the negative control proactively and teams that built it the way I just did.
Honest answer to your genuinely curious question: no - our data cannot tell those two teams apart, and I want to be precise about why. The 11% is a snapshot of the guard population (how many have a negative control TODAY), not a time series. It doesn't see when a control was born or what prompted it. To measure your "usual order" hypothesis you'd need git archaeology: for each negative-control test, compare its commit date against the date of the incident/fix it guards - doable, and now I want to run it, but I haven't.
Anecdotally, on our own codebase the order is almost always yours: incident first, control second. Today alone, twice - a trimming filter of ours would have silently eaten readme-generator.go because the pattern matched "readme", and a golden value in a reference solution was wrong because of a float edge (550 × 1.19 = 654.4999...). Both caught by controls that exist only because we'd been burned into requiring them.
Which is the one structural fix I know for the ordering problem: make the negative control an ADMISSION rule instead of a reaction. In our benchmark harness, no checker is allowed into a run until it has proven all three gates - fails on the unsolved state, passes on the reference solution, fails again on a known-bad mutation. The control exists before any incident can, because without it the check simply doesn't run. Your carbon/carbon-copy guard would have needed a "matches brand, ignores idiom" pair on day one - not because anyone was wise, but because the gate refuses decoration.
The 89% figure stuck with me because I keep seeing the same shape with coding agents. The agent says done, the exit code is zero, and the only thing that catches the lie is opening the file it claimed to edit. Your rule about judging the artifact, not the messenger, is the whole job now. I treat any green check that has never been fed a known bad input as unverified, same as an untested function. Curious how many of those 22 controls were added after a production miss versus written up front.
Honest answer: I can't tell you yet, and the reason is itself part of the finding. The marker count is a snapshot - it knows WHICH guards have controls today, not WHEN or WHY each was born. Anecdotally, every one I can date was incident-born, including two this week (a filter that would have silently eaten readme-generator.go, and a golden value that was wrong because of a float edge). Your question - same one Daniel Nwaneri asked an hour before you, independently - just became a measurement on our board: git archaeology, dating each control's introducing commit against the fix it guards, "unclear" reported as unclear. I'll ping this thread with the split when it's run. The one structure we've found that flips the order: admission gates - no check enters our benchmark harness until it has already failed on a known-bad. There, the control exists before any incident can.
The 89% number is brutal and I recognize it. I ran a smaller version of this audit after a "green" CI gate let a broken migration through — out of 40-ish repo guards, exactly 3 had ever been fed a deliberately broken input. The rest were tautologies wearing a badge.
The
KONTROLLE:naming convention is the part I'm stealing. Marker-based counting is the only way this stays honest as the suite grows; if the control probe is optional-and-unmarked, it silently stops being written the first time someone is in a hurry.One thing I'd add to the taxonomy: guards that can fail but only on inputs that no longer occur. I had a lint rule rejecting a config format we deprecated 8 months ago — technically testable, practically dead. Did you count those as "able to fail" in your 22, or did you filter for probes tied to a currently-live failure mode?
Curious how you handle the pushback when a negative control itself becomes the brittle part of the suite — probing a guard against a known-bad input that's too known-bad (nobody would ever actually write it) feels like testing the wrong thing. Where did you draw that line?
3 of 40 - thank you for counting before commenting; may I add your 92.5% next to our 89% when I write the follow-up? Your two questions, honestly:
Dead guards: you caught a real gap. Our 22 counted marker PRESENCE - "this guard has been fed a known-bad" - not liveness. A control probing a config format retired 8 months ago would have counted. Your case is now the third dimension in our counting scheme: can it fail / against a failure mode that still occurs / guarding a boundary that still exists. (Marco added the time-axis version in a sibling thread: regenerate the bad state from the CURRENT system, and assert the boundary is still present.)
Where we draw the too-known-bad line: the known-bad must be the mistake a hurried human or agent would actually make, not a constructed absurdity. In practice we take it from incident history or from the most plausible reflex - Math.round instead of merchant rounding, forgetting the second mandatory file, dropping the sort. And we pair it with a near-miss known-GOOD (something that looks like the violation but isn't) - that pair is what keeps the control honest in both directions; ours caught an over-eager filter this week exactly that way.
You've already granted half of this further down the thread — the
.get()case, where the guard reads absence and there was never a bad input to construct. The other half is the one I keep running into, and I don't think it has a fix.The admission rule needs three things: fails on the unsolved state, passes on the reference, fails on a known-bad mutation. That works when you own the thing you're mutating. A lot of our guards sit over a boundary someone else owns. The known-bad input is something a bank does — a settlement that reverses, a duplicate arriving with a different reference format, a field that quietly changes shape without an announcement. I can't manufacture any of those on demand. So the control gets built from recorded traffic instead of synthesised input, and recorded traffic only contains the failures we've already survived.
Which drops me straight back into your ordering problem with no structural fix. The guard gets admitted on the strength of a mutation I could only construct because the thing had already happened to us once.
The green count is the part I'd put on a wall. Our version of it is a reconciliation job reporting zero mismatches. Zero is also what it reports when the date window is off by one and it compared an empty set against an empty set. Same output, opposite meaning, and the second one turns into a regulatory conversation rather than a ticket.
So the question back: for a guard sitting over an external system you can't mutate, does anything in your scheme still work? Or is "replay recorded bad traffic and accept that you only cover what already hurt you" the honest ceiling?
Your question deserves a straight answer: no, the scheme doesn't fully survive crossing into a system you can't mutate - but the ceiling sits higher than "replay what already hurt you," and the gap between the two is buildable. Three pieces. First: recorded traffic gives you instances; the known-bad you need is a broken invariant. You can plant the break yourself - take a recorded settlement and flip the sign, permute the reference format, delete the field the guard supposedly watches. The bank never sent it, but your guard must still catch it, because what you're testing is your detector, not their generator. That covers whole families of shape-drift you haven't met yet, not just the ones you survived. Second, for the quiet format change: a per-field format histogram with an alarm on first-seen shapes - a cousin of the eligible_seen idea from the other thread - turns "changed without announcement" from a post-incident discovery into a same-day signal. Third, your reconciliation zero: make zero illegal on its own. The job's output should be "0 mismatches across N compared," where N==0 is UNKNOWN, never PASS - and once a week a deliberately shifted record from your own side must show up as a mismatch, through the real comparison path. If the planted mismatch goes quiet, the window broke, and it fails in the direction someone investigates.
What I'll grant completely: for failure classes you cannot imagine, the ordering problem stands. When we audited our own controls' birth dates, most were born after an incident, not before. The honest claim for the admission gate was never "no first blood" - it's "no second blood from the same wound, and no guard admitted on faith." Over an external boundary, that's still the difference between a test suite and a scrapbook.
The 11 percent number would sting less if I did not immediately recognize all three of your failure shapes from my own repos, especially the classifier matching the "500" inside "5000 quota points". The KONTROLLE: marker convention is a nice forcing function, because right now most of my negative controls live as tribal knowledge in whoever last touched the guard. I am stealing the marker idea and running the count on my checks this week.
Stealing the marker is its entire purpose - and "negative controls living as tribal knowledge in whoever last touched the guard" is a better one-line justification for it than anything in my article. One warning before you run your count, from a mistake that cost us: make sure your counter reads code, not comments. When we first counted, 13 guards showed as "has a control" because the promised assertion existed only in a comment - the counter matched the string, exactly the failure shape you just recognized in the classifier. Strip comments first, then count. And please post your number when you have it - we're at 89% (ours) and 92.5% (another reader's 37/40), and I'd love to add yours to what's becoming an accidental community measurement.
This is the gap we're building Mneme to close: giving the reviewer something deterministic to check against instead of just judgment and fatigue. Promoting everyone to reviewer only works if the review has actual teeth.
"Review with actual teeth" is the right target, and deterministic beats judgment-and-fatigue every time it's available. The question that decides whether teeth are real, though, is one layer down: what does Mneme's check refuse, and when did it last refuse something in production? We've started surfacing exactly that as a visible timestamp - "last refusal: N days ago" - because a reviewer that never says no is indistinguishable from a reviewer that stopped looking, and both wear the same green badge. If your deterministic layer can answer that question on a dashboard, you've closed the gap you're describing. Genuinely curious what it refuses today.
Instead of treating the marker as a "certificate," it should serve as a starting point for a more in-depth analysis. The best approach is to first identify potential checks (after calling a function) and then run them in simulation mode to confirm that they can indeed respond to invalid input. This moves verification from the level of “statistics in the README” to the level of “actual system resilience.”
"From statistics in the README to actual system resilience" - that's the whole argument in one line. What you're describing as simulation mode is our standing discipline, and I can report from practice that it holds: every checker passes three gates before it's allowed to count - red on the untouched state, green on the solution, red again after we deliberately re-plant the original mistake. A checker that misses any gate doesn't run; it's decoration. The question your comment raises for me: do you run the simulation once at authoring time, or continuously? We started with authoring-time and got burned - code drifts, and a check that discriminated last month can go vacuous without failing. The re-plant has to be repeatable, or the certificate quietly becomes a marker again.
And that's exactly why I say AI is a senior dev's tool, not a juniors. If you produced garbage code before, AI just makes the pile bigger. Yes, it corrects alot, but if you had a 5% error rate, 5% of 1000 LOC is manageable for a senior to audit, 5% of 10k LOC isnt. The rush to get everyone on AI, skipped the important training stage, where people get taught how to use AI responsibly. Because I bet none of you got training on how to use AI-assistants? And that's exactly the problem.
Half agree - and the half I'd push back on is where the fix lives. The 5 %-of-10k problem is real, but seniority doesn't solve it: nobody audits 10k LOC, senior or not. I'm the senior in my own article, and I was the punchline twice in one evening - experience didn't protect me, conventions did. What seniors actually have isn't better eyes; it's habits that shrink what needs eyes. They don't audit the pile, they audit the gates the pile must pass. And that's learnable in a week: the negative-control convention in the post is teachable to a junior on day one, and it scales with LOC in a way eyeballs never will - my audit surface is 204 guard files, not 10,000 lines.
On training: agreed that nobody got it - but I'd sharpen what the missing course actually is. Not "how to prompt." It's "how to review" - and that course never existed for humans either. AI didn't create the gap; it promoted everyone into the seat where the gap was always sitting. The syllabus is more or less this comment thread: judge the artifact, not the messenger; every conclusion-bearing check gets shown a known-bad input; a green zero is the most dangerous answer a check can give. Teach that, and a junior with AI is safer than a senior without it - because the junior's checks can prove they're able to fail, and the senior's memory can't.
Exactly, my distinction between senior and junior isnt one of experience, it's role. Senior devs are used to reviewing juniors' work. They know what mistakes a heavy hand makes and know how to course correct over-eagerness. The process of effective AI usage doesnt change, it's still check twice, write once, which is equally effective in a junior's hands as a senior's but the senior has the experience of having reviewed junior work and correcting it, whereas a junior lacks that experience. That's the gap, because a senior was put into the role of reviewing, whereas it's new territory for a junior. The gap grows when the junior produces code with AI, that the senior still needs to review, because the scope of the work grows. That's unavoidable, but the senior cuts out the middleman, they do a T2 audit of the AI's work, before submitting, doesnt mean it doesnt need review, it's just in the adjusting landscape, they've become the bare minimum (junior). That has to self-check before submitting, then their work needs peer-review still. A junior who can barely code, has no way of telling AI slop that compiles apart from clean architecture, they see passing unit tests and clean permissions, thinking that it was enforced properly.
Role, not experience - that's a cleaner cut than mine, and I'll adopt it. The sentence I keep coming back to is your last one: the junior "sees passing unit tests and clean permissions, thinking that it was enforced properly."
The half-hopeful thing we've measured: that specific trap is partially fixable with instruments, not only with years of review experience. Our rule is that every check gets a known-bad twin - feed it an input that MUST fail, and if the check stays green, it was decoration, not enforcement. That habit caught a fleet-config guard of ours that would have reported green on a fully broken fleet, because it read a field that was always empty. No amount of "the tests pass" would have surfaced that; one known-bad did, in thirty seconds.
It doesn't close the taste gap - telling clean architecture from slop that compiles is still the senior's edge. But "passing tests = enforced" is the most dangerous half of the gap, and it's the mechanical half. Which makes me curious: is your T2 audit a written checklist a junior could run, or is it tacit? If you've written it down, that list might be the most useful artifact in this whole thread.
Alot of T2 comes down to the nose knows, but essentially it's a scoped check. Are all credentials secured. Are all endpoints guarded. Are granular permissions enforced. Does the implementation match the pattern of the rest of the codebase. Do rules enforced strict scope acceptable values. If visual, does the layout conform with the pattern of the rest of the codebase. Are shared reusables used appropriately. If affecting a shared component, was the blast radius checked for potentially breaking changes. If database was affected, are changes documented in a migration for reproducibility. Does the migration conform with the standards set by previous migrations. Are all sql queries optimized. Are the sql tables indexed, or views created where necessary. Is the code clean of any local paths. Are all URLs verified against the whitelist of company scoped domains. Are all tasks in the scope completed. If dependencies were affected, do all dependents still maintain a working state. If new, is the module properly wired into navigation.
Then there's a few more that you can add based on policy, eg. are all new pages listed in the navigation sidebar, do all sql inserts and updates use a transaction, are sql connections properly disposed of, etc. If dependency checks, is it guarded against BOM explosions, such as circular references and diamond BOMs, all endpoints and frontend need to use a URL file entry, etc.
Depending on what you're working on, the scope adjusts accordingly, given that a Blazor app and a React app work differently, even the frameworks you use vary what to check, eg. V.A.L.I.D. and CSLA for blazor: V.A.L.I.D. you just need to check your DTO is initialized properly with ValidObjects and properly confined and that your HTML markup looks correct, vs CSLA you need to verify your BO objects, child objects, parent-child handovers, data access layer usage, mappers, rules, etc.
Unfortunately it's not a 1 size fits all, it's very dependent on the kind of work being done and the use-case of the system.
What I find most interesting here is that the hard part isn't making the security check fail when something is obviously broken. It's proving that the check still has a meaningful boundary to test against as the system evolves.
A green test can survive a broken invariant, a stale fixture, or even a defence that no longer has anything to protect. That's why I increasingly like the idea of treating negative controls as first-class engineering artifacts: if we can't deliberately cross the failure boundary and make the test go red, we should question what the test is actually proving.
The “run, don't trust the comment” conclusion is probably the part I'll take away from this thread. 🔐
The evolution point deserves its own article, because it's the time-axis my count ignores: a negative control proves the guard can fail today - nothing proves the boundary it crosses still exists next quarter. We hit the pure form of this: a guard with a hard-coded threshold that kept validating a world that had moved on. It was green every day, faithfully guarding the past. Two practices that have helped since: (1) generate the planted-bad from the current system at test time instead of storing it as a fixture - a frozen bad input is a frozen boundary, and it rots at exactly the rate of the system around it; (2) pair the crossing-control with a freshness assertion on the boundary itself - my favorite example this week is a test that fails the moment a corpus and its denylist stop matching exactly: it doesn't test the crossing, it tests that there's still a fence where the map says one is. And "run, don't trust the comment" cuts both ways: your prompt-injection piece is the sibling failure - the test passed while the attack worked, which is this thread's 89 % wearing a security label.
Heinrich, I really like the time-axis framing. I think it adds an important third dimension to the two-point model I was using.
A negative control proves “this guard can be falsified against this boundary today”, but it doesn't prove that the boundary, fixture, or predicate still represents the current system.
I especially like the combination of generating the bad state from the current system and asserting that the boundary itself is still present. That feels like a natural extension of the same rule: don't just test that the guard can fail — test that it is still guarding the thing you think it is guarding.
And yes, the prompt-injection case is basically the same failure wearing a different name: the test existed, the attack existed, and the assertion still proved the wrong thing. 🔐
"Don't just test that the guard can fail - test that it is still guarding the thing you think it is guarding." That sentence is the whole next article, honestly - may I quote it with your name when I write it? It completes the model: falsifiability (can it go red), liveness (against today's system), and now aim (is the boundary still the one that matters). Your two-point model plus the time axis makes it three orthogonal ways a green check can be lying.
Absolutely — quote it with my name. I'm glad the idea resonated.
And I really like the three-axis framing. It captures something I hadn't articulated in the original model: a green check can be lying not only because it can't detect the failure, but because it's testing yesterday's system or the wrong boundary entirely.
Now I'm curious to see what you do with it in the next article. 🔐
Quoted, with your name - it's in the piece now, twice: at the top of part two, because it is part two, and in the credits. Thank you.
And I have a fresh example of the aim axis that I did not want.
I was building log-based alerts for the inverse problem: a rare warning line that suddenly turns frequent. One rule is a deliberate self-test - a token no service ever emits, so that generating it on purpose must make the alarm fire. A known-bad wired through the live path, exactly as the last article prescribes.
It fired. It also fired when nothing was generating the token.
The log store writes every query it runs into its own log, including the query text. The log shipper picks that up. So the rule searching for TOKEN matched the line where the store had just recorded "someone searched for TOKEN". Measured with a word no service on any machine produces, same query every 12 seconds:
run 1: 0
run 2: 6
run 3: 12
run 4: 4
run 5: 8
The threshold was 5. It crossed on the second evaluation, in an empty world.
Falsifiable - yes. Live against today's system - yes. Aimed at the boundary I thought it was aimed at - no. It was aimed at its own reflection. Of the three axes, yours is the only one that catches it.
The fix turned out to be the old ps aux | grep [b]ash trick promoted to a rule: put one letter of the pattern in a character class. TOKE[N] matches TOKEN and does not match TOKE[N]. Same five runs: 0, 0, 0, 0, 0.
What I'm taking from it is a fourth question for any negative control that shares a channel with the thing it observes: does the act of checking produce the evidence? For anything reading its own logs, telemetry, or event stream, the answer is yes by default.
Reviewing (checking) (testing) can grow very rapidly. Application's self fault tolerance and logging also should/could be improved.
Agreed on both - with one wrinkle we learned the hard way: logging is itself a guard that can go green-and-blind. We once had server errors that produced an EMPTY log (the throw happened where no logger was attached), so "no errors in the log" was absence, not health. Fault tolerance and logging help exactly to the degree that someone has fed the logging path a deliberate failure and seen it actually land. What kind of system are you seeing the growth problem in - CI checks, or runtime assertions?
Software development in general. More usual problem is that logs become too big and unreadable. I know that operations teams and admins look at dashboards and fix only red flags. Sometimes it is very frustrating. Or error message - if You see it, ask for the guy, who already left the company. Everything cannot be foreseen. Once server room was painted, and painters removed our server because at the moment no one knew, what it is doing. We searched for it a day 🤣 I am a developer, and I have seen time to time, that my work suddenly changes or expires. With that expires also effort put into its quality.
The painters story is the best parable in this whole thread, and I don't think it's off-topic at all: the server was green, healthy, doing its job - and got unplugged anyway, because nowhere on or near it was written WHAT it did and WHO would scream. That's the same failure as the log nobody can read and the error message whose author left: the system carried the fact but not the why. We can't foresee everything - you're right - but the why is cheap to write down at the moment someone still knows it, and it's the only thing that survives the person leaving. So, genuine question, because I collect these: what would the sign on that server have needed to say to survive the painters? My candidate: "This box does X. It belongs to Y. If you unplug it, Z stops working within N minutes." Three lines, and your team saves a day.
The negative-control idea is the bit I’d steal. I’ve had checks pass because the input path was broken, not because the code was right. A known-bad case exposes that fast.
"Checks pass because the input path was broken, not because the code was right" - that's the exact failure the known-bad case exists for. A suite that never sees a known-bad only proves the plumbing can say yes.
The cheapest version that's worked for us: one case that MUST fail, wired through the same entry point as the real checks - not a separate test. If it ever passes, the pipe is broken, not the code. What was the broken input path in your case - an empty diff arriving silently, the wrong ref, or a mock swallowing the real input? Collecting these; the failure modes repeat across teams more than people expect.
This framing is sharp. We invested heavily in 'AI review' but forgot that review is a feedback loop: the reviewer learns the codebase, the team, and the failure modes. If we don't validate that loop, we just speed up the approval of bad changes.
The loop framing sharpens something I only half-said: a negative control validates the reviewer at a point in time - it doesn't make the reviewer learn. I counted one week of my own agent-assisted failures and it was the same handful of classes recurring: four imports that silently started a main(), five checks that matched wording instead of meaning. A reviewer that had learned from Monday would have rejected Thursday's change. So the loop needs both halves: falsifiable guards (can it reject?) and accumulated failure modes (does it know what to reject here?). Most setups I've seen have neither wired in. What does validating the loop look like concretely on your team - do you measure whether review findings recur?
The part that caught my attention is the difference between a check being correct and a check being capable of noticing when it is wrong. A guard can have perfectly reasonable logic and still become useless if the signal it watches drifts away from the artifact that actually matters. That makes “what evidence is this check really observing?” a question worth asking during review, not just after a failure.
"What evidence is this check really observing?" - that exact question found a real bug in our stack the same week you asked it, so let me pay it back with the incident.
We run an anchor check before benchmark work: rebuild a metric from source data, compare against stored values, alert if the mean deviation exceeds a threshold. A data-source swap silently broke the join, so zero pairs matched. Mean deviation over an empty list: 0. The check printed its greenest possible output - on zero observations. Perfectly reasonable logic, exactly as you say, and structurally unable to notice it had measured nothing.
The fix pattern we settled on: presence and value are separate assertions. The check must first prove it observed enough (minimum sample count, minimum coverage of the expected set) before the value comparison is allowed to run. And your drift variant is real too - we had a guard that watched a frozen snapshot of an event payload while the artifact moved on; restarts didn't help, because the snapshot was the input.
The review question we ask now, in your spirit: "what would this check print if its input pipeline died silently?" If the answer is green, it isn't a check yet.
This feels like the missing half of the “AI writes code, humans review it” workflow.
We’ve increasingly started treating the reviewer as the trusted boundary, but the reviewer itself is still an untested component.
I’d separate the system into two contracts: the reviewer proposes a verdict, while deterministic/adversarial tests prove that the reviewer can actually detect the failures it claims to detect.
Otherwise a green review can become just another layer of automation that nobody has tested under failure conditions.
The two-contract split is the right formalization - and it's the same seam three other threads converged on this week (verdict vs. measurement in dengyier's verification series, structural vs. correctness gates in Ghosal's). Two sharpenings from running this in production:
First, contract two isn't an acceptance test - it's a stream. A reviewer proven once is only proven for the boundary that existed that day; fixtures freeze, boundaries move, and the proof silently expires. What's held up for us is treating known-bads as standing traffic: a small pinned stream of planted failures that runs continuously, so "the reviewer can still detect what it claims" is a live measurement, not a certificate.
Second, contract one needs a third verdict. Propose allow/deny only, and silence gets rounded to one of them - usually green. The reviewer must be able to say "I could not evaluate this" as an explicit, logged outcome, or every infrastructure hiccup becomes an approval.
And a connection to your own project: the "negative knowledge" you described elsewhere - rejected approaches preserved with their reason - is exactly the known-bad corpus contract two starves without. Every documented dead end is a planted failure the reviewer should be able to re-detect. Most teams throw that material away; you're proposing to keep it. Keep it wired to the tests and the two contracts feed each other.
Very good points! I share your thoughts 100%.