The Verification Trap: Why Generative AI Is Making Senior Engineers Slower (and How to Fix It)
TLDR: AI compresses writing code to seco...
For further actions, you may consider blocking this person and/or reporting abuse
"If a linter could have caught it, a reviewer never should have seen it" is the line I'd pin up. One thing I'd add to Fix 1: the size of each AI change matters as much as the tests. A reviewable diff per step is far easier to check than one big PR at the end. For the trust matrix, how do you enforce the red zone in practice? Path-based rules the agent's tooling checks, or just CODEOWNERS and convention?
Thanks — good question. In practice it's path-based, not convention: CODEOWNERS for ownership plus a CI gate that runs before a human sees the diff (Fix 2's static/AST checks), so "the AI's tests passed" isn't the evidence. Your point on per-step diff size is well taken — a reviewable diff per step is far cheaper to check than one big PR at the end. Thanks for the comment.
Makes sense, running the gate before a human ever opens the diff is the part I'd most want to copy. Do the AST checks run on every push or only once the PR is ready? With an agent pushing lots of small steps, running them per push seems like it would catch problems while each diff is still small enough to read.
It PR is in draft mode, then run minimal checks and not full checks - why? It’s expensive. In my current setup, many developers will run draft PR through the system when they could have caught issues locally. However, we build cross platform and developers have Mac only. My guidance is find a trade off with economics and developers experience. What can be tested locally, do it. Don’t waste CI resources on common infrastructure. Once commits are getting out of draft, run the full suite. Engineers need to own the CI agility and cost - and they decide.
That split makes sense, and the Mac-only point explains why local can't cover everything. It maps nicely onto agent work too: let the cheap local checks (lint, types, fast AST rules) run on every small step, and save the full cross-platform suite for when the PR leaves draft. The agent gets fast feedback while the diff is small, and CI only pays for the version a human is actually going to review.
"Syntax is free, verification isn't" is the clearest way I've seen this put. AI code fails in boring-looking places, like the timezone-naive timestamp or the swallowed exception, so you have to read it like it's lying to you. Tests before generation is the fix I'd push hardest, because a test written after the code tends to agree with the code. Did the trust matrix get pushback from teams who saw it as slowing AI adoption?
Yes, writing the tests first, red and getting them to green is one way. However generating the tests directly from spec or PRD CUJ is also faulty if developer lets LLM rule blindly. The teams I am working with, Ai adoption is not exactly a No. Most engineers love vibe coding. Where Ai adoption takes a backseat is when I ask them to adopt Ai in the sdlc instead of just vibe coding. You know one is vibe coding where they have little to zero understanding of what the Ai written code does
That's a sharp line, vibe coding vs AI in the SDLC. One is fun and the other is work, so of course people pick the fun one 😄 Agree on spec-generated tests too: if the LLM writes the tests and the code, it's grading its own homework. Even one or two human-written tests per feature keeps everyone honest.
Fix 1 is the one that actually moved the needle for our team. We started having the agent write a plain-language acceptance checklist before it touched any code — not full tests, just a short "how would we prove this works" list. Reviews went from reverse-engineering intent to spot-checking against that list, and our PR threads got noticeably shorter. The surprise bonus: the agent's code got better too, because it had a target to aim at instead of guessing at ours.
Thanks for sharing this — it's the closest real-world version of Fix 1 I've heard. The detail that the agent's code improved because it had a target to aim at (not because the model got better) is the part people miss: the checklist is a specification, and the spec does the work. Glad it shortened your PR threads.
Thanks, Debashish — and you put it better than I did. A plain-language checklist written before the code forces the acceptance criteria to be explicit, so the reviewer has something real to check against instead of just rubber-stamping generated output. That's why I think of it as a cheap, team-level version of Fix 1: it gets most of the benefit without needing a full formal spec.
The point that verification used to be nearly free because it was fused to writing the code explains the whole thing really well. Alongside the structural fixes, it might help to give review its own block in the day instead of squeezing it between design work. Reading code you didn't write is a different kind of focus, and bouncing between the two costs both.
Exactly, engineers used to estimate verification as part of code development or added that task adjacent to development. It’s tremendous cognitive load when one is reviewing a AI written code especially where reviewer themselves has also outsourced coding to an AI agent.
For example, I seem to have lost the art and patience to read through long documents and often rely on AI to summarize and focus on specific key areas. This also means, writing a long document now is a tedious task and I rely on AI to take key points, write and refine.
Is this a skill degradation? Perhaps. I am hearing my team tell me - with agents working on multiple work trees, they are struggling to merge it all together :) So, we seem to have a new issue
summaries are probably fine for finding where to look. the risky part is when they replace reading the bit you're actually about to change. and merging work from agents on several worktrees sounds like the next bottleneck after review, curious how your team ends up handling that
verification used to be fused to writing, and AI breaking that fusion is what explains why reviews feel so heavy now. the race condition example is the class that kept biting us, code that passes every test because no test exercises ordering.
what helped was requiring the failing test in the same PR before the generated implementation, and tagging anything that touches locks, retries or shared state as human written only. review got faster because the reviewer knew where to look.
how do you decide what goes into the trust matrix, by module or by failure cost?
Thanks for commenting. By failure cost, not by module. The tiers are keyed to blast radius — concurrency, auth, crypto, payment paths, customer data, security boundaries — because the same module can hold both a harmless fixture and a lock-ordering guarantee. Tagging locks/retries/shared state as human-written (as you do) is the same idea at file granularity. Review got faster for you because the reviewer knew where to look — that's the matrix doing its job.
failure cost over module is the right axis. we hit the same thing where one file had a harmless config and a retry loop living side by side, so file level tagging overcounted and module level undercounted. what we do now is tag at the function level for anything touching locks or retries and let the rest go through normal review. does your matrix get revisited after incidents, or is it fixed once the tiers are set?
That’s a great learning, thanks for sharing. In my current team I have not fully managed to cultivate the culture the engineers to bring the learnings back from the CoE after incident and persist the learning. This is a work in progress. I would like to see people revisit or occasionally run even AI on CoE learnings and apply broadly or create tech improvements but sadly it does not always happen. I have come to realize this would take some time. It comes down to the desire to own the platform and treat it with pride instead of seeing that as a chore. Or maybe I need to continue to persist on this
Guarantees invisible in the diff is the core of it. A diff shows what changed, never what an ordering or a lock or a lifecycle now depends on. I'd add one rule to the trust matrix: in any file where order carries meaning, diff size stops being a signal at all.
Thank you — that's a clean addition to the matrix. Where ordering, locks, or lifecycle carry meaning, the diff is exactly the artifact that hides the guarantee, so diff size is worse than useless as a signal there. Adding it to the red-zone rule.