DEV Community

Cover image for AI Wrote Half My Codebase. The Maintenance Bill Showed Up in Month Three.

AI Wrote Half My Codebase. The Maintenance Bill Showed Up in Month Three.

Debashish Ghosal on September 15, 2026

AI made the first draft free. I treated that as the win. The bill arrived later, not in dollars, in the parts of the system nobody on the team ful...
Collapse
 
murali_gour_13cd7a6a6db2c profile image
Murali Gour

Dean's framing and Sal's example point at the same root: the specification that existed implicitly in the act of writing never got written down. The code encoded an answer to a question nobody stated. When that assumption drifts from reality, whoever fixes it has to reconstruct the original reasoning from behavior rather than from intent.

The part that compounds it for AI-generated code specifically: the model is very good at producing plausible-looking answers to unasked questions. A human writing code from scratch tends to encounter the edge case in the act of typing. The model doesn't encounter it at all unless the prompt forced it to. So the implicit knowledge gap is structural, not accidental.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal

Thank you — "the specification that existed implicitly in the act of writing never got written down" is the cleanest statement of the problem in this whole thread. And you've named the asymmetry precisely: a human encounters the edge case while typing, the model doesn't encounter it at all unless the prompt forced it to. That's why I think the fix has to be structural (make the agent state invariants and failure contract before it edits) rather than reviewer vigilance. "The model is very good at producing plausible-looking answers to unasked questions" deserves to be on a poster. Really appreciate you engaging with it.

Collapse
 
murali_gour_13cd7a6a6db2c profile image
Murali Gour

Genuinely appreciate the kind words. The structural fix you're describing is the right one — asking the agent to state invariants and failure contracts before editing shifts the implicit reasoning back into the visible layer where it can actually be reviewed. The question becomes "does this stated contract match what we actually need" rather than "does this code look right," which is a much more productive thing to review. Looking forward to reading more from you.

Collapse
 
deanlee profile image
Dean Lee

The reason the bill lands in month three rather than week one is the lag between borrowing cognitive margin and having to repay it.

When you write an implementation by hand, the physical act of structuring the logic forces you to build an internal mental model of the edge cases. You spend thirty minutes typing, but you walk away with a resident mental cache of how the components fail. When an assistant generates the draft in twenty seconds, that synthesis phase disappears. The code compiles and the tests pass, but the author never built the cache.

In financial terms, generating code without deep synthesis is borrowing against future triage capacity. The upfront velocity is real, but you are shorting maintenance variance. Month one looks artificially cheap because the code has not faced runtime state drift yet. When a subtle regression finally hits in month three, the on-call engineer has to construct that mental model from scratch under production pressure. The twenty minutes saved on boilerplate turns into three hours of forensic reconstruction.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal

Dean, "borrowing against future triage capacity" is the cleanest framing of this I've seen — better than my accounting metaphor. Borrowing is the right verb because the velocity is real; you're just short a liability that doesn't show up until runtime state drifts. One wrinkle I'd add: the debt isn't uniform. Cosmetic generation debt gets repaid in review. Structural debt — invariants, failure modes, the assumptions nobody wrote down — gets repaid under production pressure, and only if it breaks. That's what stays invisible until month three. Appreciate you shar

Collapse
 
zira125 profile image
Zira

That “cheap half of the lifecycle” framing captures the failure mode well: generation is measured inside the PR, while ownership cost appears later in retries, edge-case debugging, and the next engineer’s time.

A metric I’d add is recovery cost: time from the first failing observation to a verified fix, plus the reruns or rollback steps required. It exposes why passing tests can still be expensive when the spec or failure contract is missing. I also like the change from “does this look clean?” to “what does this assume, and how does it fail?” Asking the agent to state invariants and failure cases before implementation gives review something stronger than code cosmetics to inspect.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal

Thank you, Zira — "recovery cost" is a much sharper name than what I've been calling it, and I like that it's measured from the first failing observation rather than from the PR. That's precisely the window where AI-generated code hides its bill. The "what does this assume, and how does it fail?" reframing is the part I keep coming back to: asking the agent to state invariants and failure cases before implementation gives review something falsifiable to inspect instead of code cosmetics.

For what it's worth, we tried to make one slice of this measurable in CauterRule — the recovery-exclusion gate measures how often a recovered failure is wrongly extracted as a standing rule (measurement/recovery.py). Different angle, same instinct: the expensive part is downstream of the green check. Thanks for the thoughtful read.

Collapse
 
zira125 profile image
Zira

One practical way to make “ownership cost” measurable is to treat each agent task as a lifecycle record, not a model call: task ID, repo and commit, spec version, attempts and retries, tool calls, review findings, rollback or recovery work, and time-to-merge plus time-to-first-fix. Then split failures into missed requirements, integration errors, regressions, and environment or tooling failures. That makes a cheap first draft comparable to the downstream work it created.

I also like the plain-English contract idea in the thread. Before editing, have the agent state the invariants, forbidden side effects, acceptance checks, and recovery path. If it cannot explain how the change fails and is unwound, it is not ready to own a long-lived codebase.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal

This is a great operationalization, Zira — treating the task as a lifecycle record (task ID, repo/commit, spec version, attempts, tool calls, review findings, rollback, time-to-merge, time-to-first-fix) is the right unit of analysis, and splitting failures into missed-requirements / integration / regression / environment is what makes the cheap first draft comparable to what it created. The "if it can't explain how the change fails and is unwound, it isn't ready to own a long-lived codebase" line is going straight into my notes. Two of the sibling projects touch adjacent pieces of your list — ai-loopguard on loop/failure detection and ai-tierforge on tier/cost modeling — but nobody has stitched the full lifecycle record together yet. Genuinely useful framing; thank you.

Collapse
 
salparvez profile image
Sal Parvez | ML Systems

Yes, and for me it showed up in the second-order thing you named: the reasoning that used to happen while writing has to happen somewhere else. Where I moved it is a plain-English model of the system, written before the code. Who may write which value, what counts as true, what the agents are not allowed to do. When the code failed, I went back to the English and found the sentence I never wrote. Cheap to write. Not free to skip. Your cost-per-task number is the other half of it. I run my agents on the same rule: the smallest spend that gets a claim a person will actually sign off on.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal

Sal, "the sentence I never wrote" is the whole bug in one line. The code didn't misbehave so much as it quietly answered a question the spec never asked. I like that you put the reasoning into plain English before the code instead of reconstructing it after — that's the difference between a spec and a postmortem. And your rule — the smallest spend that earns a claim a human will actually sign off on — is a sharper cost-per-task metric than mine. Cheaper isn't the goal; defensible is. Thanks for reading.

Collapse
 
justinwilsonbi profile image
Justin Wilson

Recovery cost needs a baseline. Compare AI heavy and human heavy changes by defect density review time and restore time.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal

Agreed — a cost figure without a baseline is just a scary number. Defect density, review time, and restore time compared AI-heavy vs human-heavy is the right control. That's the same discipline we used for the field test: every claim is paired against a reference corpus and reported with a confidence interval rather than a single average (see the v0.3.1 field test report). If I ever get a clean AI-vs-human restore-time dataset, this is the exact table I'd build. Thanks for pushing on it.