Most system failures trace back to a constraint nobody wrote down. By the time something breaks, the architecture is usually still fine. What gave way is an assumption nobody agreed to keep.
Early on, most systems are tractable. The architecture is coherent, the abstractions are few, and the number of invariants in play is small enough that one person can hold all of them in their head at once. At this stage the system rewards local reasoning. You can change a component and predict the effect of that change without opening five other files to check what else might notice.
The system's behavior can still be approximated as:
where each component contributes independently, and interactions between components stay limited and explicit. Read a function, understand its inputs and outputs, and you understand what it does to the system. Nothing else needs checking.
That property doesn't survive growth, and it rarely gets killed off by recklessness. It erodes through accommodation. A new requirement adds a conditional path to a function that used to have one job. An exception bypasses an invariant because enforcing it that week would have blown a deadline. A shortcut defers a constraint under pressure, with every intention of coming back to fix it later, and the fix never gets scheduled because nothing visibly broke. Each of these changes looks locally defensible at the moment it's made. The system quietly stops being additive, one reasonable decision at a time, without anyone ever sitting down and deciding to compromise the architecture.
Interaction terms accumulate:
Once interactions grow faster than components, local reasoning stops working. The second term takes over. Behavior starts to look emergent instead of designed, and the instinct is usually to blame the people writing the code, but that's rarely where the fault lies. Coupled systems behave this way as a structural matter. It happens on well-run teams with disciplined engineers just as often as it happens anywhere else, because the mathematics doesn't care how careful anyone was being.
A validation function gets written for one form, on one page. Six months later it's being called from four other places, because it already does most of what those callers need and writing a second function feels wasteful. None of the four callers needs quite the same validation, so each one adds a flag, a special case, an early return for its own use. The function now has to know about four different contexts to do its job in any of them. A helper function turns into a load-bearing wall without any single change making it one, because reuse looked free every single time someone reached for it, long before anyone consciously chose to build it that way.
The four callers didn't need a shared function. They needed four small functions that happened to overlap in places. Leave the overlap. Writing four functions feels like waste in the moment, measured against the immediate task. Each of those four could change, break, and get deleted independently when its caller goes away. The shared version can do none of those things without a meeting first, and the meeting rarely happens until the function is already too tangled to split back apart.
Reuse is one of the few instincts in software engineering that's taught early and rewarded in code review, usually before anyone's had the chance to watch it fail. Writing the same ten lines twice doesn't get anyone promoted, but spotting the duplication and collapsing it into one place does. The complimented version is the one that, three years later, four teams are afraid to touch, and by then nobody remembers who to ask about why it's shaped the way it is.
Restraint, in this context, is what keeps that growth in check: not adding an abstraction before there's evidence it earns its keep. The same instinct holds off on generalizing a function until two call sites need it, and refuses to promise behavior the system can't keep enforcing five years from now, after everyone who understood the original trade-off has moved to a different team.
Boundaries are how restraint gets enforced day to day. Naming them in a design review once, then forgetting them, doesn't count. A boundary is a contract that names the allowed interactions, excludes everything else by default, and enforces the invariant at the interface instead of relying on the discipline of every later caller, most of whom will never read the comment explaining why the invariant exists. When a boundary holds, interaction terms stay contained near it. When it doesn't, complexity diffuses outward to wherever the boundary used to be, and it rarely announces that it's doing so.
A module with no interface at least advertises its own recklessness; everyone touching it knows they're on their own. A leaky one looks safe from the outside while failing, without any noise, to protect anything. Once internal details leak across a boundary, that boundary is a boundary in name only. The invariant it was supposed to protect is no longer enforced in one place. It's now distributed, implicitly, across every caller that has come to depend on the leak, usually without anyone on the team keeping a list of who's depending on what.
Someone returned the internal list because it was three lines shorter than writing a proper accessor. For a while, nothing goes wrong. Then a caller three modules away starts mutating that list directly, because it's right there and it works. A year later, someone tries to change how that internal list is stored, maybe to fix a performance problem, and discovers the change breaks code in a part of the system they've never had reason to open. The boundary failed silently, letting something invisible grow behind it.
At that point the cost of a small change stops being small. A one-line fix requires tracing every caller that might be relying on the old behavior, half of which were never documented as dependents in the first place. Understanding gives way to fear. Teams stop touching that part of the code because nobody can say with confidence what else is wired to it, and finding out is expensive enough that it's easier to build around the problem than through it.
In The Mythical Man-Month, Fred Brooks wrote that conceptual integrity erodes gradually, under a steady accumulation of individually reasonable additions. John Ousterhout, in A Philosophy of Software Design, describes the shallow module: an interface nearly as complicated as its implementation. It doesn't reduce complexity. It relocates it onto whoever has to call it, and hides the relocation behind the appearance of a clean function signature.
Ousterhout's counterpart to the shallow module is the deep one: a simple interface sitting in front of substantial internal machinery. The caller gets a narrow surface to learn. The complexity stays in the module. A file system is the example he reaches for. Opening, reading, and writing a file is a handful of operations to learn, and underneath that handful sits allocation, caching, and recovery from partial writes. None of it is the caller's problem. Most interfaces people call "clean" are shallow in this sense, and most people can't tell the difference until they're the one stuck maintaining what's on the other side of it.
Systems thinking, in practice, is arranging things so that most components need to know as little as possible about the rest of the system. Duplicating ten lines of logic across two modules can be cheaper, over the life of the system, than sharing one function that couples them together for years.
There's an old, unglamorous illustration of this in how Unix pipes were designed: small programs, each doing one narrow thing, connected through a plain text interface that none of them needed to understand beyond "read a stream, write a stream." No program in that chain needs to know how the others are implemented. The boundary between them is the text stream itself, and it's inconvenient in exactly the ways that keep it honest: no shared memory, and no shortcut for one program to reach into another's internals because it's under a deadline.
Not everything from that era aged the same way. The shells wrapping those pipes have accumulated more flags than most people using them could name, and plenty of small Unix tools grew their own tangle of special cases the moment someone needed one badly enough. The pipe itself mostly avoided that fate, though it's worth admitting that some of the credit belongs to the text stream being too dumb to extend rather than to anyone's restraint.
A system that preserves locality lets reasoning scale roughly linearly with size. Doubling the codebase doubles the amount you need to understand to make a safe change, not the odds that the change ripples somewhere unrelated and unrelated-looking. A system that loses locality forces reasoning to become global well before it doubles in size, and past that point it stops scaling in any useful sense, no matter how much hardware or headcount gets thrown at the problem afterward. More engineers on a system with no locality doesn't parallelize the work. It just adds more people who each need the same global context before they can safely touch anything.
In Out of the Tar Pit, Ben Moseley and Peter Marks argue that essential complexity, the complexity a system can't avoid because it's inherent to the problem being solved, is usually smaller than it looks from inside a struggling codebase. Most of what makes large systems hard to work with is accidental complexity, generated by uncontrolled state and interaction. That's what teams end up fighting for years without naming it that way.
A system that grants unrestricted access in the name of flexibility has already given up the guarantees that flexibility was supposed to make possible. A setting changes mid-request and nobody can say why. Six subsystems all have write access to the same configuration object, because passing it around explicitly felt like extra ceremony, and none of them coordinated with the others. Early on that looked like convenience. The flexibility that made the object easy to build with is what makes the bug impossible to trace.
Fixing that particular problem rarely means adding more logging or more code review. It means taking write access away from five of the six subsystems and forcing them to ask the sixth one for a change instead of making it themselves, which is a smaller ask than it sounds like until you're the one on the subsystem that lost its shortcut.
A component can only be given real freedom to change its own internals once the rest of the system knows exactly what it will and won't do at its boundary. Freedom without a known boundary reads as flexibility right up until someone needs to rely on it, at which point it's just risk nobody priced in.
Trust, inside a codebase, tends to come from limits. A component that's explicit about what it will do, and that fails loudly when it can't, gives engineers something solid to build on top of. It lets a team revisit an old decision months later without treating the whole surrounding area as radioactive, the way people learn to avoid the one file nobody fully understands anymore, the one everyone routes around instead of opening.
The same pattern shows up outside software entirely. Charles Perrow's Normal Accidents looks at failures across nuclear plants, chemical refineries, and aviation, and finds that catastrophic failure in tightly coupled systems is usually a property of how densely the parts interact, not a property of any single part failing on its own. In systems that are already tightly coupled, adding more cross-cutting paths, more ways for one part to reach into another under unusual conditions, tends to increase risk. This holds even when every individual addition looked, at the time, like it was making the system more capable or more resilient to some specific failure someone had just seen happen.
Time tests a design more honestly than any review ever will. Systems built to anticipate every future requirement tend to accumulate complexity they never use, carrying the weight of speculative features on the chance someone eventually asks for them. Manny Lehman's Laws of Software Evolution get at part of why: a system that keeps growing without a disciplined structure to contain that growth gets progressively harder to modify, on a schedule that belongs to the system no matter who is maintaining it that quarter. Good engineers can slow the decline, but none of them seem to stop it outright.
Past a certain size, discipline at the boundary stops being optional, whether or not anyone on the team decided to call it that. What remains, after the person who drew the boundary has left the room, is whether anyone kept enforcing it.
Top comments (0)