If you want agents to do what you want, you need to learn how to box them into deterministic actions. Unfortunately, this is not what current coding agents are good at. They are designed to be exploratory and open ended. They take a vague prompt, produce a plausible patch, run tests, loop a few times, and so on. This is great for prototyping and greenfield work. But when you need something reliable, when you want the same command to have the same effect every time, the same refactoring to happen to twenty files, the same deploy process to execute in the same order, the vague prompt and the "please read the docs" prompt become a liability.
So you end up with dozens of md files telling your agent to keep reading. That's the wrong place to stop.
Markdown documents aren't constraints. They're recommendations. Agents can read the CONTRIBUTING.md, AGENTS.md, CLAUDE.md, rules file, style guide, runbook, whatever, summarize them, quote from them, say "I've read the docs, and I'll do my best to follow them." They add more and more surface area that needs to be parsed and understood: "Use the existing logging conventions in the repository." Which? Where? "Use small functions." How small? "Run the tests before pushing." Which tests? Which command? Which environment? The agent has to process all of this, balance conflicting directives, and exercise judgment. It's in the judgment where the agents can stray.
In contrast, deterministic actions don't need judgment. "Read the docs and follow the coding style" becomes "Run the ./scripts/lint.sh, ./scripts/format.sh, ./scripts/test.sh scripts." The agent runs them and gets a pass/fail outcome. If one fails, the agent sees the exact error and has nothing to interpret or guess at. "I thought the rule meant…" becomes ./scripts/lint.sh. "Do the refactoring" becomes "Run the ast-grep or jscodeshift or sed tool to apply the transformation." The agent doesn't need to decide which files to update. The script does that, by design. You're boxing the agent into a sequence of deterministic actions.
This is a change in design paradigm from "teach the agent what you prefer" to "translate your preferences into deterministic actions." You can't run a markdown document. You can run a script, a checklist, a Makefile target, a GitHub Action. If you have a Makefile with make setup, make lint, make test, make build targets, the agent doesn't need to understand your coding philosophy. It just needs to run make lint and respond to the results.
The point isn't to dumb down the agent. The point is to make the environment unambiguous. An agent can still be smart when it executes a set of deterministic actions. It can read the failures, make corrections, and re-run the tests. It just can't step outside of the box. It can be clever within the constraints, but the box exists as a series of commands that either pass or fail.
This is why coding agents don't do this well. They want to be independent and explore the repo, read the files, build a mental model, and then make things happen. That works until it doesn't. The more md files you have, the more the agents think they have discretion. "The docs say to use snake_case, but this one existing file is camelCase. Maybe the docs are old." The agent is trying to be helpful, but it's not being deterministic.
Most AI models write replies that are hard to follow. So I added ASD-STE100, a plain-English standard written for aircraft maintenance manuals, to my coding agent's markdown rules. One rule says "Keep sentences to 20 words or fewer." As a script, that becomes MAX_SENTENCE = 20. When the agent ran the script over 10 of its own replies from one session, it found 9 failures. The script found them only after the replies were written, because nothing forced an earlier run. So the script goes in a hook.
I saw CI do this on a fix I sent to Forem. I don't write Ruby, and I had no Ruby installed, so CI was my test machine. My coding assistant wrote the fix and the tests. The first run failed, and the issue was in the test, not the code. That result did not depend on the assistant's opinion of its own work. But CI also passed an unused line, and only the reviewer caught it. A check only boxes in what it tests.
Full story: I Don't Write Ruby. A Fix I Sent Is Now in Forem.
If you want agents to do what you want, don't stop at md files. Write scripts.
Make every "please remember to" into a command. Every "we prefer" into a check. Every "before you push" into a pre-push hook. An agent can skip a local hook, so back it with a required CI job and a merge gate the agent has no rights to change. If a check fails, the agent can make corrections and run it again. That's how you box the agent into a deterministic sequence of actions. It's not a recommendation. It's a contract. And the agent can't ship work that fails the check.
Top comments (3)
The markdown-versus-script trap is real: prose instructions invite negotiation.
When an invariant lives as a markdown paragraph ("keep functions under 30 lines and preserve existing error formatting"), an agent with a crowded context window treats it as a soft recommendation. If a test fails later, it starts explaining why its violation was justified by the surrounding code.
The clean boundary is keeping markdown strictly for high-level intent and delegating every mechanical rule to a script that exits non-zero. Replacing a prose rule about import hygiene with
ruff check --select Ior a custom ast-grep rule leaves no room for interpretation. When a command fails with an exit code and a diff, the loop has an exact receipt to fix instead of a preference to balance. If an invariant cannot fail a command, an agent will eventually negotiate around it.The ASD-STE100 example raises a question about where the check runs: the script caught 9 failures in 10 replies after they were already written, and the hook moves that check earlier. Does the hook validate the agent's outgoing message before it is sent, or does it run on the draft while generation is still in progress? For style rules like the 20-word sentence limit, that distinction matters, since a draft still being revised could fail intermediate checks that a finished reply would pass. On the CI story, where the test failed first and the fault turned out to be in the test itself, how should a failing check be triaged — when does it mean the code is wrong versus the check is wrong, without falling back on exactly the reviewer's judgment the script was meant to replace?
Thanks xcl. The hook is actually the Claude Code Stop hook. It runs after each reply, so incomplete drafts won't be checked. When the reply doesn't pass the check, it loops back to the agent for another try.
Right now it will only try one more time. I tested on 3 prompts, in print mode. The first replies had 18 long sentences in total. For the retries the count was 1, and the replies were also 15-41% shorter. I haven't checked if the shorter replies lost anything. I also need to check if the first reply is shown when interacting.
The line in my article about judgement was perhaps an overstatement in the context of triage. I guess what I meant was that the script takes the judgement out of the run. The script's raw count on my agent's 10 replies was actually 13, not 9, as there were 4 false positives due to a bug in the script, where it doesn't correctly identify sentences ending with bold.
Finding the bug is not a simple pass or fail, it required judgement. That's why I'd use this rule: the agent can change the code, but a person will have to change the check. Although I have yet to enforce that rule.