AI coding agents are getting better at writing software. I use them in my own open-source projects, too.
But there's a distinction I keep coming back to:
Teaching an AI what it shouldn't do is not the same as enforcing what its code is allowed to do.
We ask agents not to break architectural boundaries, introduce side effects, or add unapproved dependencies. Those instructions are useful, but instructions are not guarantees.
Prompts aren't enforcement
Imagine an agent generates this code inside a Domain layer:
namespace MyApp.Domain;
public class OrderService
{
private readonly Infrastructure.OrderRepository _repository;
}
If the referenced type exists, the C# compiler will not normally know that the dependency violates your architecture.
The code may work, compile, and still be unacceptable.
A README or AGENTS.md communicates a rule. It does not independently enforce it.
Move the decision outside the agent
The workflow I want looks like this:
AI agent or human writes code
|
v
Compiler + Analyzers
|
Policy satisfied?
/ \
Yes No
| |
Candidate Diagnostic
for merge |
v
Revise code
The agent can propose a solution. An independent tool decides whether the proposal satisfies the project's checkable constraints.
For C# projects, Roslyn Analyzers let us report diagnostics during compilation. Configured as errors, those diagnostics can make a violating build fail.
This gives AI agents immediate, deterministic feedback rather than another natural-language reminder.
Three open-source projects exploring the approach
ArchitectureAnalyzer: Compile the architecture contract
ArchitectureAnalyzer uses a JSON architecture contract and Roslyn diagnostics to check rules such as layer dependencies, forbidden APIs, and dependency graph constraints.
An architecture document records intent. A machine-checkable contract helps detect drift between that intent and the codebase.
PureSharp: Enforce specific code-level contracts
PureSharp introduces functional-programming-oriented checks for C#.
For example:
[PureMethod]
public static int Add(int a, int b) => a + b;
It detects selected side effects within its documented purity contract. It also checks conventions for immutable local variables and FluentIf termination.
var _value = Calculate();
// _value = 100; // Diagnosed reassignment
This is not a whole-program mathematical proof of purity. It's enforcement of explicitly defined, mechanically checkable properties.
PolicySharp: Allow by policy, deny by default
PolicySharp takes an allowlist-oriented approach.
A denylist catches violations someone remembered to enumerate. A new API or dependency may not appear on that list.
A default-deny policy instead follows this model:
Explicitly allowed -> ALLOW
Explicitly denied -> DENY
Unknown or ambiguous -> DENY
For example, a policy can describe which namespaces a Domain scope is allowed to depend on:
{
"version": 1,
"mode": "default-deny",
"scopes": [
{
"id": "domain",
"match": { "namespace": "MyApp.Domain.**" },
"allow": {
"namespaces": [
"System",
"MyApp.Domain.**"
]
}
}
]
}
This illustrates the policy model; precise enforcement coverage depends on the implemented version.
What if the agent changes the rules?
There's an obvious bypass:
- The agent introduces a prohibited dependency.
- An analyzer reports an error.
- The agent edits the policy or project configuration.
- The build passes.
That's why an analyzer alone isn't enough.
The authority to modify application code should be separate from the authority to weaken the constraints. Policy files, project files, and analyzer references may need independent approvals and CI checks.
Static analysis also has limits. It cannot enforce properties it does not analyze, and a build gate only matters when the project actually runs it.
Freedom to propose is not permission to merge
I don't want to constrain every implementation choice an AI makes.
I want to distinguish freedom to propose a solution from permission to accept the resulting change.
Within the approved boundaries, the agent can explore different approaches. If a boundary needs to change, that should become an explicit design decision instead of a workaround silently introduced during implementation.
Conclusion
Prompts communicate intent.
Analyzers enforce checkable rules.
CI validates changes.
Approval processes protect the rules themselves.
Instead of expecting AI to remember everything it must never do, we can create development environments that reliably detect prohibited changes before accepting them.
That is the direction I'm exploring with ArchitectureAnalyzer, PureSharp, and PolicySharp.
The goal isn't less capable AI. It's more predictable, maintainable AI-assisted software development.
Top comments (5)
The "what if the agent changes the rules" section is the part most setups skip. One way to make separate authority concrete without much process: put the policy JSON, the analyzer package references and
Directory.Build.propsunder CODEOWNERS with a required reviewer who isn't the PR author, and have CI fail when those paths change in the same PR as application code. The agent can still propose a rule change, but it arrives as its own PR instead of riding along with a fix.There's a quieter bypass, though. A
#pragma warning disable, a[SuppressMessage]attribute, or an.editorconfigline that drops the severity to a warning all silence the diagnostic without touching the policy file, so a check on the policy path alone won't see them. Banning suppressions for your own diagnostic IDs, or at least counting them per PR, covers that.Does PolicySharp have a way to mark its diagnostics as non-suppressible?
Thanks, Chris! That's an excellent point, and exactly the kind of bypass I'm trying to prevent.
Currently, PolicySharp reports violations as errors, but it doesn't yet guarantee that its diagnostics cannot be suppressed.
I agree that protecting the policy files alone isn't enough. Suppression directives, attributes, and severity overrides effectively change the enforcement boundary, so they should be treated as policy changes too.
Your suggestion of combining CODEOWNERS, separate PRs for policy changes, and CI checks makes a lot of sense.
I'll look into adding explicit suppression detection and independent CI validation to PolicySharp.
The goal is not just to prevent AI agents from violating the rules, but also to prevent them from silently disabling the enforcement itself.
Thanks for highlighting this gap!
Strongly agree with putting the "can't" in analyzers and CI rather than the prompt. A prompt is advice the agent weighs against everything else in context; a failing analyzer is a fact it has to deal with.
One thing that's worked well: run the same checks inside the agent loop, not only in CI. If the agent can call the analyzer on each file it touches, it fixes violations mid-task instead of producing a PR that bounces. CI becomes the backstop rather than the first time anyone finds out.
Curious whether you've fed analyzer output straight back to the agent, and whether it tends to fix the root cause or just suppress the warning.
Disclosure: I work on Norma at Quality Clouds, which does this per-file checking through MCP. There's a free tier at portal.qualityclouds.ai if you want to compare notes.
Thanks, Nuno! I completely agree that running analyzers inside the agent's development loop is a natural next step.
My goal with PolicySharp is to make architectural constraints machine-enforceable rather than relying on prompts or coding guidelines alone.
Feeding analyzer diagnostics directly back to an AI agent could create a useful feedback loop: generate code, validate it, identify violations, and attempt a correction.
However, your question about how agents respond to diagnostics is particularly interesting.
I don't have enough systematic test results yet to say whether agents consistently fix the underlying architectural problems or simply try to suppress the warnings.
That distinction is important. An agent that makes the diagnostics disappear hasn't necessarily fixed the problem.
I think the next step is to evaluate this behavior explicitly, including attempts to bypass enforcement through suppression directives or configuration changes.
Ideally, the agent should be free to change its implementation, but not the constraints used to validate it.
Thanks for the thoughtful suggestion!
Đúng trọng điểm. Prompt-only guardrails giống như viết quy tắc an toàn lên tường xưởng mà không có cảm biến hay khóa cửa — ai cũng đọc được nhưng không ai bị cản khi vi phạm.
Tại dự án trước, team mình chuyển từ "nhắc trong system prompt" sang rules của static analyzer (custom rules cho ESLint/ruff) + pre-commit hook + CI gate. Kết quả: vi phạm giảm ~90% trong 2 sprint vì feedback loop ngay tại máy dev, không đợi PR review. Còn side-effect tốt: junior dev học pattern đúng từ error message của analyzer thay vì chờ senior nhắc.
Một điểm bài viết chưa nhắc sâu: versioning của guardrails. Khi agent upgrade hoặc model thay đổi, rule set cũng phải co-evolve. Mình thấy hữu ích khi treat guardrails như code — có test, có changelog, có rollback strategy (site: labagent .tech)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.