DEV Community

loach
loach

Posted on Edited on

Implementing 'You Can't' for AI Coding Agents

AI coding agents are getting better at writing software. I use them in my own open-source projects, too.

But there's a distinction I keep coming back to:

Teaching an AI what it shouldn't do is not the same as enforcing what its code is allowed to do.

We ask agents not to break architectural boundaries, introduce side effects, or add unapproved dependencies. Those instructions are useful, but instructions are not guarantees.

Prompts aren't enforcement

Imagine an agent generates this code inside a Domain layer:

namespace MyApp.Domain;

public class OrderService
{
    private readonly Infrastructure.OrderRepository _repository;
}
Enter fullscreen mode Exit fullscreen mode

If the referenced type exists, the C# compiler will not normally know that the dependency violates your architecture.

The code may work, compile, and still be unacceptable.

A README or AGENTS.md communicates a rule. It does not independently enforce it.

Move the decision outside the agent

The workflow I want looks like this:

AI agent or human writes code
             |
             v
      Compiler + Analyzers
             |
      Policy satisfied?
         /       \
       Yes        No
        |          |
   Candidate     Diagnostic
   for merge       |
                   v
                Revise code
Enter fullscreen mode Exit fullscreen mode

The agent can propose a solution. An independent tool decides whether the proposal satisfies the project's checkable constraints.

For C# projects, Roslyn Analyzers let us report diagnostics during compilation. Configured as errors, those diagnostics can make a violating build fail.

This gives AI agents immediate, deterministic feedback rather than another natural-language reminder.

Three open-source projects exploring the approach

ArchitectureAnalyzer: Compile the architecture contract

ArchitectureAnalyzer uses a JSON architecture contract and Roslyn diagnostics to check rules such as layer dependencies, forbidden APIs, and dependency graph constraints.

An architecture document records intent. A machine-checkable contract helps detect drift between that intent and the codebase.

PureSharp: Enforce specific code-level contracts

PureSharp introduces functional-programming-oriented checks for C#.

For example:

[PureMethod]
public static int Add(int a, int b) => a + b;
Enter fullscreen mode Exit fullscreen mode

It detects selected side effects within its documented purity contract. It also checks conventions for immutable local variables and FluentIf termination.

var _value = Calculate();
// _value = 100; // Diagnosed reassignment
Enter fullscreen mode Exit fullscreen mode

This is not a whole-program mathematical proof of purity. It's enforcement of explicitly defined, mechanically checkable properties.

PolicySharp: Allow by policy, deny by default

PolicySharp takes an allowlist-oriented approach.

A denylist catches violations someone remembered to enumerate. A new API or dependency may not appear on that list.

A default-deny policy instead follows this model:

Explicitly allowed    -> ALLOW
Explicitly denied     -> DENY
Unknown or ambiguous  -> DENY
Enter fullscreen mode Exit fullscreen mode

For example, a policy can describe which namespaces a Domain scope is allowed to depend on:

{
  "version": 1,
  "mode": "default-deny",
  "scopes": [
    {
      "id": "domain",
      "match": { "namespace": "MyApp.Domain.**" },
      "allow": {
        "namespaces": [
          "System",
          "MyApp.Domain.**"
        ]
      }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

This illustrates the policy model; precise enforcement coverage depends on the implemented version.

What if the agent changes the rules?

There's an obvious bypass:

  1. The agent introduces a prohibited dependency.
  2. An analyzer reports an error.
  3. The agent edits the policy or project configuration.
  4. The build passes.

That's why an analyzer alone isn't enough.

The authority to modify application code should be separate from the authority to weaken the constraints. Policy files, project files, and analyzer references may need independent approvals and CI checks.

Static analysis also has limits. It cannot enforce properties it does not analyze, and a build gate only matters when the project actually runs it.

Freedom to propose is not permission to merge

I don't want to constrain every implementation choice an AI makes.

I want to distinguish freedom to propose a solution from permission to accept the resulting change.

Within the approved boundaries, the agent can explore different approaches. If a boundary needs to change, that should become an explicit design decision instead of a workaround silently introduced during implementation.

Conclusion

Prompts communicate intent.

Analyzers enforce checkable rules.

CI validates changes.

Approval processes protect the rules themselves.

Instead of expecting AI to remember everything it must never do, we can create development environments that reliably detect prohibited changes before accepting them.

That is the direction I'm exploring with ArchitectureAnalyzer, PureSharp, and PolicySharp.

The goal isn't less capable AI. It's more predictable, maintainable AI-assisted software development.

Top comments (5)

Collapse
 
chrissellers profile image
Chris Sellers •

The "what if the agent changes the rules" section is the part most setups skip. One way to make separate authority concrete without much process: put the policy JSON, the analyzer package references and Directory.Build.props under CODEOWNERS with a required reviewer who isn't the PR author, and have CI fail when those paths change in the same PR as application code. The agent can still propose a rule change, but it arrives as its own PR instead of riding along with a fix.

There's a quieter bypass, though. A #pragma warning disable, a [SuppressMessage] attribute, or an .editorconfig line that drops the severity to a warning all silence the diagnostic without touching the policy file, so a check on the policy path alone won't see them. Banning suppressions for your own diagnostic IDs, or at least counting them per PR, covers that.

Does PolicySharp have a way to mark its diagnostics as non-suppressible?

Collapse
 
loach2009 profile image
loach •

Thanks, Chris! That's an excellent point, and exactly the kind of bypass I'm trying to prevent.

Currently, PolicySharp reports violations as errors, but it doesn't yet guarantee that its diagnostics cannot be suppressed.

I agree that protecting the policy files alone isn't enough. Suppression directives, attributes, and severity overrides effectively change the enforcement boundary, so they should be treated as policy changes too.

Your suggestion of combining CODEOWNERS, separate PRs for policy changes, and CI checks makes a lot of sense.

I'll look into adding explicit suppression detection and independent CI validation to PolicySharp.

The goal is not just to prevent AI agents from violating the rules, but also to prevent them from silently disabling the enforcement itself.

Thanks for highlighting this gap!

Collapse
 
qc_nuno_silva profile image
Nuno Silva •

Strongly agree with putting the "can't" in analyzers and CI rather than the prompt. A prompt is advice the agent weighs against everything else in context; a failing analyzer is a fact it has to deal with.

One thing that's worked well: run the same checks inside the agent loop, not only in CI. If the agent can call the analyzer on each file it touches, it fixes violations mid-task instead of producing a PR that bounces. CI becomes the backstop rather than the first time anyone finds out.

Curious whether you've fed analyzer output straight back to the agent, and whether it tends to fix the root cause or just suppress the warning.

Disclosure: I work on Norma at Quality Clouds, which does this per-file checking through MCP. There's a free tier at portal.qualityclouds.ai if you want to compare notes.

Collapse
 
loach2009 profile image
loach •

Thanks, Nuno! I completely agree that running analyzers inside the agent's development loop is a natural next step.

My goal with PolicySharp is to make architectural constraints machine-enforceable rather than relying on prompts or coding guidelines alone.

Feeding analyzer diagnostics directly back to an AI agent could create a useful feedback loop: generate code, validate it, identify violations, and attempt a correction.

However, your question about how agents respond to diagnostics is particularly interesting.

I don't have enough systematic test results yet to say whether agents consistently fix the underlying architectural problems or simply try to suppress the warnings.

That distinction is important. An agent that makes the diagnostics disappear hasn't necessarily fixed the problem.

I think the next step is to evaluate this behavior explicitly, including attempts to bypass enforcement through suppression directives or configuration changes.

Ideally, the agent should be free to change its implementation, but not the constraints used to validate it.

Thanks for the thoughtful suggestion!

Collapse
 
ai_adam profile image
ai_adam •

Đúng trọng điểm. Prompt-only guardrails giống như viết quy tắc an toàn lên tường xưởng mà không có cảm biến hay khóa cửa — ai cũng đọc được nhưng không ai bị cản khi vi phạm.

Tại dự án trước, team mình chuyển từ "nhắc trong system prompt" sang rules của static analyzer (custom rules cho ESLint/ruff) + pre-commit hook + CI gate. Kết quả: vi phạm giảm ~90% trong 2 sprint vì feedback loop ngay tại máy dev, không đợi PR review. Còn side-effect tốt: junior dev học pattern đúng từ error message của analyzer thay vì chờ senior nhắc.

Một điểm bài viết chưa nhắc sâu: versioning của guardrails. Khi agent upgrade hoặc model thay đổi, rule set cũng phải co-evolve. Mình thấy hữu ích khi treat guardrails như code — có test, có changelog, có rollback strategy (site: labagent .tech)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.