DEV Community

Cover image for Your AI Agent Shouldn't Be Allowed to Write Whatever It Wants
Ken W Alger
Ken W Alger

Posted on Originally published at kenwalger.com

Your AI Agent Shouldn't Be Allowed to Write Whatever It Wants

Focuses on admission control over storage in Go

Building a Write-Side Custody gate in Go

AI memory systems spend most of their design budget on retrieval. Which vector database? How should we chunk? Which embedding model? What should top_k be?

Those are useful questions, but they all arrive after something more consequential has already happened: the system decided that some piece of information deserved to become memory.

Consider an agent researching vendors for regulated workloads. It finds this statement:

Vendor X is approved for regulated workloads.

The source is Vendor X's own marketing site.

The statement might be true. It might even be current. But the source does not have the authority to establish organizational security policy. If our agent writes it directly into durable memory, better retrieval will not save us. We have only made questionable evidence easier to find.

The problem is not storage. It is admission.

I've been calling the architectural boundary responsible for that decision Write-Side Custody. Let's build a small one in Go.

What Write-Side Custody Does

A storage API answers a mechanical question:

Can I persist this object?

Write-Side Custody asks a different set:

  • Who is trying to write this?
  • Where did the information come from?
  • What authority is being claimed?
  • Is that source allowed to establish that authority?
  • Does policy permit this class of information to become durable?
  • What evidence should survive the decision?

Only after those are answered should storage become involved.

Flowchart showing a proposed write travelling from an agent or application into a Write-Side Custody gate. The gate routes accepted writes to durable memory and rejected writes to discard, while a dotted line records the decision in a Reasoning Ledger.

Note that the Reasoning Ledger does not make the decision. Custody enforces. The ledger witnesses. That separation matters a great deal once these systems have to be examined later.

Start With the Proposed Write

Go gives us a useful property for this experiment: we can make the things crossing our boundary explicit.

type ProposedWrite struct {
    Content          string
    Source           string
    ProducedBy       string
    ClaimedAuthority string
}
Enter fullscreen mode Exit fullscreen mode

Our research agent might produce:

write := ProposedWrite{
    Content:          "Vendor X is approved for regulated workloads.",
    Source:           "https://vendorx.example.com/why-vendorx",
    ProducedBy:       "research-agent-run-4471",
    ClaimedAuthority: "security-policy",
}
Enter fullscreen mode Exit fullscreen mode

Nothing in this structure says the statement is false, and that's intentional. Write-Side Custody is not a universal truth detector. It determines whether a proposed write satisfies the rules governing this particular memory system.

For this system, a vendor marketing page cannot establish internal security policy. So we need policy.

Make Authority Explicit

First, two types. Authorities and source classes are different kinds of thing, and there is no situation in which we want to accidentally use one where the other belongs:

type Authority string
type SourceType string
Enter fullscreen mode Exit fullscreen mode

Now we can define which source classes may establish which authorities:

type Policy struct {
    AuthoritySources map[Authority][]SourceType
}

var policy = Policy{
    AuthoritySources: map[Authority][]SourceType{
        "security-policy": {
            "internal-security-policy",
            "security-authority",
        },
        "user-preference": {
            "user",
        },
        "application-state": {
            "application",
            "runtime",
        },
    },
}
Enter fullscreen mode Exit fullscreen mode

In production this comes from a policy service or configuration layer rather than a Go literal. The important part is that the relationship exists independently of whatever the agent claims. The agent does not get to decide that a marketing page constitutes security authority simply because it found one saying something useful.

Give the Gate a Verdict

type Verdict string

const (
    Allow Verdict = "ALLOW"
    Deny  Verdict = "DENY"
)

type CustodyDecision struct {
    Verdict   Verdict
    Reason    string
    Timestamp time.Time
}
Enter fullscreen mode Exit fullscreen mode

Now the gate itself:

func EvaluateWrite(
    write ProposedWrite,
    sourceType SourceType,
    policy Policy,
) CustodyDecision {
    allowedSources, ok :=
        policy.AuthoritySources[Authority(write.ClaimedAuthority)]

    if !ok {
        return CustodyDecision{
            Verdict:   Deny,
            Reason:    "unknown claimed authority",
            Timestamp: time.Now().UTC(),
        }
    }

    for _, allowed := range allowedSources {
        if sourceType == allowed {
            return CustodyDecision{
                Verdict:   Allow,
                Reason:    "source may establish claimed authority",
                Timestamp: time.Now().UTC(),
            }
        }
    }

    return CustodyDecision{
        Verdict:   Deny,
        Reason:    "source cannot establish claimed authority",
        Timestamp: time.Now().UTC(),
    }
}
Enter fullscreen mode Exit fullscreen mode

Two decisions in there are worth surfacing.

The first is that conversion on the map lookup. ProposedWrite holds plain strings because that's what arrives over the wire, deserialized from JSON we did not write. Authority(write.ClaimedAuthority) is the moment an untrusted string becomes a term in our governance vocabulary, and it happens inside the gate rather than at the edge of the process. That's the right place for it. Custody is precisely the layer where foreign input earns domain meaning.

The second is that sourceType is a separate parameter. It is not a field on ProposedWrite.

That is deliberate. Source classification is a judgment about the write, not a property the writer gets to assert about itself. If sourceType lived on the struct, our agent could label its own marketing page internal-security-policy and the gate would cheerfully agree. The classifier belongs to the custody layer, or to a runtime component that can independently observe where the content came from.

Small signature choice. Most of the security property.

Our vendor claim now reaches the boundary:

decision := EvaluateWrite(write, "vendor-marketing", policy)

fmt.Println(decision.Verdict)
fmt.Println(decision.Reason)
Enter fullscreen mode Exit fullscreen mode

And receives:

DENY
source cannot establish claimed authority
Enter fullscreen mode Exit fullscreen mode

The statement never becomes durable memory. We did not store questionable evidence and hope retrieval would eventually sort things out. We governed the write while the evidence and its provenance were still in hand.

The full gate, the policy, and a table-driven test suite covering the cases above are in memory-stack-patterns. Standard library only, so go test ./... and go run ./cmd/demo work on a clean checkout with nothing to install.

Don't Throw Away the Rejection

Rejecting a write does not make the decision useless.

Imagine someone asks six months later:

Why doesn't the system remember that Vendor X was approved?

"I don't know" is not a satisfying answer, and in a regulated environment it isn't an acceptable one either.

The custody decision is observable system behavior, which makes it a candidate for a Reasoning Ledger record:

type LedgerEntry struct {
    ID               string
    Timestamp        time.Time
    Actor            string
    Action           string
    Verdict          Verdict
    Reason           string
    Source           string
    ClaimedAuthority string
}
Enter fullscreen mode Exit fullscreen mode

Our gate emits:

entry := LedgerEntry{
    ID:               newID(),
    Timestamp:        decision.Timestamp,
    Actor:            write.ProducedBy,
    Action:           "durable-memory-write",
    Verdict:          decision.Verdict,
    Reason:           decision.Reason,
    Source:           write.Source,
    ClaimedAuthority: write.ClaimedAuthority,
}
Enter fullscreen mode Exit fullscreen mode

(newID is a few lines over crypto/rand, which keeps the whole example dependency-free.)

This entry is deliberately simplified. A ledger you would actually rely on needs a canonical serialization, a hash chain linking each entry to its predecessor, and some defense against tail truncation, because an append-only log that anyone can quietly shorten is not append-only.

Even the timestamp is less innocent than it looks. time.Now() gives you whatever precision the host clock offers, and JSON drops trailing zeros, so two entries can serialize at different widths. Hash a chain over a non-deterministic encoding and you have hashed nothing. The repo linked above truncates to a fixed precision and formats with a fixed-width layout for exactly that reason.

The Python implementation in the Sovereign Systems SDK does all three, which is part of why the Go exercise interests me. The hard parts are already solved somewhere. The open question is what happens to the boundary when it moves.

What matters here is the shape of what survives. The rejected statement still doesn't enter memory. What persists is evidence that a write was proposed, evaluated, and rejected under a named rule. That is a different kind of information than the claim itself, and it's the kind that answers questions later.

The Agent Doesn't Get to Grade Its Own Homework

There's a further boundary hiding in the payload. Suppose our agent sends:

{
  "content": "Vendor X is approved for regulated workloads.",
  "source": "https://vendorx.example.com/why-vendorx",
  "claimed_authority": "security-policy",
  "retrieval_method": "fresh",
  "policy_verified": true
}

Should we believe the last two fields?

There is an epistemic difference between:

The agent says it performed a fresh retrieval.

and:

The runtime that performed the HTTP request witnessed a fresh retrieval.

The same distinction applies to tool execution, timestamps, approval events, and policy versions. A stronger custody boundary therefore doesn't only ask whether a record may be written. It asks:

Is this writer authorized to assert this particular kind of claim?

The agent legitimately owns claims about itself: its decision, the alternatives it considered, its confidence, the unknowns it identified. The runtime should mint the facts it can independently witness. Custody should not promote the former into the latter merely because both arrived in valid JSON.

This is the same principle as the sourceType parameter, applied one level up.

Why Do This at Write Time?

You could defer all of this to retrieval. Store everything, attach metadata, and let the reader decide what governs.

But then every questionable write becomes something every future reader has to reason around. It consumes storage. It becomes eligible for retrieval. It competes for context. It can be summarized, embedded, and propagated into records that no longer carry its provenance. And once provenance is gone, a future system may not have enough information to work out why the record was questionable in the first place.

A bad write today becomes bad context tomorrow.

Write-Side Custody puts the decision at the moment the system has the best possible view of what it is admitting.

Why Go?

None of this architecture requires Go, which is partly why I wanted to build it in Go.

A custody gate is a boundary service, and Go fits that role: explicit data structures, unremarkable HTTP services, small deployable binaries, and a type system strong enough to make the important distinctions visible without taking over the implementation.

The Authority and SourceType declarations we needed earlier are the clearest example. They cost one line each, and in exchange the compiler now refuses to let a source class be used where an authority belongs. That distinction would otherwise have lived in a variable name and a hope.

The same move applies elsewhere:

type Verdict string
type ActorType string
Enter fullscreen mode Exit fullscreen mode

At which point the function signatures start expressing the vocabulary of the governance system rather than just its plumbing. EvaluateWrite doesn't take three strings. It takes a proposed write, a source classification, and a policy, and no caller can shuffle them by accident.

Go didn't create the architecture. It made the contracts hard to leave implicit.

Memory Begins Before Storage

Vector databases are very good at answering questions about similarity. They cannot tell us whether something deserved to become memory.

That's an architecture decision, and by the time retrieval surfaces the problem, the questionable record may already have shaped dozens of others.

Give the proposed write provenance. Give the boundary policy. Give the decision evidence. Then let storage do what storage is good at.

Store what survived.


One thing I keep turning over: this boundary shouldn't depend on Go. If Write-Side Custody only makes sense inside one language, it isn't much of an architectural boundary. I'm curious what it would look like elsewhere. Would Rust's type system make an invalid custody decision impossible to construct rather than merely inconvenient? Would Pydantic and FastAPI make the policy check feel so natural you'd stop noticing you were doing governance at all? If you've built something like this in your stack, I'd like to hear how the boundary changed shape.

Disclosure: I maintain the Sovereign Systems specification and SDK, which is where the vocabulary in this post comes from. The Go code here is a reference implementation written to test whether the idea travels, not a product.

Top comments (13)

Collapse
 
joinwell52 profile image
joinwell52

The separate sourceType parameter is the strongest choice here. One failure I’d test next is provenance loss before the gate: a redirect, proxy, or retrieved snippet copied into a new object can make vendor marketing arrive looking “internal.” I would have the classifier return both the source type and a replayable evidence handle—URL, content hash, and collector—and make the gate deny when that evidence cannot be reproduced. Then the ledger records what the custody layer actually observed, not only the verdict.

Collapse
 
kenwalger profile image
Ken W Alger

Yes, that's a good failure case. Moving sourceType outside ProposedWrite prevents the writer from self-classifying, but it doesn't prove that provenance survived the path to the classifier. If vendor content gets copied through an internal proxy and arrives stripped of origin, SourceInternal could still be confidently wrong.

I like the evidence-handle idea, although I'd probably distinguish replayable from verifiable because some sources won't remain reproducible forever. Origin URI, content digest, collector identity, retrieval time, redirects/transformations, and perhaps the raw retrieval receipt could give custody something independently inspectable rather than another label to trust.

That suggests the stronger invariant is not merely "the agent doesn't supply sourceType." It's "source classification must be derived from witnessed provenance."

And agreed on the ledger: it should preserve what custody actually observed, including the evidence supporting the classification, rather than retroactively recording what we hoped happened upstream.

Collapse
 
joinwell52 profile image
joinwell52

Witnessed provenance is the better invariant. If an intermediate transformation cannot produce its own custody receipt, SourceInternal should not survive that hop; the result should fall back to unknown. I also agree that replayable and verifiable are different—the source may disappear, but the hash, collector, redirect chain, and timestamp should still let someone check what the classifier saw at the time.

Thread Thread
 
kenwalger profile image
Ken W Alger

Yes, I like the downgrade-to-unknown rule a lot. Provenance shouldn't be inherited merely because an upstream artifact once had it. If a transformation can't preserve or produce evidence for the classification, the downstream artifact shouldn't get to carry that classification forward by assumption.

That also gives unknown an important role: it isn't necessarily a failure state. It's the honest state when the custody chain can no longer support the stronger claim.

And your replayable/verifiable distinction is useful. A receipt may let us reconstruct exactly what the classifier observed at the time even when the original source is no longer available for independent re-evaluation. Those are different guarantees, and the receipt shouldn't imply the stronger one when it can only provide the former.

Collapse
 
mk023 profile image
Marco

This is a really interesting way to frame the problem. I especially like the distinction between storage and admission: once questionable information has entered durable memory, retrieval is already too late to fix the trust decision.

The part about the agent not being allowed to assert facts about the runtime also resonates strongly with how I think about AI systems: provenance shouldn't be something the model gets to declare about itself. The boundary has to come from something that can independently observe and enforce it.

I'm working through a very similar problem from the other side, around RAG pipelines, AI governance, and adversarial verification — particularly how to prove that a security invariant is actually enforced rather than just represented in prompts, rules, or tests.

I'll definitely take a closer look at the Go implementation. If you're interested in the other side of this problem, feel free to check out my work as well. 🔐

Collapse
 
kenwalger profile image
Ken W Alger

Exactly. "Represented as a rule" and "independently enforced as an invariant" are very different claims. That's part of why I wanted the example to be executable rather than leaving Write-Side Custody as an architectural diagram. I'll take a look at what you're doing on the adversarial-verification side too. Thanks!

Collapse
 
alexshev profile image
Alex Shev

The right default is capability scoping: give an agent the narrowest write permission that can complete the current task, then require explicit escalation for broader impact. That is more reliable than hoping a prompt remains cautious forever.

Collapse
 
vinhnguyenthanhdn profile image
Vinh Nguyen

The ledger also needs the identity of the policy that produced the verdict, not only the reason. Store an immutable policy version or hash plus the evaluated source classification; replaying an old write against the current policy answers a different question. A useful negative control is the same proposed write under two policy versions producing distinguishable ledger records even when both verdicts are DENY.

Collapse
 
kenwalger profile image
Ken W Alger

Agreed. reason explains the verdict but doesn't establish which governance state produced it. I'd want the durable evidence to include the policy identity/version or digest, the evaluated source classification, and ideally how that policy version was obtained.

Your negative control is particularly good. If policy v7 and v8 both produce DENY, storing only the verdict and reason can make the two events look equivalent even though they were governed by different evidence. The ledger needs to preserve that distinction.

And yes, replay needs two modes that shouldn't be confused: "reconstruct the decision under the policy that governed then" versus "evaluate the same proposed write under the policy that governs now." Those answer different questions. The first is forensic; the second is re-evaluation.

Collapse
 
izgorodin profile image
Edward Izgorodin

Ken, custody is the right place to put that boundary, and the invariant you arrived at with joinwell52 is stronger than the post itself: source classification has to be derived from witnessed provenance rather than asserted. What I would push on is the tense. Everything in the gate is witnessed at admission, and nothing re-witnesses afterwards.

A vendor certificate expires. A person leaves the role that made their write a policy. A policy version supersedes the one that produced an ALLOW. In each case the ledger entry stays exactly as true as it was, and the record it admitted stays exactly as authoritative as it was, because authority was checked once and never again. Custody that runs only at t0 is a bearer token with no expiry.

That gap is not yours alone, and I have a count for it. Six practitioners independently asked me for a required field naming who may revoke a promoted record, and two of those six said they do not model authority at all: they record who closed a decision, not whether that person was allowed to. So the rung nobody builds is the one going down.

The smaller point is the ledger entry itself. It records a verdict on one write and carries no pointer to what that write replaces. Supersession then has to be inferred later by comparing records against each other, which is the same postponement your post argues against on the admission side.

On the language question I have nothing useful. The invariant looks orthogonal to the runtime to me, but I have not built it in Rust and would rather say that than guess.

Collapse
 
kenwalger profile image
Ken W Alger

Yes, I think the tense is exactly where the architecture needs more precision. Admission can establish that a write was authorized and admissible at t0; it cannot establish that the authority supporting that write remains valid indefinitely.

I wouldn't necessarily make the write-side gate itself responsible for continuously re-witnessing every admitted record, but I do think custody has to preserve enough about the authority and dependencies behind the admission for later revalidation or revocation to be possible. Otherwise, as you say, we've converted “authorized when admitted” into “authorized forever.”

Your revocation-field observation is especially interesting. I'm thinking admission needs to capture not only who/what authorized an assertion, but the authority's lifecycle semantics: what can supersede it, what can revoke it, what dependency would make it stale, and perhaps whether it requires periodic or event-driven revalidation. Different authorities may have very different rules there.

And I agree on the supersession pointer. If the system already knows at admission that this write supersedes, corrects, or invalidates another record, postponing that relationship until retrieval throws away information we had at the strongest possible moment. The relationship should be admitted and custodied with the write rather than rediscovered later.

That connects to another discussion I've been having here: some relationships really won't be known until read time, but those should remain explicitly derived relationships. A witnessed supersession established at admission and a supersession inferred six months later by retrieval shouldn't become epistemically identical just because both eventually produce an edge called supersedes.

And agreed on the language point. I'm becoming more and more convinced the useful experiment isn't whether Go/Rust/Python changes the invariant, but whether the invariant survives implementation cleanly across runtimes.

Collapse
 
suraj09 profile image
Suraj Suradkar

The key issue is separating “can write” from “is authorized to write here.” Agents need scoped capabilities, not just approval prompts. Otherwise the guardrail is mostly trust.

Collapse
 
kenwalger profile image
Ken W Alger

Exactly. An approval prompt answers something like “may this action proceed?” but that isn't the same as establishing what authority the agent possesses in the first place.

I think the stronger model is scoped capability: this actor may perform this class of write, against this resource or namespace, under these constraints. The custody boundary then evaluates the proposed write against that authority rather than treating possession of a write mechanism as permission to use it everywhere.

That distinction becomes especially important as agents gain broader tool access. “Can invoke the write tool” should never silently collapse into “is authorized to write this state.”