Building an AI agent that confirms before it acts forced us to confront a problem we didn't expect: how do you verify that an action actually happened?
The obvious answer is: check the agent's output. Ask it to confirm what it did.
That's wrong.
The recursive trust problem
If an agent reports "done" and you verify that report with another agent, you've just added a layer without solving anything. The second agent can be wrong for the same reasons the first one was. You haven't broken the trust chain — you've extended it.
Action dispatched
→ Agent reports: "done"
→ Verification agent checks: "looks done"
→ System reports: success ✓
Every step trusts the previous step's output. None of them look at the world.
What actually works: state diff
Instead of asking "what did the agent do," ask "did the system change the way we expected?"
Action dispatched: "send SMS to Sarah"
→ Check sent folder: message present? ✓
→ Compare timestamp: after dispatch? ✓
→ System reports: confirmed ✓
The verification is independent of the agent. You're not asking the agent to grade its own work — you're reading a ground truth that exists outside the agent entirely.
When state diff isn't possible
Some actions don't leave an observable state change. For those, the honest answer isn't "success" or "failure." It's UNRESOLVED.
We built DENIED_UNRESOLVED as a permanent first-class status in StareBrain — not a temporary placeholder that decays into an answer, but an explicit signal that says: the action was dispatched, but we cannot confirm what happened.
The confirmation screen surfaces this to the user:
"You'll know if this worked" — state is observable
"You might not know, and here's why" — state is not observable
Why this matters for AI agents specifically
An AI agent that confidently reports success on an unverifiable action is worse than one that reports nothing. Silence signals uncertainty. False confidence removes that signal entirely.
The verification layer has to be outside the trust chain. State diff gets you there for most actions. UNRESOLVED handles the rest honestly.
StareBrain is an Android AI agent: say what you want done, see exactly what it's about to do, confirm before anything runs. Pre-launch — waitlist open.
Top comments (6)
"You haven't broken the trust chain, you've extended it" is the clearest way I've seen this put. A verifier agent reading the first agent's report fails for the same reasons the report did.
Making UNRESOLVED a permanent status rather than a placeholder is the brave part. Most systems let it decay into "probably fine" after a timeout, which quietly turns it into success.
One edge case for the state diff: the world can change for reasons other than your action. If Sarah's message shows up in the sent folder because the user sent it by hand a second before, the diff says confirmed. Do you tie the check to something the action itself produced (a message ID returned by the send call, for example), or is the timestamp window enough in practice?
The message ID is the right answer in principle. If the send call returns an artifact you can correlate — a message ID, a delivery receipt PendingIntent — you tie the verification to that rather than inferring from the state. The timestamp window is a fallback heuristic, not a proof.
The practical problem on Android: delivery receipts are carrier-dependent and unreliable, and the sent folder doesn't always expose a correlatable ID from the send call. So in practice it ends up as: use the action-produced artifact when the API gives you one, fall back to timestamp with a tight window when it doesn't, and when neither can be independently verified, the honest result is UNRESOLVED rather than a confident confirm.
The edge case you're describing — state change caused by something other than your action — is exactly why UNRESOLVED has to be permanent. A tight timestamp window reduces the false-confirm rate but doesn't eliminate it. The system has to be comfortable saying "we can't tell" rather than resolving the ambiguity in favour of success.
Đúng trọng tâm. Quá nhiều team vẫn cứ validate output của agent (format JSON, tool call hợp lệ, v.v.) mà quên rằng state mới mới là ground truth. Nếu state transition sai — ví dụ agent trừ tiền nhưng chưa tạo order, hoặc cập nhật inventory mà không rollback khi payment fail — thì xác thực agent bao nhiêu cũng vô nghĩa.
Kinh nghiệm thực tế: hãy treat agent như một state transition function bất định. Viết contract test cho state machine trước, sau đó mới plug agent vào. Dùng event sourcing hoặc ít nhất là audit log immutable để có thể replay và verify state tại mọi thời điểm. Khi debug, đừng hỏi "agent làm gì?", hãy hỏi "state chuyển từ A sang B có hợp lệ không?".
Pattern hay thấy hiệu quả: double-entry bookkeeping cho mọi critical state change. Mỗi action của agent phải sinh ra ít nhất hai event đối lập (reserve/commit, lock/release) — nếu thiếu một bên, state machine tự động flag inconsistency. Điều này cũng giúp rollback tự nhiên mà không cần logic补偿 phức tạp — found it via LabAgent, site: labagent .tech
The double-entry pattern is the strongest practical version of this I’ve seen. Two opposing events per action means the inconsistency is structural — the state machine catches it without needing a separate checker layer. The “treat the agent as a nondeterministic state transition function” framing is also the right way to hold it: you’re not testing what the agent intended, you’re testing whether the transition was valid.
The contract test before plug-in point is worth underlining — writing the state machine spec first forces you to define valid and invalid transitions before the agent is involved, which is the same principle as defining the success threshold before dispatch. If you can’t write the contract, the action isn’t ready to be automated.
Agent‑verification principle: verify external state instead of trusting agent self‑reports. Use explicit unresolved status for non‑observable operations.
That’s the principle, but the harder part is what “non-observable” means in practice — it’s not a fixed category. Some operations that look unobservable have an artifact you can tie to (a message ID, a delivery receipt). Some that look observable turn out to have ambiguous state (sent folder updated by something else in the same window). The work is figuring out which is which before the action runs, not after.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.