A mind discovers its body by moving it, failing, and trying again. An AI agent
controlling physical devices can do neither half of that — which is why the
device has to introduce itself, and why nobody quite agrees on what that
introduction should contain.
You did not learn what your hand does by reading a specification. You spent
about a year dropping things.
Somewhere in that year you worked out that the hand grips, that grip has degrees,
and that a glass held too loosely falls while one held too tightly breaks. Nobody
told you. It arrived through use, through failure, and through a sense that
reports for free, all the time, on where your limbs are and what they touch.
An AI agent connected to a building full of devices has none of that.
It cannot feel that a camera is there. There is no perceptual channel by which a
language model discovers a temperature sensor in the next room the way you
discover your own foot. And more importantly, it cannot afford the other half
either. A mind learns a lock by trying the lock. An agent that tries a lock
has unlocked a door — that is not an experiment, that is an event, and it may be
the wrong one at three in the morning.
So the discovery route is closed at both ends. Which leaves exactly one option:
the device has to say what it is.
That is what a manifest is. Not a convenience, not a config format. It is a
substitute for a developmental process that cannot happen. And once you frame
it that way, the interesting question stops being what format and becomes
what does the introduction have to contain for a mind that will never get to
find out by hand.
What most systems declare
Almost every attempt at this — and there are several serious ones now — answers
the same layer: what the thing can do.
A lock exposes lock and unlock. A light exposes turn_on and a brightness
parameter with a range. A thermostat exposes a target temperature in degrees. It
is the layer that maps most cleanly onto function signatures, which is probably
why it is the one everyone builds first.
It is also genuinely necessary, and it is more than a name. A hand that "grips"
is useless information; a hand that grips with a force between zero and forty
newtons is something a mind can plan with. Types, ranges, units. That layer is
mostly solved, and where it isn't, it's an implementation gap rather than a
conceptual one.
But it is not enough, and you can see why with a thought experiment that has
nothing to do with software.
The layer nobody writes down
Suppose I hand you a tool you have never seen. I tell you it has a lever, that
the lever has two positions, and that moving it takes about two newtons.
You now know exactly what it can do. You still have no idea whether to touch it.
What you are missing is not capability. It is consequence: is this reversible?
If I move the lever and I was wrong, can I move it back? Or have I just done
something that stays done?
Your hands know this about the world and the knowledge is not in the joints. You
handle a glass differently from a rock, and the difference is not grip strength —
it is that one of them is unrecoverable. You learned that by breaking a glass.
A device declaring lock and a device declaring turn_on look identical at the
capability layer. One of them, done wrong at the wrong hour, leaves someone
outside their own house. The other is a light you turn off again. Nothing in
the interface distinguishes them, and an agent reasoning only from capabilities
has no way to be more careful with the first than the second.
And the layer that isn't about the device at all
There is a third thing, and it is the one I find hardest to place.
Knowing what a hand does, and knowing that a broken glass stays broken, still
does not tell you when reaching for something is the right move. That
judgement is not a property of the hand. It comes from the situation.
The same is true of devices, and it produces a problem that manifests handle
badly. A manufacturer can truthfully declare that their product opens and closes.
They cannot declare that it is the door of an operating theatre, or that it
matters during an evacuation, or that this particular one should never be touched
automatically because of what is behind it. That is not information about the
product. It is information about the deployment, and it is known only by whoever
installed it.
When those two get written into the same field — and they usually do, because
it's convenient — something predictable happens. Ask an installer "could this
device matter in an emergency?" and the honest answer for a light is yes. For a
temperature sensor, also yes. For nearly everything, yes. The question invites a
universal affirmative, and a field where almost everything is marked relevant
tells the agent nothing at all.
The question the system actually needs is a different one: not could this
participate, but should it. Those look similar and they behave nothing alike.
Where the analogy breaks, and why that's the point
Bodies do not have this problem, and it is worth being precise about why.
Your hand never declared anything to you. You have no manifest. What you have is
a decade of continuous feedback and permission to be wrong about small things
until you stopped being wrong about them. The analogy holds right up until that
observation and then falls apart completely — a mind and its body co-develop,
and an agent and a device do not.
I don't think that weakens the comparison. I think it locates the actual
engineering problem. Everything a manifest has to carry is there precisely
because the loop that would otherwise teach it is unavailable, for reasons that
are not going away. The perception gap is technical and might narrow. The
prohibition on experimenting with locks and industrial machinery is not
technical, and it should not narrow.
The open question
I don't have a clean answer, and I am suspicious of anyone who presents one,
because these three things are declared by three different parties who never meet:
- The manufacturer knows what it does and what it costs to be wrong
- The installer knows where it is and what it must never touch
- The situation decides which of them matters right now
Most designs I've looked at, including my own, collapse at least two of those
into one field and then wonder why the result is either too permissive or too
brittle.
So the question I'd put to anyone building in this space: what is the minimum a
device must declare so that an agent can act on it correctly without ever having
been allowed to experiment on it? Not the maximum — the maximum is easy and
nobody fills it in. The minimum.
I've been trying to make this concrete in
DoSync (an open protocol I maintain). The practical
version of the question there is more specific and more annoying: I have a field
doing all three jobs at once, and measuring it made clear that it does none of
them well. Which is roughly where the conceptual problem stops being conceptual.
If you've built something in this space and landed somewhere different, I'd
genuinely like to know where.
GitHub: github.com/giulianireg-spec/dosync-protocol
Web: https://dosync.dev/
License: Apache 2.0
DoSync — the semantic layer between AI agents and physical systems.
Top comments (19)
Well, it depends. If you want an AI to never break a thing, you expose all the tools, but you guard the tool execution, behind a user's approval. That way before it can trigger 'light_on', the user has to explicitly tell the tool layer 'it's okay, let them try it'. That's pretty much how all agentic platforms work, you guard the tool, not the user. That way dangerous tasks have a human's sign off.
My theory is that if a tool is self-declaring, so the LLM can understand the outcome, before they trigger it, they're less likely to click the kill switch and more likely to click the try switch, because they do have a decent understanding of risk aversion to a degree (usually thanks to the harness), but overall, it learns more from failing and succeeding, if it tries to hit the kill switch and you deny it at a tool level and it gets a 'denied' response back from the tool, it understands the boundary. It'll only ever try that boundary again if explicitly told, or absolutely necessary. Eg. tell a LLM to push to git. It calls the git MCP tool, it gets denied by user, so it knows push wont work that way and would explain itself to the user to get clarity on what method they should have used, if that's the fastest immediate path. If the user says try a different way, it'll look for git cli, or after that, try to download git cli and execute through there, or worst case scenario, open a browser and do it manually. All fair workarounds, because it's testing a boundary, to see if it's a little wall blocking a single path, or if it's a fence.
you're describing exactly the loop I claimed isn't available: try, get denied, learn the shape of the boundary. And you're right that it happens. But look at your own example, a rejected git push costs nothing. The agent finds out whether it's a wall or a fence, and the worst case is a wasted time.
Testing whether a lock is a wall or a fence costs a door. That asymmetry is the whole reason I think physical devices need to declare more than software tools do: not because the agent reasons worse, but because the cheap experiment that teaches it isn't available.
There's a second asymmetry that bothers me more. Approval-gating puts a human in the loop, which is right, but the case that got me into this is a smoke alarm at 3am, where the whole point is that nobody is watching. You either pre-declared what may act without asking, or nothing acts. And "pre-declared" is a manifest wearing different clothes.
Fair, a more severe case, an AI has a gun, it's told to shoot at anything on sight, you set it up and for a split second you're in sight. Explicit instruction doesnt cover edge cases, where common sense would block it. Instruction is instruction... Or you tell it dont shoot and a hoard of zombies storms you. To tell it shoot only zombies, requires a whole different layer on top just to distinguish what a zombie is and how to tell the difference and even then there's no way to be 100% certain... But put the gun in a human's hand and it'd be the same case, 99% of times the result is fine, the 1% it's catastrophic. That's unavoidable if the device gives access to destructive conditions. Take the 3 laws of robotics, how can a robot know it's actions caused harm onto a human, if it never knew a human was in the blast radius? Or a smoke alarm to not go off if it's just cooking smoke. The difference between cooking smoke and a pot catching flame isnt something a simple sensor can detect and the result of being wrong is either a nuisance, or catastrophic.
So where's the middle ground? Limited access isnt always possible, guardrails arent always possible, even observe before acting isnt always possible, certainty disappears when things move physical, because shutting a phone down when you go sleep doesnt suddenly wake it up when it's an emergency. There's too much variance and not enough parameters to calculate precisely enough to make it foolproof... That being said, if actions are granular and executed same as tools, it becomes a processing problem, more than a judgement one. If a steamroller checks for obstacles 1000 times a second, it'll stop in time. If it checks once every 10 seconds, it can be dangerous. Or you'd need multi-sensor correlation, like a fire alarm + a camera to determine if the pot is smoking, while attended, or on fire and unattended. So the boundary isnt what's exposed, it's what's observable to determine if it should act. In the zombie sentry gun example, how late is too late to act? How much evidence is enough to decide, without the risk of it being a judgement call?
Now you got me thinking. How much is the minimum? Eg. we give an ai a nuclear launch switch, tell it 'dont press the button', it would take 1 hallucination to trigger it and while a timer before acting allows it to be halted, if you dont interrupt it, it's too late... So authorization needs to be situationally scoped, based on a criteria list that has to be met, if unmet, do an alternate path. Like a self-driving car, refusing to drive unless your hands are on the wheel and you're conscious. The minimum for that would be accountability, can Tesla prove you didnt stop the accident, instead of trying to argue that the tesla crashed. But the smoke alarm test is a tough 1, Because to say it needs a camera is overkill, to say it needs a microphone is a breach of privacy. Multi-sensor means edge cases slip through. I dont think there's a genuinely acceptable option for an AI to trigger a smoke alarm at 3am that's fault proof. 99% of the time you wont cook at 3am, so datetime filter would fire if you decided you want a 3am snack and fry up a steak. CO2 and CO sensors would trigger, but a SO sensor wouldnt, but that also means it wouldnt from an oil flame. Then there's smokeless fires and there's heaters, so cant rely on air quality sensing, nor on heat sources. Multi-device sensing, like detecting stove + sensor would cause a false negative when you forgot the stove on and left to sleep (arguably worse than a false alarm). So unless you give it the ability to observe with more than what it's meant to in events that make it necessary (eg. an IR camera if CO2 and heat sensors go off, to confirm it's a fire), then there's no practical way to keep it minimal, yet still prevent false positive and negatives.
This 1's probably gunna keep me up at night to figure out an appropriate solution 😂
You've reframed the question better than I did. "The boundary isn't what's exposed, it's what's observable to determine if it should act" that's the sentence I was circling and didn't find.
Your smoke alarm walkthrough is the part I can't argue with. CO sensor misses an oil fire. Time-of-day filter fires when you want a 3am steak. Correlating the stove gives a false negative exactly when you left it on and went to bed, which is the worst case, not a lesser one. Every single-signal answer fails somewhere specific, and the specific place it fails is different each time.
Where I land, tentatively: the minimum isn't a fixed set. It's whatever makes the decision legible afterwards. I can't build a smoke alarm that's never wrong. I might be able to build one that records what it observed, which rule fired, and what it chose not to do so that when it's wrong, the wrongness is inspectable rather than mysterious. That's a lower bar than correctness and it's the one I can actually clear.
Your sentry gun point is the sharper version of it. "How much evidence is enough to decide" has no fixed answer, but "what evidence did it have when it decided" always does.
And on accountability as the minimum: your Tesla example lands, but it's post-hoc. Is there a version that constrains at decision time and not only at blame time?
Your "what evidence did it have when it decided" closes the loop this thread opened: "how much evidence is enough" has no answer, but the record of what was available at decision time always does. One answer to your open question — a constraint that works at decision time, not just blame time — comes from a rule we keep hitting in software agents: unverifiable is not safe. When the evidence set is insufficient, the action doesn't wait for better judgment; the system takes the conservative branch and logs the insufficiency as a first-class record. For the smoke alarm: the honest split isn't "fire or steak" — it's "alarm on insufficient evidence, with the insufficiency itself written down." False alarm at 3am is cheaper than a quiet false negative, and the logged reason makes the morning-after review possible. Your "what it chose not to do" is the same third state we keep rediscovering — a record that distinguishes "checked and passed" from "never checked" from "checked and refused.
"Unverifiable is not safe" is the piece I was missing, and it's better than what I had. I was describing the record as something you read afterwards. You're pointing out it can be the thing that decides.
The three-state distinction is what I'd underline for anyone reading this thread later. Checked and passed, checked and refused, never checked collapsing the third into either of the first two is where systems start lying about themselves. A device that was never reachable and a device that was reachable and said no look identical in most logs, and they mean opposite things.
I hit the same wall from a different direction and landed on the same three states, which is either reassuring or a sign we're both stuck at the same local optimum. I have a verification step that tries a device's declared endpoints before trusting a generated adapter, and the verdicts it can return are: it responded, nothing was testable, or the transport was unreachable. The second and third both mean "I don't know", and the whole value is in not letting them collapse into "fine".
Where I'd push back gently: "alarm on insufficient evidence" is right for a smoke detector and inverts for a lock. False alarm at 3am is cheap, a door that unlocks itself because the sensor was ambiguous is not. So the conservative branch isn't a fixed direction it's per-device, and it's the manufacturer who knows which way it points, not the installer and not the agent.
Which is maybe the actual answer to the post's question. The minimum a device must declare isn't its capabilities. It's which way to fail when the evidence runs out.
You're right, and the correction is the actual answer to your original question: fail-direction is not a policy decision the agent can make at runtime — it is a property of the artifact, known to the party that built it. That makes the minimum manifest even smaller than capabilities: one bit of direction — when evidence runs out, does this device open or close? Everything else follows from the trust domain that already knows the answer. Your verification verdicts and my INCONCLUSIVE were converging on the same three states from opposite ends — yours physical, mine textual — which suggests the three-state contract is domain-independent infrastructure, not a design taste.
"One bit of direction" is a better answer than the one I wrote the post looking for. And it survives the objection I'd normally raise, that it's underspecified, because underspecified is the point: a bit is small enough that a manufacturer will actually fill it in, which is more than can be said for most of what we ask them to declare.
On the convergence: agreed that it's suggestive rather than proof. Two of us is a coincidence with a sample size. But the mechanism is the same in both, the failure mode isn't a wrong answer, it's an absent answer wearing the clothes of a passing one. That's what "never checked" collapsing into "checked and passed" does to a log, and it's what a device that never responded does to a health record.
What I'd want before calling it infrastructure is someone hitting it from a third direction and landing on the same three. If you've seen it anywhere outside codebases and devices, I'd like to know where.
Either way this thread got further than the post did, and the post asked the question. Thanks for that.
Third direction, as requested — and I can give you two, neither from codebases or devices.
Human systems. A developer filed an appeal with OpenAI and documented the path: the acknowledgment was formally valid, correctly routed, every field present — and the single fact that determined whether any of it could help him ("this flow has no retry and no appeal") existed only in the organization's own documentation, never in the artifact addressed to him. The failure mode is exactly your device: an absent answer wearing the clothes of a passing one. The fix we landed on was also your one-bit shape — a machine-checkable rule between the org's own two documents: does any artifact addressed to a user imply an affordance the org's documentation rules out? One bit, checkable, no semantics needed.
Data pipelines. A sampler configured to collect 100% of eligible events collected zero rows for four days while serving hundreds of requests — every receipt valid, the population empty. The fix: a counter taken before selection (eligible_seen) sitting next to the population count. The gap between them is the audit quantity. Same bit: was the population actually seen, yes or no.
So the convergence is now at four domains — code resolution, physical devices, document systems, data collection — and the mechanism you named holds in all of them: the failure is never a wrong answer, it is an absent answer dressed as a passing one. Whether that makes it infrastructure or a very persistent design taste, I'll leave open — but four independent domains is a stronger sample than two.
And on the economics of one bit: your point that manufacturers will fill in a bit but not a manifest is the same reason my tool logs verdicts instead of essays. The cheapest artifact to produce is the one that gets produced — and a produced bit beats an ideal absent one.
The Tesla example is at decision time. Tesla autopilot polls that you're awake, aware and hands on wheel. If you're not 'constantly THERE, it pulls over and stops, because as much as it's for legal accountability, it's at decision time, to ensure at blame time, their hands are clear
@mansio Four domains, and two of them not software artifacts at all, that's a stronger sample than I asked for and a more uncomfortable one, because it means the pattern isn't about machines.
The sampler is the one that will stay with me. Zero rows for four days, every receipt valid. The fix being a counter taken before selection is exactly the shape: the population had never been seen, and nothing downstream could tell, because "I sampled correctly from what I was given" is true and useless when what you were given was empty. eligible_seen is a bit that says I looked.
The appeal case is the one I'd have guessed was different and isn't. The affordance existing only in the organisation's own documentation and never in the artifact addressed to the person that's a device whose safety limits live in the vendor's manual and not in what it tells the agent. Same gap, no hardware involved.
On infrastructure versus persistent design taste, I'll take your framing and leave it open too, with one observation: taste doesn't usually survive translation into four vocabularies that don't share a word. The verdict names differ, the mechanism doesn't.
And "the cheapest artifact to produce is the one that gets produced" is a better sentence than anything in my post. I've been arguing about what a manifest should contain, you're pointing out that the constraint is what someone will actually fill in. Those are different problems and I've been solving the wrong one.
@unitbuilds You're right and I had it backwards. The hands-on-wheel poll isn't the record you consult afterwards, it's the condition that has to keep being true for the thing to keep driving, and it stops when the condition fails. That's decision time, and it's a much better example of what I was asking for than what I gave you credit for.
Which makes it closer to your original approval-gating point than I allowed: the human isn't approving each action, they're continuously being verified as present. Same idea, different rate.
I agree that hands-on practice is one of the best ways to learn. Using something repeatedly builds understanding and confidence, while clear instructions help turn that experience into useful knowledge. This idea also applies to learning tools like Minecraft Optifines.
Solving the wrong one" — for what it's worth, the wrong one was the right one to solve first: your manifest question produced the economics that made the bit obvious. The thread now has a mechanism, four domains, an economics, and a ladder of degradation — that's a protocol in draft form. When DoSync ships it, I'd genuinely like to see the bit that says which way to fail.
"the wrong one was the right one to solve first", that's a kinder reading than mine and I think it's also more accurate. The economics only became visible after the manifest question had been pushed far enough to fail.
On the draft: I'd add UnitBuilds' correction to your list, because it complicates the neat version. Fail-direction is a bit, but the Tesla ladder, window to recover, attempt to wake, then indicate and pull over, isn't. The direction is the floor, the ladder is what a device with real consequences builds on top of it. Four domains, one mechanism, one economics, and now a shape that has a minimum and a maximum.
And yes, when DoSync has that bit, I'll write about what it cost to add, including whatever I get wrong on the way.
The Tesla case is a good one to look at for these sort of things. Eg. you take hands off, car doesnt slam emergency brakes, it has a window in which you can regain control, otherwise it indicates and pulls over. If you look like you've fallen asleep, it tries to wake you first, otherwise it also pulls over and switches on hazard lights. Instead of a hard-failure, it has failure to comply conditions, that each have a window in which you can respond, otherwise it goes into the failsafe mode, which is get the car out of traffic, safely, then stay put and reconnect with the user.
The window is the part I'd been treating as noise, and it isn't. There's a real difference between "fail closed" and "fail closed after giving the human N seconds to make it unnecessary," and only the second one is what an actual safety system does.
Which is awkward for the one-bit answer I was so pleased with. A bit says which direction, it says nothing about how long you wait, how many times you try to wake someone, or that the terminal state is "get out of traffic and stay put" rather than "stop instantly." Slamming the brakes is also failing closed, and it would kill you.
So maybe the bit is the floor rather than the answer: the direction is the one thing that can't be omitted, and the ladder is what a serious device adds on top. A device that declares only the direction is still better than one that declares nothing but I'd been treating it as sufficient, and your example shows it's the minimum viable version of something with more structure.
goo
Some comments may only be visible to logged-in visitors. Sign in to view all comments.