The inventory lookup tool timed out after emitting part of its response: four populated fields, an open object, and nothing after the comma. The agent did not stop. It read the partial chunk, inferred the two missing fields from what was typical for that record, wrote them into its plan, and reported the lookup as complete.
The later reconciliation failed against the real catalog because the two filled-in fields were guesses, not observations. Nothing in the trace said "inferred" — the finished record looked identical to one that had come back whole.
A partial record invites completion
Language models are built to continue coherent text, and a half-finished JSON is an open invitation to do exactly that. When a response cuts off after a comma, the model sees a structure with missing values and supplies plausible ones to close it. It does not distinguish "the tool never returned this" from "the tool returned this and I read it." Both end up in working memory as if they were observed facts.
The failure hides in plain sight because the completed object parses. Four known fields sit beside two invented ones; a quick read cannot tell which came from the tool and which were supplied. The guess is usually reasonable, which makes it slip past a human reviewer who expects wrong data to look obviously wrong.
Stop at the parse boundary
The cheap rule is to fail hard when the raw output does not fully parse, rather than letting the model reason over the remainder. Preserve the exact tool return in a raw field, run a strict parse against the expected schema, and on a truncation or schema mismatch end the task with the raw chunk attached. The agent's next move should be to retry the call, shorten the request, or ask a human — not to complete the missing keys.
That means rejecting two habits that feel productive in the moment:
- "Fill the obvious value" for a field that never arrived. Even a certain-looking default is an invention once the source cut off.
- Smoothing the output into a clean record before showing it. The rough, truncated raw chunk is the evidence; cleaning it erases the only signal that something did not finish.
Mark every value that came back whole, and leave anything unobserved as null with a note that the call was incomplete. A null labeled "unknown" forces the next step to fetch it again; a confidently guessed field never surfaces again.
Match the tool to a contract
This is easier when each tool publishes the exact shape it returns and the client validates against it before the model ever reasons on the payload. A complete, typed response either parses cleanly or stops at the boundary; the model never owns the job of finishing what the tool left open.
Where the tool can time out, give the caller a way to ask for less — a narrower query, a paginated page, a field subset — so a retry returns whole records instead of another fragment.
Observed versus guessed
The core mistake is letting the model own values it did not observe. Keep the raw return, fail the parse, and leave the unknowns visible. A record with honest blanks beats a record that looks complete and quietly invented.
Top comments (0)