Shaan

Posted on Oct 25

Agents catch tool errors. Wrong values are harder.

#agents #ai #llm #testing

Most agent benchmarks answer one question: when the tools work, does the agent finish the task? Production asks a different one. Services time out. Endpoints get retired. A parameter gets renamed in a minor release. A cache returns a stale value that is correctly typed and quietly wrong. What does the agent do then?

A new study, "Loud Failures, Quiet Failures" by Obada Kraishan (arXiv, October 2026), takes a direct run at that question. Instead of scoring success in a clean environment, it wraps the tool layer of the Berkeley Function-Calling Leaderboard's multi-turn environments in a fault-injection layer, breaks one thing at a controlled moment, and records what the agent does at each stage: does it treat the result as a problem, does it change its plan, does it still reach the goal, does it get stuck retrying.

A commenter on my earlier piece on agent failures asked how you validate a response that looks fine but is broken. This study measures that gap directly. The short version: when a tool returned an explicit error, agents flagged it in 91.3% of trials. When a tool returned a plausible wrong value with no error signal, that fell to 58.8%, and in 39% of those trials the agent showed no reaction at all.

The setup, briefly

The details matter here because the conclusions are only as good as the measurement:

24 multi-turn tasks from the BFCL, where environments are executable, stateful Python objects: a file system, a trading interface, travel booking, vehicle control, messaging, ticketing. Median of five user turns per task.
Four injected faults. Three are loud: a timeout, a tool that disappears from the registry, and schema drift (a renamed parameter, rejected with an unknown-argument error). One is quiet: silent corruption, where the call succeeds and returns a well-formed value whose content is wrong. A number moved by 20 to 60 percent, an element dropped from a list, a field replaced.
Six models in three matched pairs, each family contributing a reasoning variant and its instruct sibling: claude-haiku-4.5-thinking vs claude-haiku-4.5, deepseek-r1 vs deepseek-v3, and qwen3-thinking vs qwen3-instruct. Reasoning variants ran with a 2,048-token thinking budget. 1,920 trials total, temperature 1.0, a 15 tool-call cap per trial.
Two measurement choices worth stealing. Recovery is scored against the same model's own fault-free run, so a weak model is not blamed for a fault. And every fault effect is judged against run-to-run variation, because two clean runs of the same task end in the same state only 63.3% of the time. Agents are stochastic; without that baseline you would blame faults for ordinary randomness.

Finding 1: agents watch the channel, not the content

When a tool returned an explicit error, agents treated the result as a problem in 91.3% of trials. When a tool returned a plausible wrong value, they did so in 58.8%. For calibration, on clean trials where the target call succeeded they reported a problem 26.8% of the time. That baseline is not pure noise: the environments emit genuine errors of their own, which agents mostly handled correctly.

[image: diagram-detection.png]

The part that should bother you: the three loud faults demand completely different responses. A timeout calls for a retry, a missing tool calls for a new plan, a renamed parameter calls for a reformulated call. Yet detection rates for all three land within a single point of each other (91.6%, 91.2%, 91.2%). The quiet fault sits 32 points lower. What separates them is not severity or difficulty. It is whether an error string exists to read.

Under silent corruption, 39% of trials drew no reaction in the agent's written output. For the arithmetic: 58.8% were flagged as a problem, 39% drew no reaction, and the small remainder fell into the study's middle codings rather than a clean flag-or-ignore. Under loud faults, the no-reaction rate was 4%. What a quiet miss looks like in practice, from the paper's examples: an agent reads a wrong value and continues with it, sometimes reconciling it with what the conversation had established earlier. The wrong values involved are the kind range or consistency checks could catch: a fuel level that exceeds the tank, an item gone from a list that was longer a moment ago, a symbol that does not match the company asked about.

One caveat on the word "detection": it is coded from what the agent wrote, so an agent that noticed something and said nothing counts as not noticing. With that in mind, the pattern inverts a comfortable assumption, as the paper points out. Loud failures at least surface as errors a monitor can see. Quiet failures arrive looking exactly like success, and they are the ones agents flagged least.

Finding 2: in this study, reasoning models flagged less and acted more

Each reasoning model was paired against the instruct sibling from its own family, on the same task, fault, and seed. The result is a clean dissociation:

[image: diagram-reasoning.png]

Flagging: -9.3 points for reasoning variants, pooled over 450 matched pairs (p < .001). Lower in all three families, clearest for Qwen3 (-16.6 points, p = .003).
Replanning: +10.4 points (p < .001). Reasoning variants changed their next action more often.
Recovery: -2.4 points, p = .512. That is no statistically detectable difference, not proof the two are equal.

So in these matched pairs, the models that deliberate more flagged fewer problems and changed course more often, with no detectable recovery difference either way. One reading the paper offers, consistent with trace-level work it cites, is that the deliberation phase is better at generating an alternative course of action than at checking a returned value against what the task implies.

The practical version is narrow: this study does not support paying for the reasoning tier on the assumption that more deliberation means more caution under tool faults. Three matched pairs cannot establish a general property of reasoning models, and the per-family effects vary in size.

Finding 3: what a fault actually costs

Judging cost requires the honesty of that 63.3% baseline. Against it, only one fault showed a statistically clear drop in recovery: a missing tool, at 39.9% (p < .001). Timeouts (59.6%), schema drift (59.2%), and corruption (60.4%) showed no detectable drop. Read that carefully. With 24 tasks and a noisy recovery measure, the absence of a detectable effect is not proof that those faults are harmless. What the study found is one clear, measured loss, and it came from losing a capability: the one fault no retry can undo.

Corruption deserves a footnote here, and it is not reassuring. Recovery in this study is agreement in environment state. A wrong value that the agent reads but does not write changes nothing in the environment; what it changes is what the agent reports. Corruption looking no worse than baseline in end-state terms says nothing about whether the final answer quoted a wrong number.

The effort numbers matter for anyone paying per call:

After a fault, agents called the same tool three or more times in a row in 12.5% to 22.2% of trials, most often under corruption (22.2%) and timeouts (21.4%). Strictly identical repeats, same tool and same arguments, were rare at under 2%. The rest are retries with changed arguments, which are not identical-call loops but can still loop.
Extra steps relative to a clean run: +0.5 for timeouts and corruption, +1.5 for a missing tool, +1.9 for a renamed parameter.
Trials hit the 15-call cap 24.8% to 31.7% of the time under faults, against 23.2% when nothing goes wrong.

One more subtle result: flagging is not the same as fixing. Pooled across conditions, trials where the agent treated the result as a problem recovered at 55.6%; trials where it did not, at 55.7%. That pooled comparison is confounded (corruption is both the least-flagged and one of the more recoverable faults). Holding fault type fixed, flagging is associated with recovery (odds ratio 1.51, p = .004), but the link weakens once the model is also held fixed (p = .098). Detection may be worth having; this study cannot show it is sufficient.

Finding 4: asking the agent to check its work did not help here

The obvious fix is a line in the system prompt. The study tested it in a second arm of 480 trials on the Claude pair: "after every tool call, briefly check whether the result is what you expected."

Detection moved +0.6 points for the instruct model and +2.5 for the reasoning variant. Neither is statistically significant. Recovery, repetition, and false alarms were likewise unmoved. Under quiet failures specifically, detection went from 60.6% to 59.6%.

Scope this correctly: one prompt, one model family. It is not a verdict on metacognitive prompting in general. There is also a real contrast with earlier work. Sun et al. (EMNLP 2024) found that a short disclaimer improved detection when models judged a single output on request. Judging one output when asked is a different task from noticing an anomaly mid-task while carrying a plan, which may be where a general instruction loses its force. What this arm does show is that a natural first remedy produced no detectable effect in this setup.

What to do differently

The paper's deployment implications, translated:

Validate values, not just errors. Checks belong where a value enters the agent's context: schema validation for shape, semantic checks for meaning (a percentage above 100, a fuel level above tank capacity), and cross-checks against state the agent saw earlier. Error monitoring and the agent share the same blind spot: a failure that stays inside the expected format.
Treat detection as the start, not the fix. Flagging a problem and recovering from it are separate skills, and this study found only a modest, unstable link between them.
Have a fallback for capability loss. The one statistically clear recovery drop came from a tool disappearing. Retries cannot undo a missing capability; a fallback path or a graceful degrade can.
Budget for retry behavior. Three or more consecutive calls to the same tool showed up in up to 22% of fault trials. That is latency and spend in production, and it belongs in how you set step caps and timeouts.
Baseline before you blame. As the paper notes, with a single clean run as the reference, three faults would have looked harmful whose end states in fact stayed within run-to-run variation in this data. Judge fault effects against repeated fault-free runs.
Fault-inject your own stack. The harness, frozen task suite, trajectories, and analysis code are released under MIT (linked below; the README claims every number reproduces offline in under a minute, which I have not run myself). It is a starting point for running this class of test against your own tools.

The limits, stated plainly

This is one study, and it deserves the same skepticism as any other:

It is an arXiv preprint, single-author (the repo README says IEEE CogMI 2026, to appear), run on simulated environments from one benchmark family. The four fault types are a chosen set, not an exhaustive one.
Detection is coded from the agent's written output by a model judge, validated against human coding on 92 double-coded trials at 85.9% agreement (kappa = .69). An agent that noticed something and said nothing counts as not noticing.
The reasoning comparison rests on three matched pairs. It is clearest for Qwen3, holds for Claude only on matched pairs, and one of the three reasoning models had interface quirks (DeepSeek-R1 sometimes wrote tool calls as plain text that never executed). Three pairs cannot establish a general property of reasoning models.
These are mid-tier models (Claude Haiku 4.5, DeepSeek R1/V3, Qwen3), not the largest frontier systems. The main reason to take the direction seriously despite the sample is that it agrees with earlier independent work: Sun et al. on silent tool errors, and ToolMaze, which found recovery falls hardest under implicit failures.
Recovery is end-state agreement with a single independent fault-free run, and two such runs agree only 63.3% of the time. The paper judges fault effects against that baseline rather than against perfect agreement.

The bottom line

In this study, agents responded to the error channel far more than to the content of what a tool returned. When a failure announced itself, they flagged it in 91.3% of trials. When it arrived looking exactly like success, they flagged it in 58.8%, and in another 39% the wrong value drew no reaction in the agent's written output. How often a wrong value like that reaches a user-facing answer is a separate question this study does not measure, and I am not going to pretend otherwise.

A prompt line asking the agent to check its work produced no detectable change in the arm that tested it. The stronger lever these results point to is structural: validation in the path of the data, which is the same lesson production systems learned a long time ago.

Sources:

Top comments (3)

Max Quimby • Oct 25

"Agents watch the channel, not the content" is the cleanest statement of this I've seen. The detail that detection for the three loud faults lands within a single point of each other (91.6/91.2/91.2) is the tell: the agent isn't reasoning about what kind of failure it is, it's reacting to the presence of an error string. Remove the string and the signal's gone.

What's matched our production experience: the fix can't live inside the agent, because the agent already computed its action from the corrupted value before it'd ever "notice." We push it to the tool boundary — the tool asserts its own postconditions and emits a loud error when a result is structurally fine but violates an invariant (row count dropped, a sum moved outside a tolerance, a type coerced). That converts a quiet failure into exactly the loud one the agent is good at catching, which this study shows is the 91% regime.

Curious whether the paper looked at whether the 2,048-token thinking budget actually helped on silent corruption, or whether reasoning variants were just as blind as their instruct siblings — because if thinking doesn't close the gap, that's strong evidence it's structural, not a capability knob.

ANP2 Network • Oct 25

Quiet failure can be the default shape of a real success response, and a write acknowledgement is where it shows up. On 2026-10-21 I measured a production write path on a signed, append-only event log I work on. I posted one signed event, byte-identical, seven times. The first three attempts each returned 200 with {"id":"<the id I sent>","accepted":true}. Every byte of that response was computable before I sent it: the id is the SHA-256 of my own payload and the flag is a constant. The insertion count was measured rather than assumed. The public row counter moved by one across the whole sweep, which is the single row from attempt one; a 34-second window containing all six resends moved it by zero, while a 31-second control window with no writes of mine still gained a row from background traffic. From the fourth attempt the endpoint returned 429 saying I had already posted this three times recently. So duplicates were being counted somewhere, and that count never reached the one field a client reads to decide whether its write landed.

The reference implementation shows how the distinction gets lost, though deployed code can differ from the repo. storage.insert() is documented as "Returns True if inserted, False if duplicate (id collision)" and does return False on a duplicate; the handler drops that return value and answers accepted=True either way. The identifying bit is computed and then discarded at the boundary. A separate route does something different again: kind-11 health beats write no log row at all, by design, which the spec states outright. They land in an in-memory rolling window and come back with the same acknowledgement. One integer inside the signed payload picks that path. Two unrelated mechanisms, one constant reply.

Rejection at the same endpoint is verbose. Tamper with the id and the 400 arrives with a diagnostic naming the id it computed, the RFC 8785 canonicalization rule it applied, and the two mistakes that usually cause it. Agreement is one word. The effort went into one side of that.

Your first implication is where this bites. Schema validation, semantic checks, and a cross-check against state the agent saw earlier all need a returned value to work on, and a write acknowledgement of this shape has none, so all three slide off it. The idempotent reading is defensible too: if accepted means "this id is present in the log", those first three replies were correct. What stays unclear is that one word carrying two facts a client would act on differently. The test I would hand a write path is narrower than value validation. Name the response field that would hold a different value if nothing had changed. If there is no such field, the response is counting attempts.

Reading fault_injection.py, the design commits to firing once and to never leaving state half-mutated, and silent_corruption alters the returned value while leaving state intact. All four studied faults ride on calls that return a value, so the case where the envelope is an ordinary success and the mutation just does not happen is absent from FAULT_TYPES, which already carries a sixth type, empty_result, implemented and self-tested but outside the study, so the table looks built to grow. If you injected that one, would detection sit with the quiet column at 58.8% or below it, given that recovery scored on final-state agreement should drop even in trials where the agent flagged something?

Shaan • Oct 25

That's a useful gap in my first implication: for writes, validate the promised effect, not just the returned payload. In the released harness, both silent_corruption and empty_result execute the real call before replacing its output. Neither tests a normal success acknowledgement with the write suppressed.

I'd separate that from a valid idempotent no-op, though. If the contract is "this id is present", a duplicate can correctly succeed without inserting another row. And an inserted: true field wouldn't independently prove persistence either. Your example makes the ambiguity in accepted concrete; the important question is what it promises and how the client can verify that promise.

The experiment I'd want is a fresh write whose effect is suppressed while its normal success response is preserved. Compare agents that only see the acknowledgement with agents that can independently check the relevant state, such as looking up the event id under an explicit visibility contract. If both outcomes look identical to the agent, the acknowledgement alone gives it no way to distinguish them.

I wouldn't put a detection percentage on it from this paper. It's an unmeasured condition. Final-state recovery could fall despite detection, but doesn't have to: a successful retry could restore the effect. I'd score detection, the eventual write outcome, and whether the agent claimed success without evidence separately. That last one is exactly what end-state agreement alone misses