FAQ: Five Tool-Call Myths That Never Touch Disk

Did the agent "confirm" the patch with a tool?

I hear that line in review threads constantly. It sounds like closure. It is not.

A tool call is a request with a story attached. Disk, git, and exit codes still decide. Want the shorter version? Treat every tool result as a claim.

Why this FAQ exists

Agents look busy. They list files. They grep symbols. They narrate tests.

Does busy equal true on your laptop? I do not buy it. I want a hash, a path, and a numeric exit code.

Cheap loops make the theater worse. Free model access invites extra calls. A free server invites extra retries. Extra noise is not extra truth.

What I store after a "successful" session

I keep three fields. Nothing else counts as proof.

• claimed path
• content hash
• command exit code

No hash? No merge. That is the whole policy.

Myth 1: The tool returned, so the write landed

The claim: "I called the write tool. We are done."

Why it spreads: The chat shows a tidy success sentence. Humans trust complete sentences. Runtimes do not owe you that sentence.

The model can narrate a write. The sandbox can drop it. The wrapper can skip the syscall. Which layer failed? You will not know from prose.

Evidence I actually run:

# Proposed check. Label it unexecuted until you run it.
path="src/app.ts"
test -f "$path" || echo "MISSING $path"
sha256sum "$path"
git status --porcelain -- "$path"

Did test -f fail? The write never happened. Did git stay clean? The write never entered the tree.

Corrected mental model: Treat Write as a proposal. Treat the filesystem as the judge. The chat log is a witness with amnesia.

Myth 2: A Read tool means the summary is faithful

The claim: "It read the file, so the recap is accurate."

Why it spreads: Read looks like grounding. Grounding is not summarization. Truncation happens. The model fills gaps with fluent guesses.

I do not review the recap. I review a unique string from disk.

Evidence I actually run:

# Pick a symbol the recap named. Prove it exists.
rg -n "function chargeInvoice" src --glob '*.ts' | head
# If the recap named a symbol, and rg is empty, the recap lied.

Could the agent have read a different buffer? Yes. Could the tool have clipped the tail? Also yes. Your eyes on a summary are not eyes on the file.

Corrected mental model: A Read event proves a fetch was attempted. It does not prove the tokens you saw were complete. Quote disk. Do not quote the agent.

Myth 3: Empty output means the path does not exist

The claim: "Grep returned nothing, so the code is absent."

Why it spreads: Empty feels like a negative proof. Empty is often a wrong cwd. Empty can be ignore rules. Empty can be a truncated tool cap.

I ask four boring questions before I believe "not found."

1. What is the working directory?
2. What is the git root?
3. Did the glob exclude the file?
4. Did the tool hit a size cap?

Evidence I actually run:

pwd
ls -la
git rev-parse --show-toplevel
git ls-files | rg "invoice" | head
# Still empty? Then argue absence. Not before.

Wrong folder is the cheap failure. Agents assume they are at the repo root. Many wrappers start in a scratch dir. Did you print pwd? If not, you guessed.

Corrected mental model: Empty output is an observation about a command. It is not a map of the repository. Fix the oracle. Then repeat the question.

Myth 4: Retrying the same tool repairs a bad assumption

The claim: "It failed once, so two more retries will settle it."

Why it spreads: Retries look like diligence. They are often the same wrong question. The agent assumed npm. The repo uses pnpm. The second retry still calls npm.

I do not increment retries first. I change the question.

A small decision table I keep in the PR:

Evidence I actually run:

# Proposed: record the exact argv, not the agent's nickname for it.
printf '%s\n' "$0" "$@" > /tmp/argv.txt
command -v pnpm
command -v npm
# If the lockfile is pnpm-lock.yaml, npm test is the wrong oracle.
ls -1 pnpm-lock.yaml package-lock.json yarn.lock 2>/dev/null

Same tool, same cwd, same argv. That is not investigation. That is stubbornness with latency.

Corrected mental model: A retry is only useful after the assumption changes. Count unique oracles, not identical calls.

Myth 5: A bigger tool schema makes the agent honest

The claim: "We added more tools, so it cannot bluff."

Why it spreads: Schemas look like contracts. Models still paraphrase tool JSON. They still omit error fields. They still promote a partial stdout into a full victory.

More tools means more ways to look busy. MCP lists do not add integrity. Integrity is an artifact you keep outside the model.

I want the raw payload. I do not want a haiku about the payload.

Evidence I actually run:

# Proposed: if you log tool events, hash the raw result body.
# Replace the file with your own event dump. Do not invent a vendor format.
python3 - <<'PY'
from hashlib import sha256
from pathlib import Path
p = Path("tool_result.json")
data = p.read_bytes() if p.exists() else b""
print("bytes", len(data))
print("sha256", sha256(data).hexdigest() if data else "NO_EVENT_FILE")
PY

No event file? Then you reviewed a story. A story is not a trace.

Corrected mental model: Tools are I/O. Honesty is a stored byte string. Schema size does not move that line.

Where a free loop still helps

Need a cheap place to replay the same commands? MonkeyCode's free model access and free server option can host that loop.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I still export hashes. The free box is not an archive. Kill the session and the workspace may vanish. That is expected. Plan for it.

Artifact: a proposed verify_tool_claim.sh

This is a local checklist. It talks to your disk. It does not call a vendor API. Run it after any agent claims a write, a test, or a missing file.

#!/usr/bin/env bash
# Proposed workflow. Unexecuted until you chmod +x and run it.
set -euo pipefail

usage() {
  echo "usage: $0 <path> [expected_sha256]" >&2
  exit 2
}

[[ $# -ge 1 ]] || usage
path=$1
expected=${2:-}

echo "== cwd =="
pwd

echo "== git root =="
git rev-parse --show-toplevel

echo "== path exists? =="
if [[ -f $path ]]; then
  echo "YES $path"
else
  echo "NO $path"
fi

echo "== hash =="
if [[ -f $path ]]; then
  actual=$(sha256sum "$path" | awk '{print $1}')
  echo "$actual  $path"
  if [[ -n $expected ]]; then
    if [[ $actual == "$expected" ]]; then
      echo "HASH_MATCH"
    else
      echo "HASH_MISMATCH expected=$expected"
    fi
  fi
else
  echo "NO_HASH"
fi

echo "== git porcelain =="
git status --porcelain -- "$path" || true

echo "== last test exit file (optional) =="
if [[ -f .agent-last-exit ]]; then
  echo "exit=$(cat .agent-last-exit)"
else
  echo "NO_EXIT_FILE"
fi

Wrap tests so the exit file exists. Do not trust a sentence that says "tests passed."

# Proposed test wrapper.
set +e
pnpm test
code=$?
set -e
printf '%s\n' "$code" > .agent-last-exit
exit "$code"

Commit the script if you want. Keep .agent-last-exit out of git. Hash the source, not the chatter.

How I use the script in a review

I paste four lines into the PR. Reviewers can rerun them.

path: src/app.ts
sha256: <actual>
exit: 0
porcelain: M src/app.ts

If porcelain is empty and the agent claimed a write, I bounce the PR. If the hash is missing, I bounce the PR. If exit is missing, I bounce the PR.

Harsh? Yes. Faster than arguing with a transcript? Also yes.

Limitations

This FAQ does not measure model quality. It does not rank hosts. It does not promise retention on a free server.

Hashes fail if you format the file after hashing. Wrappers fail if the agent never reaches the shell. rg fails if the ignore file hides the hit. None of that is mysterious. It is why the receipt lives outside the model.

I also do not handle binary assets here. Use a different checksum tool for those. Do not paste binaries into chat.

Who should not use this approach

Skip this if you already have locked CI and artifact storage. You do not need a second religion.

Do not use a shared free server for secrets. Do not paste tokens into tool args. Do not treat session disk as a backup.

Regulated releases need a real runner. This checklist is a reviewer aid. It is not an audit system.

The mental model I want you to steal

Ask one question after every confident agent sentence.

Did anything hit disk that I can hash?

If the answer is no, you still have a story. Stories do not merge. Commands do. Exit codes do. Trees do.

Keep the hash file next to the PR. Then close the chat.