A Tool Round-Trip Is Not a Load Test: A Myth-Busting FAQ

A pull request stalled on a quiet screenshot from an agent session that looked like performance proof. An AI coding agent had called a billing endpoint four times and received HTTP 200 on every attempt. The author treated that transcript as evidence the route would survive a launch-day spike without further measurement. Nobody had recorded concurrency, payload mix, or tail latency against a stated service objective.

That scene keeps repeating because a tool call resembles a client, and a client resembles a test. It is neither of those things once the conversation turns from wiring to capacity. A tool round-trip is a scripted, usually serial conversation with an API that happens to return JSON the model expected. The FAQ below separates a drafting loop you can run on a free coding server from the measurement loop that actually answers performance questions.

Why the myth spreads so easily

Public tool-calling demos make the confusion worse, because the happy path is visually complete. You watch a model pick a function, fill arguments, and print a 200, which feels like the system was exercised. Realistic API tests still need a defined operation mix, think time, and a clock that is not the model's turn timer. When those pieces are missing, the transcript is a functional anecdote, not a load profile.

Free coding environments add a second distortion that is easy to miss during a busy review. Shared runtimes, courtesy hosts, and leftover warm processes change latency in ways that have nothing to do with your handler. If you later quote those timings as p95, you are measuring the furniture in the room, not the floor. Keep the drafting machine and the measuring machine conceptually apart, even when both of them are cheap to run.

Myth: four successful tool calls mean the route is fast enough

The claim you will hear is that the agent reached the endpoint, the body parsed, and therefore latency is acceptable. Evidence against that claim is boring and structural rather than dramatic. Four serial requests do not create queueing, lock contention, connection-pool exhaustion, or pauses that appear only under overlap. A corrected mental model is simple: a tool round-trip is an integration ping with a narrative wrapper, useful for wiring and useless as a percentile.

Consider a small counterexample you can run without any agent sitting in the path. The script issues overlapping requests with a bounded worker pool and records wait time separately from application chatter. If your only prior evidence was an agent log, this is the first moment you have a distribution instead of a story.

# proposal: local timing harness, not an agent loop
import concurrent.futures
import json
import statistics
import time
import urllib.error
import urllib.request

URL = "http://127.0.0.1:8080/checkout"
HEADERS = {"Content-Type": "application/json"}
BODY = json.dumps({"sku": "sku-9", "qty": 1}).encode()

def one_call(_idx):
    started = time.perf_counter()
    req = urllib.request.Request(URL, data=BODY, headers=HEADERS, method="POST")
    try:
        with urllib.request.urlopen(req, timeout=5) as resp:
            resp.read()
            code = resp.status
    except urllib.error.URLError:
        return 0, (time.perf_counter() - started) * 1000
    return code, (time.perf_counter() - started) * 1000

def run_burst(n=40, workers=8):
    with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as pool:
        rows = list(pool.map(one_call, range(n)))
    ok = [ms for code, ms in rows if code == 200]
    print("ok", len(ok), "of", n)
    if len(ok) >= 20:
        print("p50_ms", statistics.median(ok))
        print("p95_ms", statistics.quantiles(ok, n=20)[18])
    else:
        print("not_enough_samples_for_percentiles", len(ok))

if __name__ == "__main__":
    run_burst()

If that burst disagrees with the agent transcript, believe the burst and keep the transcript as wiring notes. The transcript never promised a queueing model; it only promised that one caller could parse a body. Treat disagreement as a signal that the myth was doing the work of a test plan.

Myth: the agent's call pattern is your real traffic mix

The claim you will hear is that because the model chose list, then get, then update, the sequence must match production. Models choose tools to satisfy the current prompt, not to reproduce a day's traffic. Production mixes include retries, partial failure, idempotent replays, and clients that abandon a request. None of those shapes appear in a tidy chain of successful function calls.

A corrected mental model is to write the mix down before any model is invited to generate code. The YAML below is a scenario contract: it states weights, think time, and what you refuse to count. An assistant may later help fill fixtures, but it does not get to invent the weights during the run.

# proposal: scenario contract checked into the repo
name: checkout-read-write
base_url: http://127.0.0.1:8080
think_time_ms: [50, 200]
transport: dedicated-runner  # never the coding server
operations:
  - id: get_cart
    method: GET
    path: /cart/{id}
    weight: 0.62
  - id: add_item
    method: POST
    path: /cart/{id}/items
    weight: 0.28
  - id: checkout
    method: POST
    path: /checkout
    weight: 0.10
reject_if:
  - source: agent_transcript
  - source: coding_host_as_generator
  - source: model_turn_time_folded_into_http

You can ask a coding assistant to turn that file into a runner, which is a drafting task. You cannot ask it to certify that the weights are true. Truth for a mix comes from access logs, product analytics, or an explicit assumption you are willing to defend in writing.

Myth: the free coding host is a valid load generator

The claim you will hear is that the environment that wrote the harness can also fire the traffic, especially when model access and a server seat are free. That collapses two jobs that fail in different ways. A drafting host is optimized for editors, tool calls, and interactive tokens. A measurement host is optimized for clock stability, isolation, and a known network path to the system under test.

MonkeyCode is relevant only in the first job. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Its free model access and free server option can host the editing loop that turns a scenario contract into a runner, without implying that the same box should emit production-shaped load. Treat those availability claims as convenience for drafting, not as a published SLA, quota sheet, or hardware profile.

A practical split looks like the commands below, which keep source generation far away from the clock. Draft on the shared coding server, then copy the runner to a machine you actually control before any timing is recorded. If you cannot name the measuring machine, you do not yet have a measurement.

# draft: generate runner from the scenario contract
# run inside the coding environment; output is source, not metrics
python tools/render_runner.py \
  --from scenarios/checkout-read-write.yaml \
  --out harness.py

python -m compileall harness.py

# measure: execute on a dedicated runner you control
scp harness.py runner.internal:~/api-tests/
ssh runner.internal 'python3 ~/api-tests/harness.py --duration 90 --workers 8'

# never: curl the system under test from the drafting session
# and paste a single elapsed time into the design doc as p95

The analogy is a kitchen scale versus a loading dock, and it is meant literally. You can recipe-test a sauce on a countertop scale without shame. You do not certify a freight elevator with the same instrument, even when the scale was free. Free does not make the instrument wrong for drafting; free does make it the wrong clock for capacity.

Myth: model-loop latency is a proxy for API latency

The claim you will hear is that the whole turn felt snappy, so the endpoint must be snappy too. A model loop includes prompt assembly, tool selection, argument decoding, and often a second model turn after the HTTP response. Those costs dwarf a healthy API call, and they fluctuate with queueing on the model side. Reporting that blend as service latency is like timing a commute by how long the radio host talks.

The corrected mental model is to stamp three clocks, not one conversational clock. Stamp when the client starts the HTTP call, when the server begins work, and when the client has the full body. Ignore the model's thinking time in every percentile you publish. The snippet below isolates the HTTP span even if you keep using an assistant to write tests.

# proposal: isolate HTTP time from assistant time
import json
import time
from http.client import HTTPConnection

def http_span(host, port, path, body):
    conn = HTTPConnection(host, port, timeout=5)
    raw = json.dumps(body).encode()
    t0 = time.perf_counter()
    conn.request("POST", path, body=raw, headers={"Content-Type": "application/json"})
    resp = conn.getresponse()
    payload = resp.read()
    t1 = time.perf_counter()
    conn.close()
    return {
        "status": resp.status,
        "bytes": len(payload),
        "http_ms": (t1 - t0) * 1000,
    }

# Assistant-generated tests may call http_span.
# They must not add model thinking time to http_ms.

If a dashboard cannot show http_ms without model time folded in, the dashboard is a conversation log. Conversation logs are allowed in a design thread. They are not performance evidence, and they should not appear in an SLO appendix.

A compact decision table you can paste into a review

Reviews go faster when the myth is named before the screenshot is argued. The table is the artifact to copy into the pull request, not a slogan. Each row is a claim, the evidence that would support it, and the action if that evidence is missing.

Use the table as a gate, not as decoration after the argument has already happened. If the pull request only satisfies the first row, merge it as a functional change and open a separate measurement issue. Mixing the rows is how a tool round-trip becomes a fictional load test.

Limitations, and who should skip this workflow

This workflow assumes you can reach a local or staging API and that you are allowed to generate traffic there. It does not certify public internet paths, TLS offload, or multi-region failover, and it does not replace vendor-backed capacity planning. Numbers from a laptop and numbers from a shared coding server are both noisy. They are useful as detectors of gross error, not as contractual SLOs.

Do not use a free drafting server as a soak host, a security probe, or a production canary. Do not use model-generated mixes as billing evidence or as proof that a cache is honest. Teams under audit, teams with regulated latency contracts, and teams without a staging environment should run a proper performance practice and treat the agent session as comments in the margin.

A free drafting environment belongs in that margin as well: it can shorten the time to a runner, and it cannot sign the percentiles. If you try that drafting path, keep the host name in the same thread as the harness so reviewers know which clock you used. If you need a drafting environment with free model access and a free server while you write the harness, MonkeyCode is an open-source option for that editing loop rather than for the load itself.