200 agents, 2,011,438 tool calls: who's paying for your AI?

A team I worked with shipped 200 internal agents last quarter. The monthly bill landed at 2,011,438 tool calls. Finance asked the usual thing: "which model is eating our budget?"

The intuitive answer — "the frontier model, obviously" — was wrong. And why it was wrong says something useful about how agentic systems actually spend money.

The counterintuitive part

Most token spend in an agent loop isn't in the hard reasoning step you're picturing. It's in the boring middle:

• the 60-line retrieval that reformats a doc the model already saw,
• the JSON reshaping between two tools,
• the "which tool next?" classifier that fires before every action,
• the summary pass that condenses a transcript nobody reads.

None of those need a frontier model. They need a model — a cheap one, routed to that specific sub-task. But the agent was wired with one "default" model for the whole loop, so every sub-task paid frontier prices.

We measured it: roughly 80% of the bill was sub-tasks that cleared the quality bar at ~1/20th the cost on a smaller model.

This is a routing problem, not a prompting problem

You don't fix this with a cleverer system prompt. You fix it by making the sub-task boundary visible to a router.

Instead of agent.llm = frontier_model, each sub-task declares what it needs:

subtasks:
  retrieve_context:
    min_capability: embedding-lookup
    route_to: small-retriever      # cheap, no reasoning needed
  classify_next_tool:
    min_capability: light-classification
    route_to: mini-classifier      # tiny, fast
  synthesize_answer:
    min_capability: frontier-reasoning
    route_to: frontier-model       # only here do we pay up

The router picks the cheapest model that clears min_capability. The agent's behavior doesn't change. The bill does.

flowchart LR
    A[Sub-task emitted] --> R{Router}
    R -->|embedding-lookup| S[small-retriever]
    R -->|light-classification| M[mini-classifier]
    R -->|frontier-reasoning| F[frontier-model]
    S --> L[per-step ledger]
    M --> L
    F --> L
    L --> C{PDPA-aligned?}
    C -->|yes| OK[serve in SG-hosted region]
    C -->|no| B[route to compliant region]

Per-step cost attribution is the missing half

Routing only helps if you can see it. Most teams have one line item: "LLM." That hides the problem — finance sees a number, not a decision.

Attach cost to every sub-task, not every request:

def route(subtask):
    model = cheapest_that_clears(subtask.min_capability)
    cost = price_per_1k[model] * estimate_tokens(subtask)
    ledger.log(subtask.name, model, cost)   # per-step, not per-request
    return model

Now the bill reads by capability tier: "retrieval $X, classification $Y, synthesis $Z." When synthesis balloons, you know which sub-task to attack — not which model to blame.

Cost and compliance converge on the same layer

Here's what surprised the team: the router was also where data-residency rules had to live. Some sub-tasks touched regulated customer data, so they had to stay in a PDPA-aligned, SG-hosted region even when a cheaper model sat elsewhere.

So the same routing layer that cut the bill also enforced jurisdiction. Cost optimization and compliance stopped being separate meetings — they became one config block.

That's the real unlock: a routing layer isn't infrastructure you tolerate. It's where your cost and your constraints get expressed as code.

The question I'll leave you with

If you mapped every sub-task in your agent loop to the cheapest model that clears its quality bar, how much of your current bill would survive?

No tool to sell, no link to click. Just this: most agent bills are really routing bills in disguise.