Budget the Free Tokens Like Lab Chemicals: A Bootcamp Lab on Metered Agent Runs

Your free quota is not a budget until something refuses to spend past it. That single sentence is the whole lab.

Most agent tutorials teach students to call a model and print the answer. Then the cohort discovers, the hard way, that a retry loop can eat an entire free allowance before anyone notices. Sound familiar?

This is a 90-minute bootcamp lab I run as a discipline exercise: build the meter before you build the agent. It uses only the Python standard library, so a clean clone runs with zero installs. The one thing it cannot give you is exact billing numbers — and that limitation is part of the grade.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The 2 a.m. failure this prevents

Free model access and a free server option are genuinely useful for a classroom: nobody has to collect credit cards from twenty students before the first session. MonkeyCode's open-source project advertises both, and the operator describes an allowance on the order of ten million tokens. Verify the current terms yourself before you build a syllabus on them; free-tier numbers move, and I am not going to pretend otherwise.

But free access fails quietly. It does not raise an error at minute two. It raises one at minute forty, in the middle of a demo, with no record of which step spent what.

There is a running DEV thread this week about tools that sit between a developer and the model. Fair complaint. A spend ledger is the smallest honest answer to it: you can see what actually crossed the wire, and what it cost.

What students build (three parts, in this order)

1. A ledger. Append-only rows: run id, step, model, prompt size, output size, estimated tokens, wall time.
2. A guard. A pre-flight check that refuses to start a step once spending reaches the budget minus a reserve.
3. An adapter. One function that actually talks to an endpoint. Everything else must work without it.

That third point matters for grading. If the adapter is a stub, the lab is testable offline and reproducible on a plane.

Setup (about 10 minutes)

python3 -m venv .venv && . .venv/bin/activate
mkdir -p lab && cd lab
python3 -c "import sqlite3, json, time; print('stdlib ok')"

No pip install. Students write ledger.py and runner.py, then commit both.

Checkpoint 1 — the ledger writes before the guard exists

# ledger.py — stdlib only. Not run against any live endpoint; the adapter is yours.
import sqlite3, time

DB = "runs.sqlite"

def init(db=DB):
    con = sqlite3.connect(db)
    con.execute("""CREATE TABLE IF NOT EXISTS spend(
        run_id TEXT, step INTEGER, model TEXT,
        prompt_chars INTEGER, completion_chars INTEGER,
        est_tokens INTEGER, wall_ms INTEGER, ts REAL)""")
    con.commit()
    return con

def estimate_tokens(text: str) -> int:
    # Crude on purpose: ~4 characters per token for English prose.
    # Swap in the provider's own tokenizer if it exposes one.
    return max(1, len(text) // 4)

def record(con, run_id, step, model, prompt, completion, wall_ms):
    est = estimate_tokens(prompt) + estimate_tokens(completion)
    con.execute("INSERT INTO spend VALUES (?,?,?,?,?,?,?,?)",
        (run_id, step, model, len(prompt), len(completion), est, wall_ms, time.time()))
    con.commit()
    return est

def spent(con, run_id):
    (n,) = con.execute(
        "SELECT COALESCE(SUM(est_tokens),0) FROM spend WHERE run_id=?", (run_id,)).fetchone()
    return n

Why a database and not a counter in memory? Because the question students ask at hour two is "what did step seven do?" A counter cannot answer it.

Checkpoint 2 — the guard must fail on request

# runner.py
import time
import ledger

BUDGET  = 10_000_000   # headline figure as described by the operator; confirm current terms
RESERVE = 50_000       # never spend the last slice, so a demo can always finish

class BudgetExceeded(RuntimeError):
    pass

def call_model(prompt, model):
    raise NotImplementedError("Wire your endpoint here. Keep it behind this one function.")

def step(con, run_id, n, prompt, model="default"):
    if ledger.spent(con, run_id) >= BUDGET - RESERVE:
        raise BudgetExceeded(f"run={run_id} step={n} stopped at the reserve line")
    t0 = time.monotonic()
    out = call_model(prompt, model)
    ms = int((time.monotonic() - t0) * 1000)
    used = ledger.record(con, run_id, n, model, prompt, out, ms)
    print(f"step {n}: +{used} est, total {ledger.spent(con, run_id)}/{BUDGET}")
    return out

Now the red test. Monkeypatch call_model to return the same fixed string every time, set BUDGET = 400 inside the test, and assert three things:

• the run raises BudgetExceeded rather than returning a short answer;
• no row exists for the step that was refused;
• spent() equals exactly the sum of the rows printed.

If a student's guard "passes" by lowering the answer quality instead of stopping, that is a fail. A guard that degrades quietly is worse than no guard.

Checkpoint 3 — resume without re-spending

Add one function: read MAX(step) for the run and continue from there. Then kill the process mid-run and restart it. The ledger must show no duplicate step numbers and no re-charged work.

This is where free servers earn their keep in the lab. A free host is a fine place to discover that your runner is slow, that cold starts blow up your wall_ms column, and that your resume path was never actually tested. It is not a place to discover cleverness. Measure, don't assume — and if the host sleeps or throttles, your ledger tells you when, in milliseconds, not in vibes.

Stretch goals (pick two)

1. Tokens per accepted step. Divide estimated tokens by steps that produced a usable artifact. The number is usually ugly.
2. Two-model A/B on identical prompts. Same prompts, same order, compare spend at equal acceptance.
3. Hard step cap. A token budget does not stop a loop that fails cheaply. Add MAX_STEPS and a test for it.
4. Estimator error report. Twenty samples, compare estimate_tokens against provider-reported usage if available, publish the mean error.

A fair grading rubric

Note what is not on the rubric: whether the agent was impressive. Impressive is ungradable. Repeatable is not.

Who should not use this approach

The estimator is deliberately crude. At four characters per token it under-counts code, JSON, and non-Latin scripts, so anyone who needs invoicing-grade numbers should use provider-reported usage, not this. If your assessment depends on a free tier staying available for the full exam window, don't — keep a paid fallback or run the adapter in a recorded-fixture mode. And if a student only wants to call an API once to see it work, this lab is overkill and will feel like paperwork.

For a cohort that needs a shared default without a procurement conversation, the free model access and free server option are a reasonable on-ramp — with the ledger as the source of truth and the terms you measured written at the top of the repo. Try it with one group of five before you commit a syllabus to it.