Replay the Seam: A Differential Gate for the Smallest Safe Refactor

Capture calls at one seam. Replay them against both versions. Widen the change only when the diff is empty.

That is the whole method. A characterization test proves your repo still runs. A seam replay proves one boundary still answers the same way.

A passing replay is evidence, not proof. It cannot see branches your corpus never reached.

Why the diff review fails on messy repos

Most refactors start with reading a diff until it feels safe. Feeling is not a signal. Reviewers also read the new code, not the old behavior.

A rounded total is a good illustration. legacy.py returns Decimal, the refactor returns float. Code review sees two numbers. The replay sees 12.30 versus 12.300000000000001.

The gate is narrow on purpose. One seam, one corpus, one comparison function. Everything else stays untouched.

Step 1: Pick exactly one seam

A seam is a function you can call from outside without starting the app. It takes plain arguments and returns a plain value.

Use this checklist before recording anything.

If a seam fails two rows, skip it and pick a neighbor. Nested impure seams are why refactors stall.

Step 2: Record a corpus at the boundary

Wrap the current function. Log arguments and output to JSONL. Write nothing else.

The sketch below is unexecuted. Adapt names to your repo.

# tools/recorder.py
import json, pathlib

CORPUS = pathlib.Path("corpus/seam.jsonl")

def record(fn, *args, **kwargs):
    out = fn(*args, **kwargs)
    CORPUS.parent.mkdir(parents=True, exist_ok=True)
    row = {"args": list(args), "kwargs": kwargs, "out": out}
    with CORPUS.open("a") as fh:
        fh.write(json.dumps(row, sort_keys=True, default=str) + "\n")
    return out

Then call record from one place: the production entry point or an existing test fixture. Do not change logic inside the wrapper.

Let it run until the corpus covers your known cases. Thirty rows with edge cases beat ten thousand duplicates. Check the tail of the file for secrets before committing.

Step 3: Replay against the candidate

Write the candidate next to the original. Both must be importable at the same time. Replay runs the same rows through the new function.

# tools/replay.py
import importlib, json, pathlib, sys

def load(path):
    text = pathlib.Path(path).read_text()
    return [json.loads(line) for line in text.splitlines() if line.strip()]

def replay(rows, fn, eq=lambda a, b: a == b):
    bad = []
    for i, row in enumerate(rows):
        try:
            got = fn(*row["args"], **row["kwargs"])
        except Exception as exc:
            got = {"__raised__": type(exc).__name__}
        if not eq(row["out"], got):
            bad.append((i, row["out"], got))
    return bad

if __name__ == "__main__":
    rows = load("corpus/seam.jsonl")
    mod = importlib.import_module(sys.argv[1])   # e.g. billing.v2
    fn = getattr(mod, sys.argv[2])              # e.g. total_for
    bad = replay(rows, fn)
    print(f"replayed={len(rows)} diverged={len(bad)}")
    for i, want, got in bad[:5]:
        print(f"row={i}\n  want={want!r}\n  got ={got!r}")
    sys.exit(1 if bad else 0)

Run it as a normal command. Exit code 1 means stop.

python tools/replay.py billing.legacy total_for
python tools/replay.py billing.v2 total_for

The first run is your control. It should print diverged=0. If it does not, the corpus is corrupt or the seams disagree already.

Step 4: Triage divergences into four classes

Do not fix code first. Classify first. The class decides who acts.

Representation noise hides real bugs. Tighten the comparator one rule at a time. str, repr, and json.dumps are three different comparisons.

A comparator that normalizes everything compares nothing. Keep the strictest version that passes the control run.

Step 5: Move in the smallest safe steps

Use a ladder. Each rung is a separate commit with its own replay run.

1. Rename only. No logic change. Replay must show diverged=0.
2. Copy the function. Add the candidate beside the original. Nothing calls it yet.
3. Replay the candidate. Fix divergences in the candidate, not the original.
4. Switch one call site. Run the app's tests plus replay.
5. Delete the original. Only after the corpus and tests both pass.

If a rung fails, revert that rung alone. Small commits make the revert cheap and the review honest.

Set a budget before you start. Example: no rung may exceed forty changed lines. Over budget means the seam was too large. Split it.

Where the free server and free models fit

Replay runs are boring and long. They belong on a machine that is not your laptop. MonkeyCode's free server option is one place to run the harness headless, and its free model access can draft the recorder and comparator for your seam.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use the model for scaffolding, not for judgment. Ask it for a comparator covering Decimal, datetimes, and ordered dicts. Then read every line. Ask it to cluster failing rows by traceback type. Then verify the clusters by hand.

Keep the corpus out of any chat context. Send shapes, not payloads. If a row contains customer data, rewrite it before sharing.

The useful pattern is a closed loop. Generate the harness, run it on the free server, paste back the summary line, fix one rung, repeat. The model never sees your production output.

Limitations and who should skip this

Side effects break this gate. If the seam writes files or calls an API, replay changes state. Record the effects separately or pick another seam.

Non-determinism breaks it too. Clocks, random seeds, and UUIDs need pinning before replay means anything. Count them first.

Corpus coverage is the hard ceiling. A green replay means "no regression on recorded rows". It does not mean "correct".

Skip this approach when the seam is a single trivially pure function. A unit test is cheaper than a corpus. Skip it when you cannot serialize arguments without leaking secrets. Skip it when the refactor is a rewrite, because there is no control run.

Use it when the function is old, ugly, and load-bearing. That is where a forty-line rung plus an empty diff beats a long review.

The gate in one line

Record the seam. Replay both sides. Classify the diff. Widening requires zero divergences on the control and the candidate.