Pin the Failing Fixture First: A Four-Gate OSS Patch Workflow

A first-time contributor cloned a metrics library on a Sunday evening and reproduced a parser bug in twenty minutes. The failing path lived in histogram decoding after empty buckets arrived from a newly tightened encoder. The contributor then asked a coding model for a fix and received a six-file refactor of the public API. Maintainers closed the pull request because no failing test shipped with the change, and the diff outgrew the bug.

Why the fixture has to exist before the patch

Open source review bandwidth is scarce, and maintainers usually read evidence long before they read a contributor's intention. A stack trace pasted into an issue comment is not a contract that CI can fail on every later commit. A model-generated diff without a pinned input will often patch a neighboring function that merely looks related. The practical move is to freeze a smallest failing fixture, then refuse any patch that does not turn that fixture green.

This article describes a four-gate workflow that a contributor can run on a laptop before a pull request is opened. The gates are clean-clone reproduction, a frozen failing fixture, a blamed-surface patch, and a delta review against that fixture. Coding models can assist at the last gate, but they do not replace the first three. The workflow is a proposal for public, non-embargoed bugs that already have a local test runner.

Gate 1: Reproduce on a clean clone

Start from a throwaway directory so local tooling and caches cannot leak into the project's environment. Pin the issue number, the default branch SHA, and the exact command that should fail. Record those three values in a short text file that travels with the later patch.

# proposed local gate — unexecuted example
mkdir -p /tmp/oss-repro && cd /tmp/oss-repro
git clone --filter=blob:none https://github.com/example/metrics.git
cd metrics
git rev-parse --short HEAD > /tmp/oss-repro/HEAD.sha
git switch -c issue-1842-empty-buckets

python -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"

# The next command must fail before any patch is written.
pytest tests/histograms/test_parse.py::test_empty_buckets -q
echo $? > /tmp/oss-repro/baseline.exit

If the suite is green on a clean clone, the issue is not reproduced, and no model should draft a patch. Contributors should run git bisect only after the failure is both local and deterministic. A flaky failure belongs in a notes file rather than in a guessed one-line fix.

# proposed bisect only after a deterministic fail
git bisect start
git bisect bad HEAD
git bisect good v2.4.0
git bisect run pytest tests/histograms/test_parse.py::test_empty_buckets -q
git bisect reset

Gate 2: Freeze a failing fixture

The fixture is the contribution contract, because reviewers can execute it without reconstructing the reporter's laptop. It should include the smallest input, the expected output or exception, and the runtime pins that made the failure repeatable. Issue comments can stay narrative, but the frozen fixture has to remain executable in CI.

# proposed test — unexecuted example, not a claim about any real library
import json
from pathlib import Path

from metrics.histogram import parse_histogram, HistogramError

FIXTURE = Path(__file__).parent / "fixtures" / "empty_buckets.json"

def test_empty_histogram_buckets_roundtrip():
    payload = json.loads(FIXTURE.read_text())
    decoded = parse_histogram(payload)
    assert decoded.buckets == []

def test_empty_histogram_buckets_error_message():
    payload = {"buckets": None}
    try:
        parse_histogram(payload)
    except HistogramError as exc:
        assert "buckets" in str(exc)
        return
    raise AssertionError("expected HistogramError for null buckets")

{
  "schema": "histogram.v1",
  "buckets": []
}

Store the interpreter version and a lockfile hash beside the JSON so reviewers can rebuild the same failure. A short repro.md that lists the command, the SHA, and the expected traceback saves maintainers from reconstructing the reporter's laptop. The fixture is frozen when the test fails for the stated reason and for no other reason.

Gate 3: Patch only the blamed surface

Once git blame and the bisect log agree on a module, the patch budget is that module plus the new test. Public signatures stay frozen unless the filed issue explicitly asks for an API change. Import churn, formatter wars, and opportunistic refactors belong in separate pull requests after this bug ships.

# proposed scope check — unexecuted example
git diff --stat origin/main...HEAD
git diff --name-only origin/main...HEAD | sort > /tmp/touched.txt

python - <<'PY'
from pathlib import Path
allowed = {
    "src/metrics/histogram.py",
    "tests/histograms/test_parse.py",
    "tests/histograms/fixtures/empty_buckets.json",
    "repro.md",
}
touched = set(Path("/tmp/touched.txt").read_text().splitlines())
extra = touched - allowed
if extra:
    raise SystemExit(f"out-of-scope paths: {sorted(extra)}")
print("scope ok")
PY

After the patch, the previously failing test must pass, and the rest of the package tests must stay green. If the project provides a narrower marker, run that marker first to keep the loop cheap. A red-to-green transcript in the pull request body is more useful than a summary of model confidence.

pytest tests/histograms/test_parse.py -q
pytest -m "not slow" -q

Record the red-to-green transcript

Pull request descriptions that only say the bug is fixed force maintainers to rediscover the reproduction. A short transcript proves the fixture failed on the parent SHA and passed on the branch SHA. The transcript also records the exact pytest invocation, which prevents reviewers from running a different subset.

# proposed transcript capture — unexecuted example
PARENT=$(git merge-base HEAD origin/main)
git stash push -u -m patch
git checkout "$PARENT"
pytest tests/histograms/test_parse.py::test_empty_histogram_buckets_roundtrip -q; echo "parent_exit=$?"
git checkout -
git stash pop
pytest tests/histograms/test_parse.py::test_empty_histogram_buckets_roundtrip -q; echo "branch_exit=$?"

Paste both exit codes into repro.md before requesting review from a person or a model. If the parent SHA does not fail, the new test is not a regression test and the patch is not reviewable under this workflow. This check catches tests that were written to match the patch instead of the original bug.

Gate 4: Review the delta against the fixture

Human review still decides whether the project will merge, including style, timing, and API freeze. A model can score the diff against the frozen fixture, the allowed paths, and the written invariants. The prompt should receive the test file, the production diff, and repro.md, not the entire repository tree.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. Contributors without a paid inference budget can run this optional review on MonkeyCode's free model access and free server option. The product only hosts that review step; the four gates remain useful if that mention is removed.

# proposed review brief — unexecuted example, no model names implied
You are reviewing one open source patch.
You may use only the files in this bundle:
- repro.md
- tests/histograms/test_parse.py
- tests/histograms/fixtures/empty_buckets.json
- the unified diff of src/metrics/histogram.py

Reject the patch if any statement below is false:
1. The new test fails on the parent SHA and passes on this SHA.
2. The diff does not change public function signatures.
3. The diff does not touch files outside histogram parsing.
4. Error messages still include the field name "buckets".
5. No TODO, hardcoded secret, or license header rewrite appears.

Reply with PASS or FAIL and a bullet list of evidence from the diff.

Treat the model output as an untrusted checklist rather than as a merge decision from the project. Re-run the frozen fixture locally after every accepted suggestion before amending the commit. Discard advice that enlarges scope, invents helpers the tree does not contain, or rewrites tests until they match the patch.

Decision table

The table below compresses the gates into a stop-or-go rule a contributor can apply in a few minutes. Model assistance stays optional and should appear only after a deterministic fixture already exists. Anything that fails a row should block both the patch and the review prompt.

Limitations

This loop assumes the project has an automated test runner that a contributor can execute without private services. It does not estimate model quality or capacity, and it does not claim a free review host matches maintainer CI. Generated patches still hallucinate APIs, drop edge cases, and look confident while remaining factually wrong. The scope script only checks path names; it cannot prove semantic compatibility with downstream users.

Embargoed security issues, bugs that need production traffic, and changes to cryptographic code should not use a public model review step. Contributors who cannot reproduce the failure should not request a patch from any model. Drive-by refactors that lack a fixture waste maintainer time even when a review bot says PASS.

Who should skip this approach

Skip the four gates for documentation-only edits, comment typos, or release chores maintainers already scripted. Skip model review when the project's license or contribution guide forbids sending code to external services. Skip the whole pattern when the reporter cannot share a fixture without leaking customer data.

The fixture-first loop is for contributors who want a mergeable, reviewable patch rather than a plausible looking diff. Maintainers still own architecture, release timing, and the choice of whether the public API may move. Readers who try the gates should keep repro.md in the pull request so the next person can replay the failure.