Before an AI Rewrites Legacy Code, Build a Parity Gate

A team can generate a new implementation faster than it can decide whether the new implementation is still the same product.

That is the important tension behind a report that Cisco president Jeetu Patel gave a 27-year-old engineer unlimited AI tokens and a goal to rewrite 98% of Cisco's legacy code within 24 months, with a four-week proof of concept. The account comes from Fortune reporting summarized by Tom's Hardware. The headline is striking. The engineering problem underneath it is less glamorous: how do you know a rewrite still does what users and dependent systems expect?

This article is a source-based engineering guide, not a report from a Cisco project or a hands-on legacy migration. The public coverage describes an ambition and a proof-of-concept window, not completed results. It does not establish that 98% has been rewritten, that the effort will meet the deadline, or that token supply is the limiting factor. The useful takeaway is a method you can apply to a small, reversible change: create a parity gate before asking an agent to replace behavior.

Why generated code is the easy part

A legacy system often has no single complete specification. Its behavior is distributed across code, database constraints, production data, integrations, error handling, and assumptions that nobody wrote down. A rewrite can look cleaner and still change a default, rounding rule, ordering guarantee, or failure mode that another system depends on.

The first question is not, “Can an agent reproduce this function?” It is, “What observable behavior must remain stable, and what evidence would show that it changed?”

The distinction is practical. A test that merely calls the new code and checks that it returns a plausible value proves little about equivalence. A parity test runs the same representative inputs through the old and new implementations and compares the outcomes. A reviewer can then inspect disagreements instead of trying to infer correctness from a large diff.

Microsoft's guidance for migrating AI agents recommends defining acceptance gates before a model change, capturing a baseline from the current system, using the same test data and environment assumptions for both versions, and investigating individual regressions rather than trusting an average score. Those ideas apply just as well to a code rewrite. See Microsoft's migration-validation guidance. For coding-agent work, Visual Studio Code's testing guide also says to preserve assertions, report what ran, and treat unrun tests as unverified.

Build the smallest useful parity harness

Start with a behavior that is bounded, deterministic, and important enough to matter. A pricing rule, date conversion, permission decision, or serialization function may be a better first slice than an entire service. Do not begin with the hardest code in the repository. Begin with a seam where you can call both implementations under controlled conditions.

For illustration, imagine an old and new implementation that both accept JSON inputs and return JSON results. The harness below reads one case per line, sends each input to both commands, and records exact output mismatches. It intentionally treats a process error as a failure rather than silently skipping it.

#!/usr/bin/env python3
"""Compare legacy and candidate commands on newline-delimited JSON cases."""
import json
import subprocess
import sys
from pathlib import Path

CASES = Path(sys.argv[1])
OLD_COMMAND = ["python3", "legacy_impl.py"]
NEW_COMMAND = ["python3", "candidate_impl.py"]


def run(command, case):
    completed = subprocess.run(
        command,
        input=json.dumps(case),
        text=True,
        capture_output=True,
        check=False,
        timeout=10,
    )
    if completed.returncode != 0:
        raise RuntimeError(
            f"{command[1]} exited {completed.returncode}: "
            f"{completed.stderr.strip()}"
        )
    return json.loads(completed.stdout)


def main():
    cases = [json.loads(line) for line in CASES.read_text().splitlines() if line]
    mismatches = []
    for index, case in enumerate(cases, start=1):
        old = run(OLD_COMMAND, case)
        new = run(NEW_COMMAND, case)
        if old != new:
            mismatches.append({"case": index, "input": case,
                               "legacy": old, "candidate": new})

    print(json.dumps({
        "cases": len(cases),
        "mismatches": len(mismatches),
        "details": mismatches,
    }, indent=2, sort_keys=True))
    return 1 if mismatches else 0


if __name__ == "__main__":
    raise SystemExit(main())

Save representative inputs in cases.jsonl, one JSON object per line, then run python3 parity.py cases.jsonl. Replace the two command arrays with real, isolated entry points. The example is intentionally strict: if the output formats differ, the harness reports a mismatch. Normalize only fields that are genuinely nondeterministic, and document each normalization so it cannot hide a meaningful regression.

This small script is not a migration framework. It is a starting gate. Production systems may need database snapshots, service stubs, message queues, feature flags, and stronger isolation. The key is that each version sees the same input and that you retain the evidence for every comparison.

Choose cases that reveal contracts

A collection of ordinary happy-path examples is not a useful characterization set. Include cases that make hidden behavior visible:

• Typical input: common values from real use, with sensitive data removed or safely synthesized.
• Boundaries: empty strings, minimum and maximum values, timezone transitions, rounding edges, and large payloads where relevant.
• Invalid input: missing fields, malformed values, and unsupported combinations.
• Failure behavior: timeouts, downstream errors, unavailable dependencies, and retry limits.
• Compatibility details: field ordering if consumers rely on it, exact error codes, null versus omitted values, and stable identifiers.
• Previously fixed bugs: regressions that the old code now avoids should be explicit cases.

Do not ask an agent to invent the expected result for every case and then treat its own answer as the oracle. Where possible, derive expected outcomes from existing tests, approved requirements, recorded behavior, or a domain owner. When the old implementation is the reference, it can reveal current behavior, but it cannot tell you whether that behavior is correct. Mark known defects as exceptions with an explicit decision rather than preserving them accidentally.

Make mismatches reviewable

A raw “17 tests failed” message is not enough. Each mismatch should show the input, the old output, the candidate output, and a stable case identifier. Group related mismatches by behavior: rounding, validation, authorization, serialization, and so on. A reviewer can then decide whether a difference is a bug, an intended improvement, or an old defect that should not be carried forward.

Keep three outcomes distinct:

1. Equivalent: the difference is not observable to supported callers, and the reason is documented.
2. Approved change: behavior is intentionally changing, with an owner and a migration or compatibility plan.
3. Unresolved: the difference blocks promotion until someone decides what the contract should be.

Never turn a mismatch into a pass by weakening the test just to make the candidate green. Microsoft's guidance explicitly recommends investigating individual regressions rather than relying on an aggregate rate. A 99% pass rate can conceal the one authorization or billing case that matters most.

Put the agent inside the gate, not above it

A coding agent can help inventory call sites, propose test cases, explain mismatches, and implement a candidate behind a feature flag. Those are useful tasks. It should not be allowed to change the acceptance criteria and approve its own output in the same unreviewed step.

A safer task boundary looks like this:

1. Ask the agent to map the selected behavior and identify callers, tests, data assumptions, and failure paths. Require file and line references so a human can verify the map.
2. Have a person review the map and choose the initial cases. Add cases for critical requirements that are missing from the existing suite.
3. Ask the agent to implement the replacement without editing the parity cases or their expected values. Protect those files through repository permissions or a separate review boundary where possible.
4. Run both implementations on the same fixed corpus. Save the command, version identifiers, environment, results, and mismatch report with the change.
5. Review every mismatch, inspect the final diff, and run the relevant existing test suite before merging.
6. Roll out gradually behind a reversible switch. Compare production signals where safe, and define a rollback trigger before the rollout begins.

This also makes “unlimited tokens” less important as a planning metric. More generation can increase the amount of code to inspect. The scarce resource may be trustworthy test cases, time from domain owners, or a review path that catches an unintended behavior change. That is an engineering inference, not a reported finding about Cisco's internal project.

A promotion checklist worth saving

Before replacing a legacy component, ask:

• Is the selected slice small enough to revert independently?
• Does the old implementation run in a controlled baseline environment?
• Are cases drawn from real contracts and known incidents, not only agent-generated examples?
• Are critical paths weighted separately from low-risk cases?
• Can the harness reproduce each mismatch from a saved input?
• Are nondeterministic fields normalized narrowly and transparently?
• Are test files and expected outputs protected from the implementation agent?
• Does every accepted behavior change have an owner and compatibility plan?
• Do performance, security, privacy, and operational checks supplement functional parity?
• Is rollout gradual, observable, and reversible?

A parity harness is not proof that a rewrite is safe. It is a way to turn a vague claim of equivalence into inspectable evidence. It tells you where to focus human attention, and it makes the decision to proceed, pause, or roll back more concrete.

Start with evidence, then scale

The public Cisco account is a striking example of how AI changes the perceived cost of attempting a large rewrite. But a deadline and token budget are not evidence of preserved behavior. Until results are independently reported, the 98% figure belongs to the stated ambition, not the outcome.

For an engineering team considering a similar effort, begin with one narrow behavior and a fixed corpus. Make the old and new outputs comparable. Let the agent help produce the candidate, but keep the contract, mismatch decisions, and promotion gate reviewable by people responsible for the system. Expand only when the gate catches the kinds of mistakes you care about and the rollback path has been exercised.

I write about software engineering, developer tools, and practical AI workflows. Subscribe if you want more guides like this. Have you used a parity harness for a rewrite or migration? What behavior did it catch that ordinary unit tests missed?