Detect Breaking JSON Schema Changes in CI with a Zero-Dependency Python Check

Someone changes "id": {"type": "string"} to "integer" in a JSON Schema, the PR diff is one line, it gets approved, and three days later a downstream consumer starts dropping events. Text diffs don't know what a breaking change is. What you want is a check in CI that reads the old and new schema, understands that turning id from a string into an integer or removing an enum value breaks things while adding an optional field doesn't, and fails the build for the first kind only.

SchemaGuard is a single Python file, standard library only, that does that for JSON Schema files. Everything below was run locally with the shipped schemaguard.py, and the output is copied from those runs (header lines trimmed, terminal colours removed).

A schema and two candidate changes

The baseline, user.v1.json:

{
  "type": "object",
  "required": ["id", "email"],
  "properties": {
    "id": {"type": "string"},
    "email": {"type": "string"},
    "plan": {"type": "string", "enum": ["free", "pro", "team"]},
    "tags": {"type": "array", "items": {"type": "string"}}
  }
}

Candidate A (user.v2-safe.json) adds an enterprise plan and an optional locale field.
Candidate B (user.v2-break.json) makes locale required, turns id into an integer, drops the team plan and changes tags items to objects.

Running it

python3 schemaguard.py user.v1.json user.v2-safe.json --contract-id users.v1
echo $?

 Contract ID : users.v1
 Gate Status : APPROVED
 Risk Score  : 2.0 / 100.0
 Summary     : 2 total changes, 0 breaking mutations
------------------------------------------------------------------------------
DETAILED MUTATION LEDGER:
 [SAFE]     [LOW     ] ENUM_VALUE_ADDED          at $.properties.plan.enum
            ↳ New enum variant 'enterprise' added
 [SAFE]     [LOW     ] FIELD_ADDED_OPTIONAL      at $.properties.locale
            ↳ Additive optional field 'locale' introduced

Exit code 0. Now the breaking candidate:

python3 schemaguard.py user.v1.json user.v2-break.json --contract-id users.v1
echo $?

 Contract ID : users.v1
 Gate Status : REJECTED_BLOCKED
 Risk Score  : 100.0 / 100.0
 Summary     : 4 total changes, 4 breaking mutations
------------------------------------------------------------------------------
DETAILED MUTATION LEDGER:
 [BREAKING] [CRITICAL] FIELD_ADDED_REQUIRED      at $.required[locale]
            ↳ Added new required field 'locale'
 [BREAKING] [CRITICAL] FIELD_TYPE_CHANGED        at $.properties.id.type
            ↳ Type mutated from 'string' to 'integer'
 [BREAKING] [CRITICAL] ENUM_VALUE_REMOVED        at $.properties.plan.enum
            ↳ Allowed enum variant 'team' was removed
 [BREAKING] [CRITICAL] FIELD_TYPE_CHANGED        at $.properties.tags.items.type
            ↳ Type mutated from 'string' to 'object'

Exit code 1. Every finding includes a JSONPath-style location, so you can point the PR author at the exact line. Nested objects and array items are compared recursively, which is how the tags change was caught.

Output for machines and for humans

--format json gives you something to parse:

{
  "schema_id": "users.v1",
  "analysis": {
    "total_mutations": 4,
    "breaking_mutations": 4,
    "risk_score": 100.0,
    "decision": "REJECTED_BLOCKED"
  },
  "mutations": [
    {
      "path": "$.required[locale]",
      "category": "FIELD_ADDED_REQUIRED",
      "severity": "CRITICAL",
      "description": "Added new required field 'locale'",
      "is_breaking": true
    }
  ]
}

I trimmed the arrays here. --format markdown --out report.md writes a table ready for a PR comment:

| Severity | Status | Category | Path | Description |
| :--- | :--- | :--- | :--- | :--- |
| CRITICAL | 🔴 BREAKING | `FIELD_ADDED_REQUIRED` | `$.required[locale]` | Added new required field 'locale' |
| CRITICAL | 🔴 BREAKING | `FIELD_TYPE_CHANGED` | `$.properties.id.type` | Type mutated from 'string' to 'integer' |

Wiring it into CI

The gate is the exit code: 0 for APPROVED, 1 for anything else. In CI, compare the schema on the PR branch with the version on main:

- name: Schema compatibility gate
  run: |
    git fetch origin main --depth=1
    git show origin/main:schemas/user.json > /tmp/user.base.json
    python3 tools/schemaguard.py /tmp/user.base.json schemas/user.json \
      --contract-id users --format markdown --out schemaguard.md

If the step fails, the job fails, and schemaguard.md is there to upload as an artifact or post as a comment. Loop over a folder of schemas if you have more than one contract.

How it works

The comparator walks both schemas together. At each node it checks type, then enum, then required, then properties (recursing into fields that exist in both), then items. The required rule tells a brand-new required field apart from an existing optional field that became required:

new_required = c_req - b_req
for field_name in new_required:
    if field_name not in b_node.get("properties", {}):
        self._record(path=f"{path}.required[{field_name}]",
                     category=ChangeCategory.FIELD_ADDED_REQUIRED,
                     severity=Severity.CRITICAL, desc=f"Added new required field '{field_name}'", ...)
    else:
        self._record(..., category=ChangeCategory.FIELD_BECOMING_REQUIRED, ...)

Each finding has a severity weight (CRITICAL 25, HIGH 15, MEDIUM 7.5, LOW 1). The risk score is the sum, capped at 100. The decision is REJECTED_BLOCKED if anything is breaking or the score is ≥ 50, NEEDS_REVIEW from 15, and APPROVED below that.

The report also prints an "Est. Blast" figure in hours and dollars. It's computed from a fixed formula driven by --consumers (default 5), --hourly-rate and --sla-penalty. Set them to your own numbers and treat the result as a way to make severity visible to non-engineers, not as a measured cost.

Limitations

• Only local $refs are followed. References such as #/$defs/Address or #/definitions/Node are resolved before comparing, recursive definitions included. References to other files or URLs aren't, so bundle multi-file schemas first (most JSON Schema tooling has a "bundle" step).
• It covers the keywords listed above (type, enum, required, properties, items). Constraint changes such as maxLength, pattern, minimum, additionalProperties or oneOf branches aren't classified.
• Adding an enum value is reported as safe. That's true for producers, but a consumer with an exhaustive switch may still need an update, so read the LOW findings.
• JSON input only. Convert YAML first, for example with yq -o=json.
• Terminal output always includes ANSI colours. In CI logs, prefer --format json or --format markdown.

FAQ

Is this for OpenAPI specs? It compares JSON Schema documents, such as event payloads, data contracts and request/response schemas you keep as files. It doesn't walk OpenAPI paths.

Does it need dependencies? No. Python 3 with the standard library is enough.

Can I call it from Python? Yes. run_schemaguard(baseline_dict, candidate_dict, contract_id=...) returns (report, passed).

SchemaGuard is a one-time purchase that includes the source and README: get SchemaGuard ($19). More details are on the SchemaGuard product page.