This is article 2 in a series about building PlannerCritic, an open-source engine where one LLM writes a plan and a second LLM reviews it. Article 1 is here — it covers the 157-goal field test and what it found. This one is about a specific bug that taught me something about LLM judgment.
The critic was supposed to catch unsafe plans. Instead it blocked plans for being incomplete. The fix was a frozenset.
What Happened
I gave the critic this system prompt:
You are an adversarial plan reviewer.
Six words. Seemed fine at the time.
After running 16 strict goals, every single one escalated. Not because the plans were unsafe. Because the critic was blocking on completeness concerns — "this plan could also cover edge case X" — instead of concrete defects.
The annoying part: the critic was technically doing its job. I told it to be adversarial. So it found something to object to in every plan. The problem was I didn't specify what kind of adversarial.
The product intent for strict mode was: "no concrete safety, ordering, or rollback violations." The critic was enforcing: "no reviewer concerns whatsoever." Those are different contracts, and I didn't notice until 16 goals had failed for the wrong reason.
The Real Examples
Here is what the critic flagged as a blocker before the fix:
A plan for cross-account VPC peering got blocked because the critic said the rollback plan "could also consider" DNS failback scenarios. That is a completeness suggestion, not a safety defect. The plan had rollback. It just didn't cover every hypothetical.
A plan for embedding index migration got blocked because the critic flagged a generic "risk" finding about potential latency during cutover. That is risk commentary, not a structural defect.
Both of these should have been warnings. The engine should have approved the plan with acknowledged risk. Instead, it escalated to a human.
The Fix
Two parts.
Prompt change: Replaced "adversarial plan reviewer" with "plan reviewer" and added explicit severity rules:
SEVERITY RULES — blocker is reserved ONLY for concrete, plan-local defects
that make the plan unsafe to execute:
blocker = unsafe_sequencing, weak_rollback, unverified_dependencies, feasibility
warning = risk, missing_steps
info = minor observations
Do NOT escalate completeness or thoroughness concerns to blocker.
Code guardrail:
_BLOCKER_ELIGIBLE_FAMILIES = frozenset({
"unsafe_sequencing",
"weak_rollback",
"unverified_dependencies",
"feasibility",
})
if severity == Severity.BLOCKER and item.heuristic_family not in _BLOCKER_ELIGIBLE_FAMILIES:
severity = Severity.WARNING
Even if the LLM returns blocker for a risk or missing_steps finding, the code downgrades it before it enters the findings list.
What Changed After the Fix
Before: every strict goal failed because the critic was blocking on advisory concerns.
After: zero advisory findings appeared as blockers across 92 post-fix runs. All 132 blockers that fired belonged to concrete defect families or deterministic gates.
The balanced goals still approved. The strict goals still escalated. But the reason changed. They escalated because the plans had real structural defects, not because the critic was being thorough.
Why the Prompt Alone Wasn't Enough
I tried fixing it with prompt engineering first. I added "only block on concrete defects" to the system prompt. The critic still escalated completeness concerns to blocker about 30% of the time.
That's the thing about LLMs. They don't have a stable concept of "this is bad enough to stop." It depends on the model, the temperature, and how the plan is worded. You can ask nicely. The model will still disagree with you when it feels like being thorough.
The code guardrail is deterministic. It does not depend on the LLM behaving correctly. That is why it works.
The prompt is helpful. The guardrail is the contract.
What I'd Do Differently
Start with the severity taxonomy, not the persona. Define what counts as a blocker in code. Write the prompt to match. Test with real data. The unit tests passed for me because they used hand-crafted inputs. The field test caught the bug because it used a real LLM producing real plans.
The research backs this up. The self-correction literature shows LLMs are unreliable at evaluating their own output — the "self-correction blind spot" affects ~64.5% of models (arXiv 2507.02778). If the model can't reliably evaluate correctness, it can't be trusted to decide what is fatal.
Article 2 of 5 in the PlannerCritic series.
Series: Article 1: "I Ran 157 Agent Plans Against a Real LLM" · Article 3: "The Planner Made the Same 3 Mistakes" · Article 4: "The Field Test Found 10 Issues" · Article 5: "I Tried to Prompt-Inject My Own Engine"
Links:
- Repo: github.com/deghosal-2026/planner-critic-engine
- README: README.md
-
PyPI:
pip install planner-critic - Field Test Results: 157 goals across 35 domains
- Field Test Plan: 156-goal corpus
- Architecture: architecture-v0.1.0.md
- User Guide: quickstart.md
- CHANGELOG: CHANGELOG.md
Top comments (26)
This is a good separation between probabilistic classification and deterministic consequence. The model can propose a finding, but code decides whether that finding is allowed to stop execution.
One failure mode I’d test next is label migration. The allowlist prevents
missing_stepsfrom becoming a blocker, but the model may describe the same concern asunsafe_sequencing, which is blocker-eligible. The guardrail still trusts the model to choose the family correctly.I’d keep the raw finding and normalized family in the evaluation data, then build boundary cases that differ by one fact: optional step versus required dependency, possible latency versus unsafe ordering, rollback improvement versus no viable rollback. A confusion matrix around those boundaries would show whether the guard removed the bad decision or merely moved it into classification.
For the highest-impact cases, a deterministic invariant such as “all irreversible steps have a verified predecessor and rollback condition” is stronger than accepting any model-selected blocker label.
This is a valuable extension, and it names exactly the next weakest seam in the design. The reason I kept model findings in the first place was to avoid pretending the model is deterministic; but if the label it chooses is what decides consequence, then I've just loaded the door decision back into the model. Separating the raw finding from the normalized decision that is allowed to hold authority is exactly the honesty I need.
On the boundary-case idea you lay out, differing by one fact, that's a really productive test to structure. If the guard only changes the classification while the decision still maps to the model, then nothing has been gained. I'll adopt the confusion-matrix framing in the next pass, and I'll keep both the raw finding and the normalized family in the eval data so the migration shows up as data instead of silently.
Counter-question: in your experience, has the label migration shown up mostly around a single fact boundary, or as a general drift? I'd genuinely like to know where you'd put the first boundary case. Thanks for the read, this is exactly the kind of review that moves the design forward.
I’d expect both, but I’d look for single-fact boundary failures first because they are easier to diagnose. General drift often appears later as the prompt, model or surrounding context changes.
Rollback is probably the cleanest first fixture. Give the critic two otherwise identical plans:
The first is a completeness warning. The second is a concrete execution-safety defect. If both become
weak_rollback, or the first migrates intounsafe_sequencing, the classifier is absorbing the consequence decision.I’d run each pair repeatedly and retain the explanation alongside both labels. That reveals not only label variance, but whether the model is changing the underlying fact it claims to have observed. The latter is more dangerous because a normalization layer cannot repair invented evidence.
Thank you — the rollback boundary twin you described already exists: generate_boundary_cases() in label_migration.py, shipped with #171 in v0.2.0, builds exactly the executable-but-imperfect versus named-but-nonexistent pair. What we had not built is your repeated-trial protocol — running each pair against a live critic multiple times while retaining explanations to separate label variance from invented evidence. That is now tracked: Live-critic boundary-case runner. Your point that a normalization layer cannot repair fabricated facts became an explicit evidence-drift metric in the design. Appreciate the precision here.
That is a solid implementation. One detail I’d preserve in the runner is the complete decision context for every trial: model version, temperature, system prompt hash, tool-schema hash, normalized finding, and the evidence cited in the explanation.
Otherwise a changed label can look like stochastic variance when the real cause was a prompt or tool-definition change between runs. I’d also score evidence validity separately from label stability. A critic that chooses the same label ten times while repeatedly citing a nonexistent function is stable, but still unsafe.
The useful output is probably two rates: decision disagreement across identical trials, and unsupported-evidence frequency. They expose different failure modes.
Both gaps conceded — they are real in the runner as shipped. Trial records today store verdicts and explanation text only, so a label shift caused by a prompt or tool-definition change between runs is indistinguishable from stochastic variance; and the evidence metric measures cross-trial consistency, which means a critic citing the same nonexistent function ten times scores perfectly stable and perfectly undetected. Your stable-but-still-unsafe case is the sharpest formulation of the blind spot.
Filed as decision-context capture plus unsupported-evidence frequency (#242): every trial records model id and version, temperature, system-prompt hash, tool-schema hash; explanations get their claimed facts extracted and validated against the boundary plans, whose ground truth is fully known; and the report leads with your two rates — decision disagreement across identical trials, and unsupported-evidence frequency — as separate axes. Thank you again; both rounds of this thread improved the harness materially.
One thing worth pinning down before this ships: where do model id/version, prompt hash, and tool-schema hash get captured from? If they're read off anything the critic itself reports, a critic that's already drifted (wrong tool binding, stale prompt) can misreport the very metadata meant to explain the drift, and now decision-context capture inherits the same self-report problem the eval was built to avoid. Pull those four values from the harness's own call parameters — the request you sent, not anything in the response — so the record stays a source of truth independent of what the critic says about itself.
This is a good bug because the code fix admits the prompt will drift. I like the severity whitelist more than the wording change. Any reviewer prompt can learn to sound stricter over time; the downgrade rule gives you one boring place where the product contract wins.
I think you've nailed the real point, which is to stop fighting prompt drift by trying to freeze the wording, and instead to invert it. The system admits that whatever the critic prompt becomes over time, there's going to be a fixed place where the product contract wins: a list that says which finding becomes a stop and which one is only advisory. That's a more durable separation than trying to get the model never to sound over-strict.
Where I'd keep digging: keep the raw finding and the normalized severity in the event data, always, so drift is visible instead of silent. That way the missed mapping shows up as data instead of as a surprise. Thanks for the read, it's validating, and "one boring place where the product contract wins" is a line I'm going to steal. Appreciate the time.
The six-word prompt is a perfect case of a spec that seems obvious but isn't. Completeness objections vs safety objections look the same inside the critic - you never told it which one counts as a blocker. I've hit similar drift with a "strict" flag that ended up meaning something different in each pipeline stage. Did you consider a warning-mode path where completeness concerns surface but don't block, so the human reviewer still sees them?
Tae, yes — and the warning-mode path is exactly how the critic layer works today after the fix. The _BLOCKER_ELIGIBLE_FAMILIES frozenset means every critic finding lands as a warning by default; only findings in the concrete defect families (unsafe_sequencing, weak_rollback, unverified_dependencies, feasibility) can escalate to blocker. Completeness concerns stay visible as warnings in the escalation payload — the human still sees them — but they can never block the pipeline alone.
The six-word prompt was a good lesson in how a persona instruction is not a severity taxonomy. We've since made the severity an API contract enforced in code, not a sentiment the prompt negotiates (github.com/deghosal-2026/planner-c... tracks the red-team harness for this boundary).
On the drift problem — the notable part of this design is that the blocker/warning split lives in code, not in a "strict" flag. The six gates are hard blockers (deterministic, injection-immune — the LLM never votes on them), and the critic's findings are advisory by default: only families in _BLOCKER_ELIGIBLE_FAMILIES can escalate to blocker. So a warning-mode path isn't an add-on here — it's the critic layer's default lane, and the taxonomy can't drift between stages because it's a frozenset at the boundary, not prompt text.
The seam I'd watch is the warning lane's coverage: the gates are a fixed list, so any under-claim defect class they don't enumerate (e.g. a rollback that's present but unreachable — runs after the failure point, or depends on the resource it destroys) lands in the advisory lane no matter how strict the taxonomy is. That's where a fixture pair pinning "verifies-before-mutate vs verifies-after-mutate" would earn its keep.
Worth noting what the frozenset guardrail does not do: it is one-directional. It can only downgrade an over-claim (blocker -> warning); it can never upgrade an under-claim. A real structural defect that the critic happens to label 'risk' or 'missing_steps' passes straight through as advisory and the plan proceeds. In safety terms the under-claim is the worse failure: an over-claim costs a human a false escalation, an under-claim lets an unsafe plan run. Pairing the frozenset with a small structural pre-pass that runs before the critic would cover that direction - deterministic checks like 'does the plan contain a rollback section at all', 'do irreversible steps name a rollback condition'. If one fails, that is a blocker the severity taxonomy never gets to vote on. Then the frozenset owns the over-claim direction, the pre-pass owns the under-claim direction, and both stay code rather than prompt.
Great observation — and you're absolutely right, the guardrail is one-directional by design. But the good news is: the engine already has exactly the pre-pass you're describing.
The six deterministic gates (schema_valid, no_dep_cycles, ordering_sane, verification_present, rollback_present, preconditions_referenced (github.com/deghosal-2026/planner-c...) run before the LLM critic and are injection-immune — the LLM never gets to vote on them. If rollback_present finds a high-blast-radius step with no rollback, that's a blocker regardless of what the critic labels it. The frozenset guardrail only applies to the critic's own findings; the deterministic gates own the under-claim direction independently.
So the architecture is exactly what you described:
Both stay code, not prompt. The field test (github.com/deghosal-2026/planner-c...) verified this across 65 goals — zero gate false negatives, zero blocker noise from the critic. The injection-immunity regression corpus (github.com/deghosal-2026/planner-c...) (coming in v0.2.0) will back this with real-CVE-grounded test cases.
Good to know the six gates are exactly that pre-pass - the two directions do split cleanly. One seam remains: the gates are a fixed list, so the under-claim direction is only as strong as its coverage. rollback_present catches 'no rollback section', but does it catch a rollback that is unreachable (runs after the failure point, or depends on the resource the plan destroys)? And does ordering_sane cover verification-before-irreversible, or only dependency ordering? If verification can pass yet run after the mutate step, a critic mislabeling that defect as 'risk' reopens the under-claim direction at the seam between gate list and taxonomy. A fixture pair - verifies-before-mutate vs verifies-after-mutate - would pin that down.
Both seams confirmed — I checked the gates directly before responding. rollback_present fires only when a rollback section is absent, and ordering_sane only validates hard-dependency positions, so an unreachable rollback and a verification that runs after its result was consumed both pass clean today. Filed two issues: Rollback credibility gate covering unreachable, self-dependent, inconsistent-state, and post-consumed rollback patterns, and Verification-before-mutate ordering gate including the verifies-before-mutate versus verifies-after-mutate fixture pair you specified, wired into the label-migration harness. Together they close the under-claim direction at the seam between the fixed gate list and the severity taxonomy. Thanks for naming the exact fixtures — they went straight into the acceptance criteria.
The fixture pair went in — one addition that will save #219 from its own false-positive class: position alone can't be the predicate, because legitimate verification exists on both sides of the mutation. A check that reads pre-mutation state or constraints (the rollback precondition, the invariant before the write) must run before the mutate; a check that reads the mutation's output must run after it, or it verifies nothing. If the gate flags "after" unconditionally, that second class becomes a permanent false positive, and the natural fix under pressure is to move verification earlier — which silently turns output checks into pre-state checks. That's the same vacuous-verification drift #219 is meant to kill, just wearing the gate as its cover.
Keying the gate on the verification's data subject (what it reads) instead of its position handles both directions: subject = pre-state/constraint → require before; subject = mutation output → require after, and require it to actually consume the output (a reading-the-output assertion, same spirit as the rollback gate's reachability check). The fixture pair is really a triplet: verifies-before-mutate (must pass), verifies-after-mutate-on-pre-state (must fail), verifies-after-mutate-on-output (must pass). Third member is the one that stops the false-positive class from training the gate into submission.
Shipped exactly as you specified. The verification-ordering gate now keys on data subject rather than position: checks reading pre-mutation state or constraints must run before the mutate and are legitimate there; checks reading the mutation output must run after it and must actually consume that output. Position alone flags nothing (#230, completing #219). The fixture is your triplet — verifies-before-mutate passes, verifies-after-mutate-on-pre-state fails, verifies-after-mutate-on-output passes — with the third member doing exactly the job you named: stopping the false-positive class from training the gate into moving every check early. Interim subject derivation uses a prose prefix convention; a typed optional subject field replaces it in the 0.3.0 schema evolution. Thanks for refusing to let the fixture ship as a pair.
This is a great example of why AI systems need boundaries beyond the prompt itself. An LLM can be excellent at identifying potential issues, but deciding which issues should actually block a workflow is a different problem. In the kind of AI automation work we explore at IT Path Solutions, this distinction is especially important, reasoning can stay probabilistic while critical enforcement remains deterministic.
Glen, that is the exact design principle the engine now follows. The LLM critic is excellent at surfacing issues — it found 346 findings across the field test. But the decision about which issues are allowed to stop execution belongs to code, not the model. The deterministic gates (schema, cycles, ordering, rollback, preconditions, parallel safety) run before the critic and are injection-immune — the model never votes on them. The frozenset guardrail then constrains which critic families can escalate to blocker. Probabilistic identification, deterministic consequence.
The next step is making the under-claim direction observable too — tracking when the critic mislabels a structural defect as an advisory family, so that direction is visible as data rather than silent (github.com/deghosal-2026/planner-c..., planned for 0.2.0).
Hi Debashish, I've run into this same problem. I have created a quality control and a claude.md checker subagent to make sure that the plan is up to my standards and follows my Claude.md file. I found that when I tell claude to review a plan, code review, etc... it will always find at least one problem. Like you said, it's usually a completeness concern rather than a concrete example.
How did you come up with the _BLOCKER_ELIGIBLE_FAMILIES set, and is it possible to extend the set to include business logic cases? The blockers that my agent finds is usually related to business logic that is not possible.
Hi Shayan, thanks for reading and sharing — great to hear someone else hit the same wall!
The _BLOCKER_ELIGIBLE_FAMILIES set came directly from the 0.1.0 field test. I classified 346 findings by heuristic family and noticed: families like unsafe_sequencing, weak_rollback, unverified_dependencies, and feasibility produce structurally verifiable defects — you can check them in code. Families like risk and missing_steps produce advisory noise — "this seems risky" isn't actionable.
So the guardrail is simple: if the critic says "blocker" but the family is advisory, downgrade to warning. Only code-verifiable families earn blocker status. This cut blocker noise to 0% across all findings (field test results (github.com/deghosal-2026/planner-c...) · finding-quality audit test (github.com/deghosal-2026/planner-c...).
Yes, you can extend it for business logic — and that's exactly what the v0.2.0 Domain Pack framework (#139 (github.com/deghosal-2026/planner-c...) is for. The principle: add a new family to the blocker-eligible set only if you can write a deterministic gate for it. Your claudemd checker sounds like a perfect candidate — "does the plan reference every mandatory section?" is structural, not a judgment call.
If the business rule can be expressed as code (or an OPA/Rego policy — #129 (github.com/deghosal-2026/planner-c...), it earns blocker status. If it can't, keep it as a warning — the critic's opinion, not a hard gate. Full details in the field test plan (github.com/deghosal-2026/planner-c...) and test suite (github.com/deghosal-2026/planner-c...).
The frozenset downgrade is the durable part of this fix. Prompt severity rules will drift as models update, the code guardrail won't. I ran into the same pattern with an adversarial reviewer on infra changes: left unconstrained it optimizes for having an objection, and 'could also consider X' is the cheapest objection to generate. One thing I'd add: log every blocker-to-warning downgrade as a metric. If that rate climbs after a model swap, it's an early signal of judge drift before escalation counts blow up. Did you end up tracking downgrades, or just the post-fix blocker counts?
Great instinct — and yes, downgrades are tracked as of v0.2.0. drift.py records raw_severity versus normalized_severity for every critic finding, computes downgrade_rate with a per-family breakdown, and check_drift_alert() raises a z-score alert when a family's drift rate spikes over a trailing window — precisely the early judge-drift signal you describe after a model swap. It tracks critical_underclaims separately too, since a blocker demoted inside risk or missing_steps is the direction that actually hurts. You're right that post-fix blocker counts alone would never have caught it. Thanks for pushing the metric framing.
The deterministic guardrail is the right instinct and I'd generalize it past severity labels. We had a version of this with a hallucination judge: the prompt said something like 'flag anything unsupported' and it started flagging paraphrase and citation style as unsupported, not just factual errors, so our false positive rate sat around 30 percent for weeks before customer complaints made us look. The judge prompt was vague, so we rewrote it with concrete examples of what counted as unsupported versus what didn't. That took false positives from 30 percent down to 4. Your frozenset check is a stronger version of that because it doesn't rely on the model reading the examples correctly every time, it just refuses to let a disallowed severity out the door. One thing I'd want to see on your 132 post-fix blockers: did you sample any of them to check the code guardrail never downgraded something that actually deserved to block? A deterministic filter on top of a judge label is only as safe as the taxonomy underneath it, and that's usually where these things quietly go wrong.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.