The previous post ended with a question: which of your zeros are just absences? This one is about what happened after we found ours.
Our research pipeline had a citation problem. A blind review of 150 generated reports found what three independent analyses converged on: references that did not match the text, method claims with no data behind them, and quality scores computed on a truncated prefix of each report.
We fixed it in one day, but not by making the model smarter. The fix was a pipeline: shadow audit, body-driven rebuild, then a gate. This is the recipe.
Step 0: Blind review before you fix anything
Two agents analyzed the same corpus independently — one implementation agent, one audit agent from a different harness, without seeing each other's conclusions. Both found the same three priority failures:
- Citation mismatch: the reference table was built from the source material list, not from what the text actually cited.
- Method dishonesty: reports claimed "survey research" when no survey had been conducted.
- Scoring on a prefix: the quality grader read only the first 6000 characters; references and conclusions were appended after that. Independent convergence is the cheapest signal you will ever get that the problem is real. Step 1: Shadow audit — record, do not block The first change was a shadow audit. It ran before every report was saved and computed two numbers:
- inbound rate: what fraction of in-text citations appear in the reference table?
- dirty-entry count: how many entries contain internal markers, section fragments, or pipe residue? It wrote to a JSONL file and changed nothing. On the existing 148-report batch, the inbound rate median was 0.32 — roughly one in three citations was broken. The point of shadow mode is calibration without risk: you learn the distribution before you set any threshold. Step 2: Fix the source, not the symptom The root cause of the mismatch was structural: the reference table was reconstructed from the input material list, not from the body text. The fix inverted the direction — at finalize time, the system now rebuilds the reference table from the citations that actually appear in the body, resolving each through the knowledge base for a canonical title. Replaying the same 148 reports: inbound rate median 0.32 → 1.00 reports rebuilt 142/148 (6 correctly fell back) dirty entries 120/326 → 8/326 The remaining 8 were legitimate long titles, not defects. Step 3: Gate — block ingestion, not delivery Only after shadow numbers were calibrated did we turn on a gate. The design choice matters: A failed citation check blocks the report from entering the knowledge base. It does not block delivery.
Rationale: a report with broken citations may still be useful to the requester; poisoned knowledge is not. The cost asymmetry runs in one direction. We set the threshold at 0.5 — below the 10th percentile of the repaired batch (0.857), high enough to catch structural failure without blocking honest edge cases.
Step 4: A dashboard that runs itself
The final step was a batch quality contract: five mechanical metrics computed after every batch, stored in a governance database, with thresholds calibrated from the pre-fix distribution.
Metric Before After
Citation reconciliation (median) 0.45 1.00
Dirty entries (P50) — 0
Fixed closing-sentence rate 97.3% 0%
Method honesty — 100%
Deep-source share 31.6% 58%
Blank-source share 74% 20%
No LLM in the metrics. No human in the loop for the counting. If the pipeline regresses, the dashboard catches it on the next batch.
What made it work
Three things, in order:
- Shadow before gate. We could not have chosen 0.5 without knowing the distribution. Blocking on a guess would have either frozen production or let everything through.
- Fix the mechanism, not the prompt. The reference table was wrong because of how it was built, not because the model was careless. Rebuilding from body citations is deterministic.
- Block ingestion, not delivery. This is the absent-zero principle applied to failure handling: a bad report is visible and correctable; a poisoned knowledge base is silent and compounding. The part that is reusable None of this is specific to our stack. Any agent pipeline that produces claims with citations can run the same sequence: blind review → shadow audit → source fix → calibrated gate → batch dashboard The three-state validity rule from the previous post does the same job at the measurement layer: absent evidence is unknown, not zero. This recipe does it at the production layer: a broken citation blocks knowledge, not the user. The trust layer that made the audit reproducible is at janzong/agent-lab-trust, and the validity-state specification is docs/validity-states.md. If your agents write anything that gets cited later, run the shadow audit first. The first number is usually worse than you expect.
Top comments (4)
The "block ingestion, not delivery" asymmetry is the design decision I'd underline for anyone copying this. A report with a broken citation can still help the human who asked; a poisoned knowledge base silently corrupts every future retrieval. Those costs run in opposite directions and most people gate on the wrong one.
One failure edge I hit building a similar body-driven citation checker: the inbound-rate metric assumes the source text is stable between generation and audit. It often isn't. If you resolve each in-text citation against a feed or a knowledge base that gets re-fetched, the canonical title can drift — and your substring match reports a false miss on a citation that's actually fine. A non-breaking space or a trailing fragment in the source title was enough to break my match until I switched to a normalized shorter-prefix comparison.
So a question: when you rebuild the reference table at finalize time, are you snapshotting the source material at generation time and resolving against that, or against the live knowledge base? The gap between those two is where my false positives lived.
Great question — and you're right that this is where the false positives live.
We resolve against the live knowledge base at finalize time, not a snapshot. The citation body text is treated as the anchor: whatever the writer actually cited in 《》 brackets gets resolved through the KB for a canonical title.
To absorb exactly the drift you describe, we don't use exact substring matching. The check is bidirectional inclusion with a minimum prefix of 4 characters: a match passes if either the body citation contains the KB title's first ≥4 chars, or the KB title contains the body citation's first ≥4 chars. This handles the two most common drift cases:
The numbers move together, but one thing about the metric needs flagging: after the rebuild, inbound rate is 1.00 by construction. If the reference table is regenerated from the in-text citations, every citation is in the table, so the median can't go below 1.00 unless the resolver fails. That makes it a check on the rebuild code, not on whether the reports are right. A citation can resolve to a real title in the knowledge base and still not support the sentence it's attached to.
The same applies to the gate threshold. 0.5 sits below the 10th percentile (0.857) of the repaired batch, but the repaired distribution is nearly all at 1.00, so the gate is calibrated on data where it can barely fire. The pre-fix batch (median 0.32) would have been blocked in more than half of cases, which is a better picture of what the gate is for. I'd calibrate it on the shadow-mode distribution and add a second, independent check: sample N reports and have a person (or a different pipeline) check that the cited source contains the claim, then report the support rate with an interval.
Small thing: the text says inbound median 0.32 and the table says 0.45 for the pre-fix citation reconciliation. Are those two different measures, or two different batches? Worth stating, because the before/after reads differently depending on which one is the baseline.
Question: what happens to the 6 of 148 that fell back? If they carry the old table, they're the ones where the gate matters most.
Thanks — this is the most careful read the post has gotten, and two of your four points are documentation errors on my side.
On "1.00 by construction": you're right, and it's intended. After the fix the reference table is generated from the body's citations, so inbound measures the rebuild code, not whether sources support claims. Our own internal table even annotates the post-fix number "by construction." The honest framing is: inbound is a structural consistency check (table ↔ body), which is necessary but not sufficient. A citation can resolve to a real title and still not support its sentence — we don't claim otherwise, and an independent support-rate sample (human or a second pipeline, reported with an interval) is exactly the right next layer.
On the 0.32 vs 0.45: same pre-fix batch, two versions of the metric. During the shadow audit we found 370/633 body citations were truncated forms of longer KB titles, so exact intersection systematically undercounted. 0.32 was the earlier exact-match number; after switching to bidirectional containment matching (≥4 chars), the same pre-fix batch re-measured at P50 0.45. Post-rebuild is 1.00. The article should have said this explicitly — good catch.
On gate calibration: there are two layers, and your critique applies to the coarse one. The per-report gate at 0.5 is deliberately a backstop — it blocks knowledge ingestion on structural failure, and on the repaired distribution it barely fires, as you note. The sensitive alarm is the per-batch contract: P10 reconciliation must stay ≥0.60, a threshold set between the pre-fix P10 (0.176) and the post-fix P10 (0.83–0.889), so a batch drifting back toward the pre-fix regime trips it even when individual reports pass. Calibrating the per-report blocker on the pre-fix median would permanently reject ~half of a structurally sound batch, which is why the pre-fix distribution defines "broken" while the post-fix distribution defines the new floor.
On the 6/148 fallbacks: they don't silently carry the old table. Those were low-density reports (<3 body citations) where replacing the material table would have been worse. The follow-up fix keeps the material table but merges every KB-resolvable body citation into it (numbering continued, no fabrication), and the citation gate applies to all reports regardless of which path produced the table.
The general point I take from your comment: a gate calibrated where it can barely fire invites complacency, and "resolves to a real title" is a much weaker property than "supports the claim." Both are worth separating explicitly in any citation contract.