A report said "41 mentions of the token in 6 channels." It was true, reproducible, and useless — because three days later nobody (including me) could answer: which messages, captured when, by which run? The number was right and the report was trash. That's the week I started treating every published figure as a verdict with a chain of custody.
Court evidence has a rule: an artifact is only as trustworthy as the documented path from collection to presentation — who handled it, when, from where, unaltered. Public-data reporting has exactly the same problem and almost nobody applies the discipline. Your reader has no way to know whether "41 mentions" came from a scripted sweep at 07:00 today or a manual glance at a phone last Tuesday.
The provenance triple
Every figure in a data report should carry three things, cheaply, in the artifact itself:
- Source identity — not "Telegram" but the exact channel, URL, and the endpoint/params used to collect it. If two aggregators disagree, this is the field that lets someone arbitrate.
- Collection window — the interval the data covers, and the collection timestamp separately. "Mentions from Oct 10–12, collected Oct 13 07:00" and "Oct 10–13" are different claims; collapsing them is how reports start lying by rounding.
- Pointer granularity — a link from the figure down to its rows. For 41 mentions: 41 permalinks, a CSV, a dump — anything that lets a skeptic walk the number back to primary evidence in under a minute. If the pointer is "trust me, it's a lot," the number is decoration.
The test that keeps me honest: can a hostile reader reproduce this figure without contacting me? If yes, it's evidence. If no, it's a rumor with good formatting — and on the public internet, rumors with good formatting have a half-life of one screenshot.
The layers where custody breaks
- Dedup, silently. Sweep the same event from 6 channels and a naive count reports 6 mentions of one thing. Provenance answer: dedup keys recorded, dedup report shipped alongside the count — the count and its reduction ratio travel together ("1,271 raw → 41 canonical" is more informative than "41").
- Derived numbers with no parents. "Tone shifted positive" derives from mentions; mentions derive from messages; messages from a collection run. Each derivation needs its input named, or one drift at the bottom silently rewrites the top. When your summary layer cites a stat, it cites the intermediate artifact, not the raw firehose — that's the chain, and every link is a file, not a memory.
-
Timestamp laundering. Re-running a pipeline "refreshes" timestamps and makes old collection look current. Two columns (source-confirmed
observed_atvsingested_at) and a rule: published figures quoteobserved_at. Chains of custody die when the log book gets rewritten. - The deletion problem. Public sources vanish (domains expire, channels go private). A permalink that 404s six months later turns your citation into a claim of your own good faith. Defense: raw snapshots (save what you fetched), and in long-lived documents, quote the material span in-text, not just link it — a quotation is a snapshot with human-readable hashing.
Why this pays (it's not just hygiene)
Provenance is the only property of public data that compounds into trust, and trust is what converts a free sample into a paid pack. When a reader can walk one number back to its source in a minute, they stop evaluating your conclusions and start relying on your pipeline — which is the product. Every paid deliverable I ship opens with a provenance header (window, run id, per-source table with permalinks); it costs about 200 bytes, and it is the single most-consulted part of the artifact by the people who pay.
The closing rule is the one they teach in evidence law and skip in data work: a figure without a chain of custody is inadmissible. Publish the chain with the number, and let skeptics do their worst.
Part of a series on building trustworthy data pipelines from public sources. Previous: Public Data Has No SLA — what happens to the chain when the sources die.
The chain of custody is the product: every row in my shop bundles carries source, capture time, and run ID: free Telegram OSINT sample (pay what you want) · tg-crawl Pro: the scraper wheel + daily-crawl recipes. The documentation half of the same discipline: Your Dataset's README Is Part of the Data.
Top comments (2)
You've put your finger on exactly the right seam, and I'll concede the ground before defending the rest: chain of custody proves provenance, not veracity. It records who touched the bytes and when — it never vouches that the bytes were true at any hop. Paperwork can be immaculate over a lie, and the two failure modes in your step-two example are distinct: a corruption (tz shift, schema drift, silent coercion) that the chain can locate, versus a false source that enters cleanly at step one and rides a perfect chain all the way out — that one the paperwork structurally cannot catch, no matter how pristine.
Where the chain earns its keep is bisectability: with immutable raw snapshots at every hop, when someone finally notices step two was wrong, you can name the hop, recompute everything downstream, and ship a correction with receipts. Without the chain, "the number was wrong somewhere" is unbounded — you re-audit the whole pipeline by hand, which is to say: never.
So I run the custody chain and the hostile-reader test as separate controls: custody audits the process, hostility audits the content. Your scenario is precisely why one without the other is theater.
the hostile reader test is the sharpest line in this. my only worry is that a chain of custody can be perfectly documented and still be wrong at step two, and nobody notices because the paperwork looks pristine.