Which phrases make a paragraph read like AI? I built a heatmap and measured it on HC3

I wanted to see which exact phrases make a paragraph read like a language model wrote it, rather than get one opaque score. So I built AiSniff.

What it does

You paste English or Traditional Chinese text. AiSniff underlines the phrases that match a documented AI writing pattern. Hover one and you see the rule, a public source and a plain rewrite hint. A side panel groups the hits by type, a meter shows hits per 1,000 words or characters, and a compare view puts a before and after text side by side. You can export the marked text as a PNG.

It runs fully in the browser. There is no server and no model call, so nothing you paste leaves the page. The same JSON rule packs also ship as a command line tool and an agent skill, so you can run npx github:Arthur031221/AiSniff lint draft.md in CI and fail a build when the density is too high.

The rules

There are 61 rules in versioned JSON packs for English and Traditional Chinese. Every rule has a pattern, an explanation, a source citation and examples that should and should not match. The examples run as unit tests, so a rule cannot be added without showing a legitimate use of the same words. Sources are the Wikipedia guide on signs of AI writing, three public pattern lists, the Chinese Wikipedia page on Europeanized Chinese and a peer reviewed study of excess vocabulary.

How well does it work

I measured it on a held out half of HC3, a public set of questions with human and ChatGPT answers. At least one English rule fires on 28% of ChatGPT answers and 2% of human answers. For Chinese it is 24% and 6%.

That is a weak detector, and I would not use the density as proof of anything. The AUROC is 0.63 for English and 0.59 for Chinese, and 72% of the English ChatGPT answers have no hit at all. The data comes from an old model, in a question and answer register, and the Chinese half is Simplified only, so the rules run through a character by character conversion. Some rules, such as the markdown bold rule, did not hold up on HC3, and the evaluation page lists them.

So AiSniff reports patterns, not authorship. A clean result does not prove a person wrote the text, and a hit does not prove a model did.

Try it

The repo is at https://github.com/Arthur031221/AiSniff and the live demo is at https://arthur031221.github.io/AiSniff/. It is MIT licensed. I built it with help from Claude Code and Codex. If a rule misfires on your own writing, please open an issue with the sentence.