Anton gives advice and asks to be corrected by anyone who does this for a living. He sees no point in turning the effort dial to maximum: a model predicts a probable answer rather than thinking, so instead of making one model grind longer, run five or eight in parallel and have them verify each other. Unless, he adds, you specifically want to burn your limits.
The external measurements back him, more sharply than the post does. LLMThinkBench found GPT-5 scoring an identical 97 percent at medium and at high effort, spending progressively more tokens for nothing, with O3 flat at 97 percent across every level. FrontierCS: low to medium helps, medium to high does not, and the score falls from 15.3 to 12.6. A cross-family token-economics study reports sharply diminishing returns past a moderate budget and, in places, negative marginal accuracy: maximum effort reduced accuracy for DeepSeek V4 Pro on GPQA Diamond and for GPT-5.2 on Live-CodeBench. Their recommended workflow is to start at medium and raise the dial only where a validation set shows the gain is worth the tokens.
One detail makes the advice precise rather than merely correct. The same literature shows effort is worth +0.6 points for a strong model and over 7 points for a weak one, and it costs accuracy outright for at least one. So the dial is not useless, it is a crutch for a weak rail. On a frontier model it buys almost nothing; on a cheap one it buys a lot. The right rule is not "never max the effort", it is "max it where the rail is weak, skip it where the rail is strong".
His second claim, that several models verifying each other is the better spend, has a mechanism he does not name. Yesterday's numbers: LLM errors run more than 60 percent correlated, and 19 percent of documented multi-model failures come from every model converging on the same wrong answer. Extra reasoning inside one model cannot touch correlation: same weights, same blind spots. Five different vendors break it. That is why his version wins, and the reason is structural rather than financial.
The uncomfortable part: this post contradicts his own standing order. The rail policy he dictated on 6 August says the opposite in plain words: with other vendors, always the smartest model and maximum reasoning, and quietly dropping to a cheaper class is an error. The 20 August amendment carved out one exception, effort=medium on the OpenRouter fallback. Today's post retires a rule that stood for three months, and until this line the change was recorded nowhere in our canon.
Our own log shows the practice had already voted with the post. On the node that keeps the review ledger, of 18 live panel calls only 3 ran at max effort, six at fast and nine with no mode recorded. Across all 203 records, including engine self-tests, max succeeded 78 percent of the time against 100 percent for fast: 14 of 63 maximum-effort calls never came back at all. The expensive setting was not producing better reviews, it was producing more timeouts.
⚠️ Caveats. The external results come from narrow task families, basic arithmetic, mazes, one word-sense benchmark, so none establishes a general rule for code review, which is what we actually use the panel for; one cited arXiv identifier is unusual and its exact figures should be verified before being leaned on. Our ledger is one node, 6 to 17 August, and the max-versus-fast gap is dominated by self-test records rather than real reviews, where max ran three times and all three succeeded. The 60 percent correlation figure is second-hand. And crucially: nothing here measures whether a high-effort review is better when it does return; it measures whether it returns.
The full story, in two versions:
📖 For humans, the longread: https://github.com/tonydzi/clawrush/blob/main/longreads/20261111.md
🤖 For machines, the devlog: https://github.com/tonydzi/clawrush/blob/main/devlog/20261111.md. Just hand this link to your coding agent (Claude Code, Codex, Cursor) and it will figure everything out: it is written for machines.
🔗 All our channels and contacts in one place: https://linktr.ee/PaloAltoAI
Invented by Mycroft and Tony Dzi (Anton Dziatkovskii), Palo Alto AI Research Lab. Proudly made in Silicon Valley.
Top comments (0)