TwIL-8B vs Qwen3.5-9B: The 81.6 Code Score Isn't the GGUF You Download

An 8B model that scores 98.2 on MATH-500 and 81.6 on LiveCodeBench, and fits in a 4.98 GiB GGUF. That's the pitch for TwIL-8B, webAI's post-train of IBM's Granite 4.2 8B (8.8B parameters, per the card), and on paper it beats Qwen3.5-9B at both.

I read the model card twice before writing this, and most of what follows came out of the second read. The LiveCodeBench number comes from a different variant than the one you download by default. GPQA Diamond drops more than 11 points behind Qwen. Every TwIL score is webAI's own run. And the license on the new weights says non-commercial, even though the base model is Apache 2.0.

Below: how TwIL-8B compares to Qwen3.5-9B, how to run it locally with llama.cpp, and the one setting that wrecks your results if you leave it at the default.

TwIL-8B vs Qwen3.5-9B: the numbers webAI published

These are straight from the TwIL-8B model card. The card says competitor scores "are published figures and were not re-run." Rows marked † were measured on the verbose variant and, per the card, "not yet run on balanced."

That's 7 wins to 6 across 13 shared benchmarks, which matches the count in explainx's breakdown. So it's close to even overall, with two big gaps pulling in opposite directions.

The LiveCodeBench win is a verbose-variant number

TwIL-8B ships in two flavors. Balanced is the default, with shorter reasoning traces. Verbose thinks longer and is meant for hard contest problems.

The 16-point LiveCodeBench lead over Qwen (81.6 vs 65.6) is a † row, so it's verbose. The MATH-500 and GPQA numbers are balanced. If you pull the default TwIL-8B-Q4_K_M-balanced.gguf and expect 81.6 on code, nobody has published a number that says you'll get it.

There's more fine print on that row. The card notes competitor LiveCodeBench scores are v6, while the version and date window of TwIL-8B's run "are not recorded." LiveCodeBench is on the 13-gram decontamination list, but that list was applied to the math, STEM and instruction-following training pools, and "decontamination of the code training data is not recorded." HumanEval, MBPP and CRUXEval weren't in the decontamination set at all. LiveCodeBench exists to fight contamination with date windows, so not recording the window takes away the one thing that makes the benchmark trustworthy.

I don't think anyone's hiding anything. The card is unusually honest about all of this. But the headline number sits in the intro and the caveat sits in the limitations section, and only one of them fits in a tweet.

GPQA Diamond: 70.2 vs 81.7

This is the gap I'd care about if I wanted a general assistant. GPQA Diamond is graduate-level science Q&A, and TwIL trails Qwen3.5-9B by 11.5 points. Gemma 4 12B gets 78.8 on the same card. MMLU-Pro and HLE point the same direction.

The card says it plainly: "It is not a knowledge model." Math and code RL sharpened reasoning on problems with checkable answers, but RL can't add facts the model doesn't have. If your workload is "explain this chemistry paper" or "what does this obscure API do," Qwen is the better 9B by a wide margin.

Other caveats worth knowing

• Every TwIL score is vendor-run, measured on bf16 weights, not on the GGUFs. The card expects "small deviations, most likely at Q4_K_M." The explainx piece found "no independent reproduction" as of November 4, 2026.
• AIME and HMMT sets have 30 to 33 problems. One problem is about 3 points, so TwIL's 1.8-point HMMT Feb 2025 win is less than a single question.
• HMMT Nov 2025 (TwIL's +3.8 math win) was not in the decontamination set.
• Competitor numbers come from different harnesses, prompts and token budgets.
• SWE-bench, tool use and agentic tasks were not evaluated, and there's no tool-use training beyond what Granite 4.2 already has. Don't drop this into an agent loop and expect Qwen-level tool calling without testing it.

How to run TwIL-8B locally with llama.cpp

webAI publishes four GGUFs in the main repo:

Note that only the balanced Q4_K_M is listed with an importance matrix. If you want the verbose variant above 4-bit, Q8_0 at 8.70 GiB is the option.

The command from the card:

llama-cli -hf webAI-Official/TwIL-8B --hf-file TwIL-8B-Q4_K_M-balanced.gguf -cnv -c 40960 -n 32768

You need --hf-file because both variants exist at Q4_K_M, and a bare :Q4_K_M tag doesn't tell llama.cpp which one you want. To chase the LiveCodeBench number, swap in TwIL-8B-Q4_K_M-verbose.gguf.

The template is ChatML-style with <|im_end|> as EOS, and it opens a <think> block by default.

Don't skimp on max tokens

The card's example uses max_new_tokens=32768, which is why the llama.cpp command above sets -n 32768 with a 40960 context. Balanced averages about 19.5k output tokens per AIME 2026 problem and about 33k per HMMT Feb 2026 problem. The card warns that short budgets truncate reasoning and score much worse.

Check your frontend's default output cap. If it's a few thousand tokens, TwIL-8B will often run out mid-thought and give you nothing usable. Set 32k or don't bother comparing it to anything.

Same warning for the shortcuts: enable_thinking=False and reasoning_effort="low" cut latency, but per the card "low-effort runs score substantially lower," and none of the published numbers use them. The shipped generation_config.json uses temperature 1.0 and top_p 0.95; keep those unless you're testing something specific.

Budget for the time, too. TwIL is actually cheaper than Qwen here: the card has Qwen3.5-9B at 31.2k tokens per AIME 2026 problem, so TwIL uses 38% fewer. But GPT-5.5 does it in about 5.0k. On a laptop running a 4-bit 8B, 20k tokens per answer means you go make coffee while it thinks.

The license is the bigger catch

The base model, Granite 4.2 8B, is Apache 2.0, and IBM's license text sits in the repo as apache-2.0-LICENSE.txt. TwIL-8B's weights ship under the webAI Non-Commercial License ver. 1.0, which defines non-commercial as "for non-commercial research and educational purposes only." Derivatives inherit that limit.

So you can't put it in a product, and I'd read "research and educational purposes only" as excluding internal business tooling too. Qwen3.5-9B is Apache 2.0. So is the Granite base TwIL was built from.

webAI took an Apache model, spent compute making it better at math, and closed the commercial door on the result. That's legal and it's their call. It also means TwIL is mostly a hobbyist and research model, whatever the benchmark table says.

My take

If you want a local math tutor or a competition-problem grinder, and you're not shipping anything, TwIL-8B balanced at Q4_K_M with a 32k token budget is worth an evening. MATH-500 at 98.2 is strong for its size, and it uses fewer tokens per contest problem than Qwen.

For everything else I'd stay on Qwen3.5-9B. It's ahead on GPQA, MMLU-Pro and AIME 2026, it's Apache 2.0, and its headline numbers don't depend on which variant you downloaded. The LiveCodeBench lead is the one result that would change my mind, and it's the least verified number on the card.

Would you run a stronger small model under a non-commercial license, or stick with Apache-licensed Qwen or Granite? And if you've actually run TwIL's verbose GGUF on LiveCodeBench-style problems, did you get anywhere near 81.6?

Sources

• webAI-Official/TwIL-8B model card
• webAI Non-Commercial License ver. 1.0
• Qwen/Qwen3.5-9B model card
• explainx: TwIL-8B claimed vs verified