Someone fine-tuned a 360M model to be a terminal. There's no shell behind it.

WorldTUI hit 540+ points on Hacker News this week, and it deserves it. Someone took a 360M-parameter language model, shrank it to 208 MB, and made it behave like a terminal. It runs in a browser tab. You type, it predicts how the screen changes.

There is no shell. There is no filesystem. Type pythoon and it still launches Python, because the model is guessing, not executing.

It creates files, runs vim and tmux, plays snake, and usually remembers what you wrote a few commands ago. That last part is the interesting one.

Credit where it's due: this is simedw's project and write-up. Everything below comes from that post. I'm here for the engineering tricks, because they transfer.

The core idea: predict screen diffs, not text

A chat LLM appends tokens to the end of a stream. A terminal doesn't work like that. clear wipes the screen. esc in vim does something on its own. Output isn't always at the bottom.

So the model never sees a text stream. It sees a grid of characters and predicts edits to it.

INPUT_START 'l' INPUT_END

OUTPUT_START
ROW_0 17 'l'
CURSOR 0 18 1
OUTPUT_END

Read that as: on row 0, from column 17, write l, then put the cursor at row 0, column 18, visible. Keypresses go in as raw bytes. Enter is \r, Escape is 0x1b, left arrow is ESC [ D.

Time is an input too. With no key pressed, the model gets a WAIT token with a duration. That's how snake keeps moving by itself.

Trick 1: a COPY op, so scrolling costs 3 tokens per row

Scrolling a full screen means every line moves up one. Re-emitting all of it is about 3,000 tokens for one keypress. So the author added a copy operation:

ROW_0 COPY 1
ROW_1 COPY 2
ROW_2 COPY 3

Three tokens per row instead of roughly 96. The same op handles clear, vim line inserts and deletes, and man page navigation.

The whole vocabulary ends up at 333 tokens: 256 byte tokens, 64 row tokens, and a few special ops. Decoding is grammar-constrained, so every prediction is a valid screen update. The model can be wrong, but it can't emit garbage.

Trick 2: train on your own mistakes

Plain supervised fine-tuning worked for typing. In longer sessions, errors compounded. One wrong character in a filename and the rest of the session drifted.

The fix borrows from DAgger. Let the model predict the next 1 to 4 events, compare its screen to the real one, and if it's close, build a correction that repairs the screen on the next keypress. Then train on that.

There's one rule that I'd steal for any similar project. A correction may only use words that appeared on an earlier real screen or in the user's typing. Otherwise you're rewarding the model for hallucinating a filename it had no way of knowing.

Result: PresetBench went from 24/47 to 38/47 at int4. Three rounds of this, each collecting mistakes from the latest model.

Trick 3: rows are independent, so decode them in parallel

A full redraw was about 3,000 tokens decoded one at a time. Typing felt fine at ~38 ms per key. vim, man and clear took up to 12 seconds on an RTX 4090.

Most rows in a frame don't depend on each other. So: one batch element per row, all sharing the same history and KV cache. Cost drops from the sum of all rows to the length of the longest row, around 100 steps. Large-frame p95 latency went from 6.7 s to 0.87 s.

The catch is real. Independent rows can't coordinate, so vim opened on row 3 and snake spawned food on several rows at once.

The fix is speculative decoding with the same model as drafter and verifier. The parallel rows are the draft, then the model verifies them sequentially, keeps tokens up to the first disagreement, and re-proposes the rest. On an M4 Mac in the browser, man test takes 2.5 s sequentially and 0.9 s with draft and verify.

Trick 4: delete the vocabulary

The base is Qwen2.5-Coder-0.5B, about 494M parameters. Its 151,936-token embedding table is 136M of those. WorldTUI needs 333 tokens (padded to 384 for the GPUs).

Drop the rest and you're at about 360M parameters, 990 MB down to 716 MB. Quantize to int4 and it's 208 MB. Inference is a custom JavaScript Qwen decoder with WGSL compute shaders on WebGPU. Weights unpack from 4-bit in the shader. Everything runs locally in the tab.

The part nobody should skip: memory lives in specific layers

The author tried to distill into fewer layers and found something ugly.

Removing just layer 7 from the teacher barely moved next-token accuracy: 99.3% to 98.9%. Exact memory readbacks went from 64/64 to 0/64.

A student that kept every even layer looked fine and returned the right value in 31/64 readbacks. That's coin-flip. It always printed one of the two values it had written, and picked wrong half the time. A 16-layer student that kept layers 1, 7, 15 and 22 got 64/64 at about 138 MB, but lost ground on ShellBench: 65 correct versus the teacher's 78.

If you ship distilled models, this should bother you. Next-token accuracy can look healthy while a capability is gone. Test the behavior you care about directly.

The numbers

Fine-tuning on 336k synthetic sessions (about 2B tokens, generated by running real shells in Docker with a 1.2% typo rate) took roughly 75 GPU-hours on one RTX 4090.

Fine-tuning roughly quadrupled memory recall over prompting. That's the headline.

What it can't do

Be honest about this, because the author is.

• It isn't a reliable filesystem. After echo green >> colors, cat colors often misses the appended line.
• Nested directories sometimes teleport you to a different path.
• Python handles simple expressions. Harder ones give GPT-3-era hallucinations.
• The demo is monochrome, and it ran about 20x slower in Firefox than Chrome in the author's testing. Cause unknown.

The author's own verdict: "Is it useful? Not really."

Why you should care anyway

Nobody needs a fake terminal. But a pile of ideas here apply to any small-model product:

1. Change the output format to match the problem. Diffs and copy ops beat regenerating the world.
2. Constrain decoding. A 333-token grammar means invalid output is impossible.
3. Train on the model's own mistakes, with a hard rule against teaching it facts it couldn't have known.
4. Drop what you don't use. Vocabulary pruning alone cut 134M parameters.
5. Measure the capability, not the proxy. Perplexity stayed fine while memory went to zero.

The author's next experiment is training on thousands of randomly generated apps, to see if the model can predict an app it has never met from context alone. If that works, "apps that are learned, not coded" stops being a joke. Prior art is there: GameNGen simulated Doom with diffusion, NeuralOS did a desktop GUI, and people prompted ChatGPT into acting as a Linux terminal back in 2022.

The demo is a 208 MB download and runs entirely locally. Use Chrome or Edge on a desktop with WebGPU. Type pythoon, then try to break it: simedw.com/projects/worldtui. Weights are promised soon.

Source: WorldTUI: I trained a 360M model to pretend to be a terminal · HN discussion