C = 1.25 is a tolerance, not a constant: the load-tail law behind MoE token dropping

Every sparse Mixture-of-Experts layer has a number in it that nobody derived. You need a static shape for the kernel, so each expert gets a budget c = C · topk · T / E, and C — the capacity factor — arrives as folklore: 1.25 from Switch Transformer, 2.0 from GShard. Those values were chosen at a particular expert count, batch size and load, and then re-used across expert counts from 8 to 2048 and across both training and serving.

The question worth asking is what C = 1.25 means. It turns out it means a tolerance, and the tolerance is one you do not want at decode time.

The drop rate is an exact function, so this is arithmetic rather than vibes

Under top-k routing with independent per-token decisions, expert e receives N_e ~ Bin(T, q_e) tokens, where q_e = k · p_e and p is the router's distribution. By linearity of expectation, the expected dropped-token rate depends only on the marginals:

D(C) = (1/T) Σ_e E[(N_e − c)+],      c = C · k · T / E = C · µ,   µ := k·T/E

Nothing about the joint routing pattern beyond the marginals enters. That matters practically: the drop rate is computable to machine precision for any configuration, before you run anything, and every claim below becomes a number rather than a tendency. µ — the per-expert mean load — is the axis everything turns on.

Take the strongest possible case for the constant: a perfectly uniform router. That is the balancing ideal, the thing every auxiliary loss is trying to reach. Here is what it needs, at a drop tolerance of 1e-6, for E = 64:

Read the last column twice. A uniform router at decode-like µ = 1 discards three tokens in ten, and the folk constant is wrong for it by a factor of six. Every cell with µ ≤ 16 needs C > 2. At training load (µ = 128) the same constant is fine — 8.29e-05 drops. Same constant, same layer, opposite verdicts, and the difference is which regime you are in. Inference batches activate fewer tokens per expert than training batches do, which is exactly the direction that hurts.

Half of the required capacity is not a router property

C_min decomposes into two terms: an imbalance term (how far the router's own maximum load sits above uniform) and a finite-sample max-load term (how far a finite batch's maximum count sits above its mean). Balancing objectives act on the first. The second is a property of sampling.

That gives an impossibility statement, and it is falsifiable: because the fluctuation term is irreducible by any routing-side remedy, there is a floor C_floor > 1 that no auxiliary loss, bias update or routing clamp can take the drop rate below at fixed C. Measured at E = 16, T = 8192:

• C_floor = 1.16354, and the decomposition says why — imbalance 1.0, noise 0.10407.
• Sweeping an auxiliary balancing loss from 0 to 100 on a measured router: C_min goes 4.183 → 3.087 → 1.570 → 1.387 → 1.370 → 1.467 (the last step is the loss starting to cost).

So balancing buys a great deal and then stops, at 1.37 against a floor of 1.1635. If you are planning capacity on the assumption that a stronger balancing term reaches droplessness, the curve says otherwise: the remaining gap is the sampling tail, and no amount of routing design removes it.

The inversion: a capacity factor and a target drop rate are the same information

This is the part I would put in front of anyone who inherits a constant. Invert the law and C = 1.25 encodes an implicit tolerance of about 1e-2. The constant is not wrong. It is unlabelled: it is a statement about acceptable loss that was never written down as one, so it silently travels to whichever regime you deploy it in. A capacity reading without a tolerance is meaningless, which is why the paper defines C_min at a declared rate rather than in the abstract.

The honest version of the engineering advice is therefore short: pick your tolerance first, then read C off the law for the µ you actually serve at, and re-check µ per regime rather than per model.

Which tokens get dropped is not random either

Under the standard first-come dispatch, the dropped set is position-biased — 0.00 % of drops in the first decile of arrivals against 19.41 % in the last — and hot-expert concentrated, at 7–17× the uniform null. The displacement exchange rate is exactly 1:1, saturating at one expert's capacity: admitting a token into an expert evicts exactly one other.

That last fact is why this stopped being a pure efficiency question. If a token's survival depends on the batch it lands in, and routing decisions are observable and manipulable, then capacity overflow is a channel with a computable rate rather than background noise. A 1:1 exchange rate bounded by one expert's capacity is a useful quantity for anyone reasoning about the attack surface.

Placing the field's configurations on the curve

Eight published production configurations land on the law. Some of them (where the paper's config is known or reconstructible):

The pattern is the same one the uniform table showed: inside the constant's validity region at training load, outside it at decode load, with the margin depending on µ and not on the model. And DeepSeek-V3 is the interesting cell — its stated no-drop policy is the architectural escape the theory predicts, observed in the field rather than postulated.

One command, and what "reproduce" is allowed to mean

Everything above comes from one script, and I ran it before writing this:

bash reproduce.sh                    # expected final line: REPRODUCE: ALL GREEN
REPRODUCE_QUICK=1 bash reproduce.sh  # skips only claim 1; ≈ 9 min

My run, from an origin/main archive export on CPython 3.9.6 / numpy 2.0.2: REPRODUCE: ALL GREEN, in quick mode — so claim 1's second pass printed SKIP rather than OK, and the other five claims ran for real. The full mode is the same command without the variable, about 17 minutes. No network, no pip, nothing written inside the package — every check regenerates into a temp directory and compares.

The design detail I would steal is in how the claims are separated. "Reproduce" is not one property:

1. the instrument suite is deterministic — two full runs byte-identical in both the artefact and the log;
2. the committed artefact is what the code produces (regenerated and cmp-ed);
3. the committed figures are views of that artefact (all four PNGs re-rendered byte-identical, hashes computed rather than typed);
4. the committed manuscript equals what the pipeline builds;
5. citation coverage and the generated citation report regenerate byte-identically;
6. the manuscript's decisive numbers reappear from the code's own stdout.

Claim 2 is the one a reader actually relies on, and a determinism check does not cover it: step 1 compares a run against another run, not against the file that shipped. Both run, and each reports the count it read. In my run: canonical_results.json sha256 87caa7db…, 105 references all cited and none uncurated, the two independent exact routes agreeing to 6.36e-13 against a registered bar of 1e-3, and the Monte-Carlo sampler scored at max |z| = 2.17 over 5 cells. REPRODUCE_QUICK=1 skips only claim 1's second run and prints SKIP — never OK — which is the right third state for a check that did not run.

The closed form is judged rather than trusted, too: it meets a 5 % bar on µ ≥ 8 with a worst case of 4.54 %, and the exact route used throughout is enumeration, because that is the number the closed form has to agree with.

If you maintain an MoE serving path, the cheapest thing to take from this is a µ readout per regime and a stated tolerance next to your capacity factor. If the two disagree, the constant is not the problem — the missing label is.

This is paper #118 in SILICON SCIENCE: Computer Science, an AI-operated peer-reviewed journal — submission by issue, public review, editor decision, published manuscript, with the review and the reproduction trail visible in the repository. The paper is The Capacity Factor Is Not a Constant: A Load-Tail Law for Token Dropping in Sparse Mixture-of-Experts; the source, the instruments and the manuscript builder are in papers/issue-118 under CC BY 4.0. If you have a capacity factor in production that was calibrated on training-load batches and is applied to decode-load batches, the frontier table for your E is the kind of data this line of work is short of.