Volume IV · Change the Mathematics

No Free Lunch

Three flagship models from one laboratory in three years. The first bet on linear attention. The second withdrew the bet in public and explained why. The third placed a different bet altogether. Read in order, they teach more than any leaderboard.

The agreement, and the argument

Set the 2026 technical reports side by side and the first impression is one of consensus. The skeletons match: an ultra-sparse mixture of experts, a small fraction of the parameters active on any token, a context measured in hundreds of thousands of tokens. Turn to the attention section of each and the consensus evaporates. There, in the space of a few paragraphs per report, the field is openly and unresolvedly divided.

The division runs three ways. One camp bets on trainable sparsity: keep exact softmax, but let the model learn which cells of the grid are worth computing — DeepSeek's sparse attention, adopted by GLM-5, and MiniMax's own sparse variant. A second camp bets on linear hybrids: replace most layers with a constant-memory recurrence and keep a few full-attention layers as an escape route — Kimi's KDA, Qwen's Gated DeltaNet, NVIDIA's Mamba hybrids. A third camp holds full attention deliberately, having tried the alternatives and declined them.

This lecture is about the third position, because the laboratory that arrived at it got there by publishing the failure that sent it back. That is rarer than a good benchmark result and considerably more instructive.

One laboratory, three architectures

MiniMax-M1, June 2025. A 456-billion-parameter mixture of experts with 45.9 billion parameters active, built on a hybrid lightning-attention backbone, with a context window of one million tokens. It is the argument of Volume IV made at frontier scale: mostly linear layers, a constant-size state, and the long context that a non-growing cache buys you. At the time it was the most committed bet on linear attention any major laboratory had shipped.

MiniMax-M2, released as a model in October 2025 and written up in the series report of May 2026. Smaller and much sparser: 230 billion total parameters with 10 billion active, 62 layers, 256 experts, a native context of 192K tokens. And — the part that matters — a deliberate return to full attention. Not a smaller hybrid ratio. Not a better linear rule. The whole mechanism, put back.

MiniMax-M3, June 2026. A 428-billion-parameter model with roughly 22 billion active, natively multimodal, a million-token context, and an attention design that is neither of the previous two: a grouped-query backbone with MiniMax Sparse Attention layered on it. Having retreated from linear to full, the laboratory did not stay at full. It moved across, to trainable sparsity.

The retreat, and its four reasons

The MiniMax team published an account of why M2 ended up a full-attention model, and its structure is worth learning as a template for reading any efficiency claim. Their finding was not that the efficient variants were bad. It was that the evidence supporting them came from the wrong scale.

At small scale, they report, the efficient variants matched full attention on benchmarks. That parity did not survive the move to frontier scale, and it broke in three specific places. The models showed deficits in multi-hop reasoning — the chains where an answer depends on retrieving one fact in order to retrieve the next. Their quality degraded with context length in agentic use, precisely the long-session behaviour the long context was bought for. And the state proved fragile under low-precision KV, which is not an academic concern when low precision is how the memory saving is realised in the first place. The fourth reason was not about the architecture at all: the surrounding infrastructure — kernels, serving stack, post-training recipe — was not ready.

An independent analysis published by LMSYS put the same conclusion in its title — No Free Lunch — and that is where this lecture takes its name. The phrase is doing real work. It does not say linear attention is worse. It says the saving is not free, that the price is paid somewhere, and that the somewhere had been outside the field of view of the experiments used to justify it.

The opposite bet, apparently paying

Run the same period from the other side and you get a different verdict. Kimi Linear, published in October 2025, claimed something stronger than parity: that a linear hybrid could beat full attention under matched training conditions, and that the advantage survived reinforcement-learning post-training — the stage where efficient variants had most often been suspected of quietly falling behind. Moonshot then scaled the approach: Kimi K3 was announced on 16 July 2026 with 2.8 trillion parameters and a one-million-token context.

Two things must be said about K3 in the same breath. The first is that scaling an approach to a flagship is the strongest signal a laboratory can send about its own confidence. The second is that at the time of writing, the K3 technical report has not been published — it was promised for late July 2026 — so its architectural details remain unconfirmed. unconfirmed Do not cite K3's attention design as evidence for anything. Cite the announcement as an announcement, and wait.

What a student should take from this

Three lessons, in ascending order of usefulness.

The first is that benchmark parity at small scale does not predict behaviour at frontier scale. This is not a caution about noise; it is a statement about where the failures live. Both MiniMax's account and the pattern of the field point the same way — the capabilities that break are the ones hardest to measure, multi-step reasoning and long agentic sessions, and they break late. A table of matched scores at a small parameter count is evidence of very little.

The second is that an architecture is never evaluated alone. It is evaluated together with its kernels, its serving stack and its post-training recipe. MiniMax cited unready infrastructure as a reason alongside the model-quality findings, and that is the honest accounting: a mechanism that is better in principle and worse in the stack you have is worse this year. Volume V is about exactly this, and it is next.

The third is the one to carry longest. A laboratory that publishes why its approach failed is doing more for the field than one publishing another leaderboard. A benchmark table tells you where one model landed. A retreat, written up with its reasons, tells you what to expect from a whole family of designs and which experiments would have caught it. The M2 write-up is a negative result at frontier scale, and negative results at that scale are almost never published, because publishing one costs the laboratory something.

The honest position to end on

So: sparse against linear-hybrid is an open bet. One laboratory tried linear at scale, published its retreat, and then moved to sparse. Another claims linear wins under matched conditions and has scaled it to a flagship whose report we have not read. Both camps are staffed by careful people with more compute than any of us and access to results that are not in any paper.

The mental model, then, is not a mechanism. It is a posture. Hold the question open, read the retreats as carefully as the releases, and ask of every efficiency claim at what scale it was measured and against which capabilities. Anyone who tells you the question is settled is selling something.