The Other Way to Sample

For a decade the language model had one shape: a transformer that writes left to right, one token at a time. Inception Labs released Mercury 2.5 this week — a diffusion LLM that refines whole blocks of text in parallel — and reported 1,107 tokens per second on a single H200, with quality near the autoregressive frontier.

The mechanism is the story. Autoregressive models are chained to causality: token 500 cannot be written until token 499 exists, so speed means stacking more silicon or guessing ahead with a draft model. Diffusion models break the chain — they start with noise and refine an entire passage simultaneously, each denoising step touching every position at once. Mercury 2.5 is the first diffusion LLM with numbers that make the trick production-relevant: 1,107 tok/s on one H200 (about ten times typical frontier decode), no speculative decoding, and benchmark parity claims against autoregressive models several times its size on code and agentic tasks.

Provenance, because this page does not do press releases uncritically. Inception Labs is the Palo Alto startup from Stefano Ermon and Stefan Heck — the team that shipped Mercury Coder in February 2025 as the first commercial diffusion LLM and has been walking the quality gap closed ever since. The 2.5 release claims state-of-the-art diffusion results across coding, agentic, and general benchmarks, with per-token costs framed as a fraction of frontier autoregressive inference. These are the company’s own benchmarks; independent replication is the usual open question. But the direction of the curve has been independently visible for eighteen months, and it points one way.

Why it matters is where the tokens go. Agentic workloads are the whole industry’s growth story, and agents are decode-bound: thousands of tool calls, long chains of thought, retry loops. When the bottleneck is tokens per second per dollar, a 10x decode advantage is not a benchmark curiosity — it is a different cost curve for the entire agent economy. The labs’ own disclosures this month (3.1 agent-workdays per human workday) describe compute-hungry loops; parallel sampling is one of the few levers that changes the loop’s price rather than the loop’s capability.

The honest caveats, in order of severity. Parity claims at this scale always mean “on the benchmarks the company chose” — the long-tail reasoning behaviors that make frontier models feel general are exactly where niche architectures show their seams. Diffusion LMs still trail on tasks where the next token genuinely depends on fine-grained causal structure. And Inception is a startup shipping on rented GPUs against labs that own their silicon — the cost advantage is real today and contestable tomorrow. None of this makes the release less interesting; it makes it a data point in an architecture argument, not a verdict.

What to watch is the incumbents’ response, not the startup’s next release. Every frontier lab has a diffusion research lineage — they all read the same papers. If Mercury 2.5’s parity claims survive independent testing, sampling strategy stops being a research footnote and becomes a roadmap item at every lab that sells tokens by the million. The transformer is not going anywhere. But its sampling rule just got a competitor, and the price of a token has a new floor to argue about.