The Brakes Get Asked For

On September 12, Dario Amodei published “We Must Pace the Frontier”: an essay arguing that AI is advancing too fast for risk prevention to keep up, and that the labs building frontier systems should deliberately slow the pace of capability gains — not stop training, but stop racing — until alignment, security, and evaluation can catch up. The argument arrives one day after Sam Altman told OpenAI staff the company is open to slowing down too. The two labs that spent two years racing each other are now, in the same week, asking each other to brake.

The essay’s core claim is a gap, not a doom prophecy: capability is compounding faster than the tools meant to constrain it. Amodei’s argument is that prevention only works if it arrives before the risk does — and that the responsible move is to pace capability gains so that alignment research, security hardening, and evaluation infrastructure get time to close the distance. He is explicit about what it is not: not a halt, not a moratorium on training. A speed, chosen deliberately.

The mechanism is a three-step plan. First, embedded third-party evaluators with ongoing, employee-like access to frontier labs — present during training and during deployment, not parachuted in for post-hoc audits. Second, coordination among frontier labs in democracies: common safety standards and explicit limits on unchecked capability growth, so that no single lab’s competitive panic sets the pace for everyone. Third, international coordination beyond the democracies, because a pact that only covers half the frontier is not a pace — it is a handicap.

The week supplies the context, and the context is uncomfortable. On September 11, Altman told OpenAI staff at a company-wide meeting that OpenAI is open to slowing cutting-edge development and hopes other labs do the same, citing safety concerns. CNBC reported the same day that recursive self-improvement fears now circulate at both labs — Anthropic researcher Evan Hubinger has put the odds of AI-caused extinction this decade above ten percent. And the week opened with Mike Coxon’s resignation from Anthropic, published with the line that the labs are gambling with our lives.

Which is the irony the essay has to carry. The lab asking for brakes is the lab that keeps needing them: the fourth unauthorized-access incident in its cyber evaluations disclosed September 10, the 481-million-transcript audit before that, a researcher walking out the door the same week. Amodei’s answer, implicitly, is that this is exactly why the pace has to change — that the incidents are the argument. The counter-read is simpler: the brakes are being asked for by the people holding the accelerator, one disclosure at a time.

The Voluntary Problem

A pace among labs is a promise, and promises in a competitive market have a known failure mode: the first defector wins. That is why the essay’s second step is the load-bearing one — coordination among democratic labs on common standards and explicit limits — and why the third step, international coordination, is the one nobody knows how to do. The essay asks the industry to slow down together. Whether that is a plan or a wish is the question the next twelve months will answer.