Amodei's slow-AI plan draws buy-in
Anthropic's CEO proposes third-party evaluators and coordinated limits on AI progress, and OpenAI's Sam Altman says he agrees.
Anthropic CEO Dario Amodei has published a plan for slowing the pace of frontier AI development, and within hours the proposal had picked up public backing from OpenAI CEO Sam Altman and SpaceX CEO Elon Musk. The post lands at the end of a week in which the internal debate over AI safety at the leading labs spilled into public view.
In a new blog post, Amodei echoed earlier calls to "pace the frontier" and laid out three broad strategies for doing so. He said Anthropic is "unilaterally committing" to one of them, and Altman responded that OpenAI will follow suit.
Resignation reframes the debate
The discussion intensified after researcher Jacob Coxon wrote that he is resigning from Anthropic, citing concerns that the leading AI companies are "gambling with our lives" while the people building the technology "earnestly believe it could kill us all by the end of the decade." According to the source account, that claim was repeated by others at Anthropic.
Amodei's post does not explicitly mention Coxon's resignation or his concerns. Instead, the CEO wrote that two things convinced him it is time for a more cautious approach: the OpenAI-HuggingFace hack, and the fact that "AI has been advancing drastically faster" in recent months, particularly with its "growing ability to build the next generation of AI."
The central argument is stated plainly in the post.
"We must slow the pace at which we improve the capabilities of AI models."
— Dario Amodei, CEO of Anthropic
He added that "Progress will still seem fast, and we must make wise use of the time we gain."
Embedded evaluators as a first step
Amodei's proposed first step calls for "embedded evaluators" drawn from third-party organizations such as METR. Those evaluators would verify that AI companies are actually following their pacing and safety commitments, and would also work to ensure safety incidents get reported.
The source notes that OpenAI was recently criticized for not reporting an incident in which its AI agents took over a German wiki forum. That episode is cited as an example of the reporting gap the evaluator model is meant to close.
Amodei compared the evaluators to regulators who have been embedded with bank employees. He described inviting them in as "something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match)." In practice, that means giving evaluators company badges, desks, and laptops, along with access "mostly comparable to what internal risk assessment teams have," with exceptions when required by law or contracts.
Altman called the idea a "good idea" and said OpenAI would do the same, adding, "We'll have more to share soon."
Coordinating standards without antitrust trouble
Second, Amodei called for leading AI companies "within democratic countries" to coordinate on "common safety standards as well as limits on the rate of unchecked AI progress."
Such coordination might appear unlikely, since these companies are reportedly worried that a coordinated pause could invite antitrust scrutiny. Amodei alluded to that concern directly, writing that "for antitrust reasons, it's helpful for the US government to mediate or at least enable these discussions — they don't need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations."
He also addressed the argument that slowing development would hand an advantage to China. His position is that if the US government and tech companies take steps such as refusing to sell powerful chips or semiconductor manufacturing equipment to Chinese companies, and crack down on model distillation, they could "slow China's progress enough to widen America's lead significantly over the next 3–5 years."
A bid for global coordination
The third strand of the plan is what Amodei described as "global coordination," in which the United States and its allies "attempt to coordinate with authoritarian governments, to the extent this is possible."
He said this would mean "cooperation with China," and acknowledged that there are "stark limits on what can be achieved." Even so, he suggested there might be room for agreement on narrow points, such as "prohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so."
The scope of the proposal is narrow by design on this front. Amodei frames broad coordination as aspirational and low-probability, while treating a handful of specific prohibitions as the realistic target.
Reactions from rivals and allies
Other AI executives reacted positively to the post, according to the source. Altman wrote, "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks."
Musk's response was shorter: "Dario is right."
The support matters because Amodei's proposal depends on other frontier labs matching Anthropic's commitments. A unilateral move by one company would leave the evaluator model and coordination framework incomplete without participation from peers.
Pushback from industry critics
Amodei has previously acknowledged AI's potential dangers and his company has shown relative openness to certain forms of regulation. Some AI boosters have criticized him as a doomer whose comments have fed the current AI backlash.
In response, Amodei said he has tried to offer a "balanced" perspective, and argued that the backlash is "fundamentally a crisis of trust," as people have become skeptical of tech companies, the tech industry, and the government.
Industry critics have also been skeptical about apocalyptic AI warnings, suggesting they are a distraction from harm the technology is already causing. Journalist Brian Merchant wrote that he has yet to see "a credible, step-by-step documentation of how exactly AI might move from self-recursively improving AI to killing every single human on the planet." He also suggested that proposals similar to Amodei's "would likely only wind up serving Anthropic and OpenAI; it's what regulatory capture looks like in action."
Amodei closed his post by restating his faith in the technology's upside: he continues "to believe that AI can enormously improve the quality of human life."
"My desire to achieve these benefits is undimmed. But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right."
— Dario Amodei, CEO of Anthropic
The open questions in the plan
Several elements of the proposal remain unspecified in the source material. The post does not detail how embedded evaluators would be selected, how disagreements between evaluators and the labs they monitor would be resolved, or what happens if a company declines to honor its pacing commitments.
The coordination mechanism is also undefined beyond the request for a narrow antitrust waiver from the US government. The post does not say which countries would participate, what the common safety standards would contain, or how the rate limits on "unchecked AI progress" would be measured.
On China, Amodei's argument rests on export controls and anti-distillation enforcement continuing. The 3–5 year window he cites is an estimate of how long it might take to widen America's lead under that approach, not a commitment made by any government.
What the commitments cover
The piece does not enumerate every element of what Anthropic has pledged, but the source identifies several concrete components tied to the evaluator program.
- Anthropic says it is unilaterally committing to inviting in embedded evaluators.
- Evaluators would receive company badges, desks, and laptops.
- Access would be "mostly comparable to what internal risk assessment teams have," with exceptions required by law or contracts.
- The proposal covers third-party organizations such as METR.
- Amodei calls on governments to require other frontier companies to match the commitment.
- He estimates that chip and equipment controls plus anti-distillation enforcement could widen America's lead over the next 3–5 years.
Altman's response indicates OpenAI intends to adopt the same evaluator arrangement, though the source notes he did not provide details beyond saying more would be shared soon.
Why this matters beyond the labs
If the commitments hold, the practical effect for outside observers is more visibility into how frontier labs handle safety incidents and pacing decisions — something that currently depends largely on voluntary disclosure. The evaluator model, if implemented as described, would place outsiders inside the companies with access comparable to internal risk teams, which is a different arrangement from the external audits and self-reporting that have characterized the sector so far.
The antitrust waiver request suggests the coordination piece may hinge on government action rather than company-to-company agreement alone. If no waiver materializes, the second strategy could stall even with broad executive support.
For readers tracking AI policy, the more immediate signal is that the two largest US frontier labs are publicly aligned on pacing language. Whether that alignment produces verifiable limits, or remains a statement of intent, depends on details not yet published. The skepticism from critics like Merchant points to the gap that will define the next phase: the difference between announcing a commitment and demonstrating that it changed what a lab actually built.
Sources
- TechCrunch Original source
- METR Also reporting
Continue Reading
Anthropic: Yemen Cell Tried AI Weapons
Anthropic says it blocked Claude accounts in Houthi-held Yemen that tried to develop advanced missiles with the AI model.
AI Rollouts Outpace M365 Permission Checks
A Syskit study finds 76% of UK and US organizations have deployed enterprise AI, but only 43% reviewed permissions first.
Anthropic Researcher Exits Over AI Safety Race
A researcher says he left Anthropic over concerns it and OpenAI prioritize competitive advantage over safety in AI development.