Anthropic CEO calls for pacing AI frontier model development and warns 'in 6–12 months such a swarm [of agents] could be capable of taking over the entire internet' | Tech Radar

Overview

News, deals, reviews, guides and more on the newest computing gadgets

Start exploring exclusive deals, expert advice and more

Details

Unlock and manage exclusive Techradar member rewards.

Unlock instant access to exclusive member features.

Get full access to premium articles, exclusive features and a growing list of member rewards.

Anthropic CEO calls for pacing AI frontier model development and warns 'in 6–12 months such a swarm [of agents] could be capable of taking over the entire internet'

When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.

Anthropic CEO Dario Amodei calls for frontier model pacing

It'll require cooperation from other AI companies and, yes, even China

Maybe you're tired of hearing the three-year AI industry veteran, Jacob Coxon, warn us on every available media platform that AI could kill us all by the end of the decade.

It sounded hyperbolic, and maybe it is. But when the longtime CEO of Anthropic (Coxon's former employer), Dario Amodei, tells us frontier model development is going too fast and we "risk losing control of AI systems," you might be inclined to listen.

In a roughly 3,000-word blog post, Amodei outlined on Saturday the growing risks of unfettered, global frontier model development and laid out a multi-part plan for gaining some level of control and safety.

In a way, Amodei's post echoes Coxon's concerns, who also called for "pacing."

"Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity," wrote Amodei. He warns, though, that we are facing "the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption."

'No one is prepared for the consequences': Even Open AI chief scientist is saying AI development needs to slow down

Open AI CEO Sam Altman is worried about AI being controlled by just a few powerful players

Naturally, Amodei points to the summer's incidents, the most notable of which is when Open AI's AI models escaped the sandbox and then attacked Hugging Face's system in a coordinated effort to complete its objectives.

Amodei contends that despite no one getting hurt, the incident should serve as a warning about what could come next.

"A similar level of misalignment could have caused catastrophic damage...it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet."

Amodei's post differs from Oxon's alarmist X post in that it offers a framework for global frontier pacing, basically slowing down and managing model development without calling for a pause.

It's an ambitious plan that includes an internal but independent ombudsman at each AI company who might have a series of checkpoints they can use to evaluate ongoing work and to ensure that the AI companies are following standardized guidelines and rules. They can also be there to offer a point of clarity, without the cloudiness of commercial demands.

JPMorgan CEO Jamie Dimon says Anthropic's AI model poses some serious risks

"Either you betray your values, or you become irrelevant": Quote of the day by Anthropic CEO Dario Amodei

Why are so many AI models going 'rogue'? The experts weigh in

Amodei also wants "Democratic Coordination," which would mean companies like Open AI, Google, and Anthropic agree on standards, which of course the third-party evaluators can then use. He even proposes global coordination, though Amodei seems less certain that it can even work.

More interestingly and perhaps in response to recent news that AI's new agentic and recursive model capabilities are making them more inscrutable than ever, Amodei thinks pacing will provide more time for better interpretability. "Despite all the progress," Amodei writes, "we still only understand a tiny fraction of what goes on inside these models."

Throughout the document, though, the theme remains almost entirely on "pacing" and not "pausing". In fact, Amodei is quite clear that we can't afford to slow down too much, lest we fall behind the chief AI global competitor, China: "Thus, a key part of pacing within democracies is to keep democracies’ AI lead over autocracies as large as possible, to give us the breathing room we need in order to pace effectively." Not doing so would create a "significant national security risk."

Amodei briefly floats the idea of a global frontier model development pause as participating governments reach an agreement on the pace of AI development, but also adds that such an agreement is "unlikely."

Part of Amodei's plan, and to help, maybe, keep China in line, is a call for us to stop selling AI chips to China, something Nvidia's Jensen Huang will surely have something to say about (he actively lobbied the White House to let his company sell AI chips to the CCP). He also calls for penalties for "frontier model distillation," basically China and other countries using Anthropic and, perhaps, Open AI models to train their own.

Naturally, Amodei also calls for a "global standards body, though he admits that it won't be easy to give it "real teeth."

Coxon's comments created a firestorm of debate around the safety of AI and the advisability of allowing development to continue at this pace. That debate, though, was couched in "consider the source." Coxon worked for just three years as a model trainer and at two different companies. Some wondered if his posts and subsequent media blitz were just a cry for attention.

Amodei's post and plan, by contrast, carry the gravitas of deep experience and a macro view of all the pieces at play. Amodei knows the capabilities because he sees them up close every day; he knows the benefits from a global and a financial perspective, and he understands the risk, likely even better than Coxon does.

While much of his plan is based on "only if everyone cooperates and is generally on their best behavior," it's impossible to ignore the warning. Amodei admits his plan "won't be easy" but thinks we must try because "we owe it to humanity.

Amodei dropped the post over the weekend, perhaps hoping to give his counterparts at Google, Amazon, Meta, and, especially, Open AI time to consider it before responding on Monday. Amodei actually name-checks Google's Demis Hassabis in the post, but doesn't mention Altman. The Open AI chief and Amodei have a notoriously chilly relationship, which might make Altman embracing Amodei's seemingly sensible plan a long shot.

Follow Tech Radar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.

A 38-year industry veteran and award-winning journalist, Lance has covered technology since PCs were the size of suitcases and “on line” meant “waiting.” He’s a former Lifewire Editor-in-Chief, Mashable Editor-in-Chief, and, before that, Editor in Chief of PCMag.com and Senior Vice President of Content for Ziff Davis, Inc. He also wrote a popular, weekly tech column for Medium called The Upgrade.

Lance Ulanoff makes frequent appearances on national, international, and local news programs including Live with Kelly and Mark, the Today Show, Good Morning America, CNBC, CNN, and the BBC.

You must confirm your public display name before commenting

NEC drops its quantum computing work, marking an end to nearly 30 years of development

Anthropic CEO calls for pacing AI frontier model development and warns 'in 6–12 months such a swarm [of agents] could be capable of taking over the entire internet'

Dell Ultra Sharp U3226Q review: A creator's dream, a jarring price

I rode Volkswagen's ‘world’s safest e-bike’ at IFA 2026

Six years later, the PS5 is still light on exclusives — but this space-age shooter is an absolute must-play

Tech Radar is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site.

© Future US, Inc. Full 7th Floor, 130 West 42nd Street, New York, NY 10036.

Key Takeaways

• News, deals, reviews, guides and more on the newest computing gadgets
• Start exploring exclusive deals, expert advice and more
• Unlock and manage exclusive Techradar member rewards
• Unlock instant access to exclusive member features
• Get full access to premium articles, exclusive features and a growing list of member rewards