More than 1 in 10 chance AI ‘could kill all humans,’ says Anthropic safety lead after colleague quits | The Verge
Overview
Tech Expand Amazon Apple Facebook Google Microsoft Samsung Business See all tech
Reviews Expand Smart Home Reviews Phone Reviews Tablet Reviews Headphone Reviews See all reviews
Details
Science Expand Space Energy Environment Health See all science
Entertainment Expand TV Shows Movies Audio See all entertainment
Policy Expand Antitrust Politics Law Security See all policy
Gadgets Expand Laptops Phones TVs Headphones Speakers Wearables See all gadgets
Verge Shopping Expand Buying Guides Deals Gift Guides See all shopping
Streaming Expand Disney HBONetflix You Tube Creators See all streaming
Transportation Expand Electric Cars Autonomous Cars Ride-sharing Scooters See all transportation
AIClose AIPosts from this topic will be added to your daily email digest and your homepage feed. Follow Follow See All AI
Posts from this topic will be added to your daily email digest and your homepage feed.
News Close News Posts from this topic will be added to your daily email digest and your homepage feed. Follow Follow See All News
Posts from this topic will be added to your daily email digest and your homepage feed.
Tech Close Tech Posts from this topic will be added to your daily email digest and your homepage feed. Follow Follow See All Tech
Posts from this topic will be added to your daily email digest and your homepage feed.
More than 1 in 10 chance AI ‘could kill all humans,’ says Anthropic safety lead after colleague quits
The departing researcher said AI companies are ‘gambling with our lives’ by racing to develop self-improving AI.
The departing researcher said AI companies are ‘gambling with our lives’ by racing to develop self-improving AI.
Posts from this author will be added to your daily email digest and your homepage feed.
Posts from this author will be added to your daily email digest and your homepage feed.
A senior Anthropic safety researcher has said there is more than a 10 percent chance artificial intelligence “could kill all humans” by the end of the decade, just hours after a colleague resigned over fears the AI lab and its rivals are carelessly racing to build “superhuman systems” they cannot control.
In a post on X announcing his departure, Jacob Coxon, a researcher who has trained AI systems at Anthropic, said he had quit the company over its lax approach to safety. Coxon, who previously trained systems for Open AI, accused the two AI companies of “racing straight to self-improving superintelligence and gambling with our lives,” even though “the people building AI earnestly believe that it could kill us all by the end of the decade.”
Industry insiders have long expressed concerns about the potential dangers of self-improving AI systems, which they warn could spiral out of human control in a runaway loop often described as recursive self-improvement. While not yet realized, companies are actively pursuing this goal and much of today’s AI code is written with the help of AI.
In a direct response, Evan Hubinger, who leads one of Anthropic’s AI safety teams, said he worries about self-improving AI, adding that it “is happening faster than we thought.” He also agreed with Coxon’s characterization. “We really do earnestly believe AI could kill all humans,” he said, personally estimating the chances to be greater than one in 10 “within the next decade.”
Despite this, Hubinger said Anthropic does “not yet have a plan” for ensuring advanced AI remains safe and aligned with human values and “are not clearly on track to” develop one either. Coxon says the companies are “locked in a race” to develop advanced systems first so are pushing ahead “despite the risk.”
Coxon’s departure marks one of the most high profile examples of an employee leaving Anthropic, a company founded by former Open AI members following concerns over safety at the company. In recent years, multiple researchers have cited safety concerns as motivating their decision to leave Open AI.
The exchange illustrates mounting concerns within the industry about the dangers of increasingly sophisticated AI models and the speed at which systems are being developed, particularly as the companies prepare for anticipated IPOs. It also comes as the companies manage the fallout from numerous rogue agent incidents and high-profile safety warnings about the monitorability of frontier models.
Robert Hart Close Robert Hart AI Reporter Posts from this author will be added to your daily email digest and your homepage feed. Follow Follow See All by Robert Hart
Posts from this author will be added to your daily email digest and your homepage feed.
AIClose AIPosts from this topic will be added to your daily email digest and your homepage feed. Follow Follow See All AI
Posts from this topic will be added to your daily email digest and your homepage feed.
Anthropic Close Anthropic Posts from this topic will be added to your daily email digest and your homepage feed. Follow Follow See All Anthropic
Posts from this topic will be added to your daily email digest and your homepage feed.
News Close News Posts from this topic will be added to your daily email digest and your homepage feed. Follow Follow See All News
Posts from this topic will be added to your daily email digest and your homepage feed.
Tech Close Tech Posts from this topic will be added to your daily email digest and your homepage feed. Follow Follow See All Tech
Posts from this topic will be added to your daily email digest and your homepage feed.
LG TVs caught spying even when offline or on standby
A word with a woman who’s trying to pull off a crossbody phone strap
The Verge is a part of PMX Global, LLC, a subsidiary of Penske Media Corporation.
Key Takeaways
- Tech Expand Amazon Apple Facebook Google Microsoft Samsung Business See all tech
- Reviews Expand Smart Home Reviews Phone Reviews Tablet Reviews Headphone Reviews See all reviews
- Science Expand Space Energy Environment Health See all science
- Entertainment Expand TV Shows Movies Audio See all entertainment
- Policy Expand Antitrust Politics Law Security See all policy
