China silent as US touts plan for AI safety alerts that omits tech experts - Ars Technica

Overview

China silent as US touts plan for AI safety alerts that omits tech expertsvar abtest_2173411 = new ABTest(2173411, 'click');

Trump focus on winning “AI race” may deter China from sharing safety intel.

Details

This week, the US and China, the two world leaders on AI, finally started talks meant to keep the whole world safe from emerging safety risks. But whether meetings between Donald Trump and Xi Jinping can lead to a reliable global governance framework for AI depends on how much the fierce rivals are actually willing to cooperate.

It’s clear Trump wants it to seem like discussions are going well. Before Trump and Xi meet on Thursday and Friday, Treasury Secretary Scott Bessent announced that both sides had already discussed setting up a new AI safety notification mechanism. If China agrees to participate, that would create an open line for either side to send alerts when their country’s AI systems pose threats or behave unpredictably. The same channel, Bessent said, could allow the countries to share their “vision of common goals.”

However, the proposal comes at a time when China may not be ready to trust the US to have AI safety conversations that aren’t about limiting China’s capabilities. Not only has Trump steadily increased export controls that explicitly seek to block China from accessing technology to advance AI, but Congress is mulling bills that would cut the country off from more chips and hardware. Adding to tensions, just two weeks ago, Trump’s Justice Department accused six of China’s leading AI makers of stealing from US frontier labs, which China has strongly disputed.

Some US-China experts told Ars that there were concerns that China might cancel talks, but China seemingly isn’t taking offense to Trump’s rhetoric. Even Truth Social posts where Trump insists that America must dominate China on AI—declaring “WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so”—didn’t keep China from showing up.

Notably, China has yet to publicly acknowledge the proposed alert system. Even if China bought into the system, people familiar with Bessent’s plan told Politico that the plan to monitor safety alerts appears superficial, since it involves no officials with technical expertise who could be trusted to assess the urgency of risks described in communications.

Describing the significance of the arrangement, one source mocked the plan, saying “the scientific, technical term is BS.”

Sam Bresnick, a research fellow at a Georgetown University think tank called the Center for Security and Emerging Technology, monitors national security applications of AI and US-China tech competition. He agreed it’s a red flag that there’s no tech body of tech experts monitoring the channel.

“It’s an existing channel, which I think is better than creating a whole new channel,” Bresnick told Ars. But if both sides had tech experts, the US and China could have really high-level conversations about how they’re assessing risk. That’s particularly important when discussing military risks, Bresnick emphasized.

“I think it’s better than nothing for sure, but I don’t have a lot of confidence that that is going to make a major difference going forward,” Bresnick said.

Su Lian Jye, a chief analyst at a global technology research and advisory group called Omdia, told the South China Morning Post that the AI alerts may help “promote more transparency,” but it won’t make either side less wary of the other.

“I don’t think the dialog will remove all concerns and differences on distillation, pricing competition, and pacing,” Su said.

Praising the plan as a good start to open dialogues, Su suggested that much more cooperation would be needed to make the safety mechanism meaningful.

“Turning the proposal into an effective tool would require both countries to agree on risk classifications, notification triggers, response protocols, and regular review procedures,” he said.

Perhaps that will come up when Trump and Xi meet later this week, with AI supposedly at the top of the agenda.

It’s unclear what will be discussed, but critics concerned about the most urgent AI safety risks fear that Bessent’s announcement indicates that Trump has modest ambitions for the summit.

On CBS’s “Face the Nation,” US Representative Ro Khanna—the ranking Democrat on the US House Select Committee on ⁠Strategic Competition Between the United States and the Chinese Communist Party—suggested that talks between Trump and Xi should go much further than setting up a line between two officials with no tech expertise.

He argued Trump should be asking Xi for an international agreement that urgently includes “banning any recursive self-improving AI” and “safeguards so that AI can’t be misused by people for biological nuclear weapons.”

Many experts described sharing safety alerts as a decent first step in improving US-China coordination on AI, which Trump’s incoming “AI czar” could then possibly build on.

Alvin Wang Graylin, a global technology strategist and digital fellow at Stanford Digital Economy Lab, has spent decades weighing geopolitical risks. Most recently, he’s advised governments and tech leaders on AI policy and global governance, including sharing recommendations on how to set up a notification system like the one Bessent proposed.

Graylin told Ars that he was surprised to see that such a “positive” development had already come out of this week’s talks with China. And it seemed encouraging that both countries already have another meeting scheduled in November.

“What that shows is that both sides are seeing value in the discussion,” Graylin said. “Otherwise, there would not be a follow-on.”

Graylin suggested the US and China share many AI risks. Not just the supposedly existential risk that AI leaders started recently claiming required a slowdown, but also risks from bad actors like hackers or terrorists wielding AI to attack either country’s infrastructure.

There’s much that the US and China can learn from each other if they talk seriously. Graylin said that China regularly audits AI models and could share knowledge from hundreds of audits, since the US has only just begun auditing AI. What China could gain from the US is perhaps more experience auditing frontier models.

“I think both sides are going to come to the realization” that controlling AI risk will “require cooperation,” Graylin said.

Bresnick suggested that ongoing talks could be an opportunity to “explore how each side views risks.” While the US has been combating public fears of uncontrollable AI, in China, “there’s just not the same fear right now about AI,” Bresnick said, largely because the state feels that it can control AI.

That may change as China’s advanced AI catches up to the US, Bresnick said. He expects that US-China discussions could ramp up in the next six months to a year, as China starts to see the same level of emerging capabilities.

However, for now, the very different way that China views risk may deter America’s biggest rival from coming to the table to talk about risks from terrorists or hackers, Bresnick said. Especially if the US continues tying the AI safety discussion to keeping America ahead.

When Trump or CEOs like Sam Altman advocate for American-led global AI standards, all China probably sees is the US wielding safety discussions as a way to lock in its advantage, he said. Those concerns may even render Bessent’s alert system useless in the long term if China feels that “AI safety” is a code word for suppressing Chinese AI.

“I think there are reasons [for China] to view this notification mechanism with suspicion,” Bresnick said.

Graylin has warned that the US is too caught up in pushing a “deeply flawed” narrative about “winning” an “AI race” that most likely will benefit no one.

In reality, there are at least two races, one for intelligence and one for market dominance. Currently the US is making more money on frontier AI but is falling behind in the race for users globally. In the last six months, adoption of China’s open models has exploded, with Graylin predicting some might soon exceed 80 percent market share. For many CEOs cracking down on IT budgets, open models from Chinese firms offer businesses cheaper alternatives to frontier AI that better protect the privacy of internal and customer data.

The US cannot afford to ignore this other AI race while focusing only on a fake finish line that will never come in frontier AI, Graylin wrote in a September article for a nonpartisan think tank, the Quincy Institute.  Graylin expects it might be impossible for any country to keep a “durable and possibly permanent lead” that would allow them to reshape the global balance of power, as Big Tech CEOs with Trump’s ear claim could happen.

“US-China capability gaps at the frontiers of AI technology typically last only a few months, and there is no competitive ‘moat’ that will secure a permanent advantage to either nation,” Graylin wrote. Further, US safety testing of frontier models delays access to technology, while giving rivals time to catch up. And deploying any sooner would create even more safety risks.

“Even if we achieve [artificial superintelligence] first and can somehow expedite the certification process, unless we are willing to deploy it untested in a pre-emptive all-out strike against China, the temporal advantage could evaporate in a matter of weeks or months,” Graylin wrote.

To Graylin, the US pretending that China achieving artificial superintelligence first would pose the “greatest AI security threat to the United States” seems flawed, and it’s obstructing cooperation that would help both countries prepare for actual imminent AI risks. Those include threats to workers, national economies, and national security.

“The most dangerous models are often those smaller specialized ones that could be created or utilized by non-state rogue actors for the purpose of deploying biological, chemical, or cyber weapons,” Graylin wrote. “The assumption of US-China conflict prevents the exact global cooperation to protect against this very real threat.”

Graylin suggested that the US could take steps to ease public fears globally about AI, but some of them might seem extreme in Trump’s eyes.

Along with sharing high-level intelligence and opening “bilateral and multilateral channels on AI safety,” Trump might consider restoring Joe Biden’s “small yard” and loosening an ever-widening range of export controls that are likely currently keeping China from conducting better safety testing. If export controls were narrowed to only block access to “genuine military and CBRNE threats” and “not all advanced technology,” then China might deploy safer AI products, he suggested.

Additionally, Trump could consider following China’s lead and softening his AI deregulation stance. By enacting reasonable safety regulations, he could possibly start to “lessen the growing societal fear over AI that’s been brewing domestically,” which Graylin wrote, “is certain to hinder the nation’s ability to deploy this technology effectively into the economy and reduce our global competitiveness.”

“We cannot manage the multifaceted risks of the most powerful technology in human history while refusing to talk to the other nation building it,” Graylin wrote.

In a perfect world, this week’s meeting between Trump and Xi would one day lead to the US abandoning this narrative of “winning an AI race,” Graylin said.

“There is no way to compete yourself to a safer world,” he told Ars.

1.          Woman's brain worm infection confirmed after eggs grow tails in lab test
   

2.          New Anthropic, Open AI models make same promise: A little more for a lot less money
   

3.          Toyota orders workers to train humanoid robots but says humans won't be replaced
   

4.          Aftermarket driver assist under federal probe following fatal crashes
   

5.          Cities across US oppose Trump FCC plan to preempt local broadband rules
   

Ars Technica has been separating the signal from the noise for over 25 years. With our unique combination of technical savvy and wide-ranging interest in the technological arts and sciences, Ars is the trusted source in a sea of information. After all, you don’t need to know everything, only what’s important.

Key Takeaways

• China silent as US touts plan for AI safety alerts that omits tech expertsvar abtest_2173411 = new ABTest(2173411, 'click');

• Trump focus on winning “AI race” may deter China from sharing safety intel

• This week, the US and China, the two world leaders on AI, finally started talks meant to keep the whole world safe from emerging safety risks

• It’s clear Trump wants it to seem like discussions are going well

• However, the proposal comes at a time when China may not be ready to trust the US to have AI safety conversations that aren’t about limiting China’s capabilities