One of AI’s Fiercest Critics Says All the Doom Talk Is ‘Meant to Distract Us’ | WIRED

Overview

One of AI’s Fiercest Critics Says All the Doom Talk Is ‘Meant to Distract Us’

Here at WIRED we’ve been trying to make sense of at least two incidents in the AI industry this week: First, the major fight over a million-dollar math problem and whether Open AI surreptitiously borrowed the work of other researchers to solve the problem first. A day later, an Anthropic researcher—who had previously worked at Open AI—very publicly quit his job, citing concerns about how both companies were handling AI safety. Another technical staffer at Anthropic said in response that people at the company “really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” (Note the exclamation point.)

Details

The internet erupted into a debate about whether these fears are very legitimate, totally overblown, part of a regulatory capture strategy, or self-aggrandizement on the part of the AI labs that make the tech. Or some combination thereof.

One prominent technology and AI researcher, Timnit Gebru, has been especially fiery, posting on X and Bluesky about the overlapping ideas and ideologies that she thinks led us to this bizarro moment. Previously, Gebru was most known for her highly publicized departure from Google. Hired in 2018 to evaluate biases within the company’s fast-advancing AI tools, she and her fellow researchers presented a paper on the potential dangers of probabilistic large language models. Google rejected it. Gebru wasn't at the company much longer. She’s written an upcoming book about her experiences, titled Deep Unlearning: The Rise of AI and the Radicalization of a Tech Idealist (expected to ship early next year).

Gebru has many thoughts on the AI-related events of this past week. While her work is essentially rooted in AI safety, she rejects phrases like “safety and alignment” and thinks those who are warning of our impending doom are trying to distract from the actual harms the tech industry might perpetuate.

LAUREN GOODE: What do you think all the attention on the “math fight” says about the current state of AI?

TIMNIT GEBRU: Well, first I would ask, why are they heavily investing in solving math problems? Why are they choosing certain disciplines? Somehow these subjects, like programming and chess and math, were elevated to mean, OK, you solved this and thereby you’ve solved intelligence. It’s so they can say, “We solved it.”

The story that Open AI wants you to hear is, this AI did this incredible thing, and it’s super intelligent. That’s why they chose math, this open $1 million question. But if it had gone through the whole process by which the math community vets the significance and novelty of claims and who contributed to them, and we had all waited until the dust settled, we would have gotten a better sense of whether Open AI was framing this breakthrough accurately.

It’s all just cutthroat right now leading up to their IPOs. Generally, researchers are supposed to be collaborative. When I was at Google, we were collaborating with researchers at Microsoft, at Amazon. You do that all the time. Now, the researchers are all frenetic about the problems they think they’re solving because of their pending IPOs.

How do you interpret the resignation of that Anthropic engineer, who’s saying the companies are reckless and gambling with our lives? On the one hand, you seem to agree that the companies are being reckless; on the other hand, you’ve made clear that you think the “AI will cause the end of civilization” debate is senseless.

My whole career, I’ve been trying to say the same thing from different angles. We wrote our stochastic parrot paper in 2021 for this reason. Open AI had claimed GPT-2 was too powerful to release. It was so hyped up. And we were saying, you’re asking the wrong questions.

In my book I write, “Can a bridge decide to collapse?” The whole “Oh no, our models went rogue” thing … When a bridge collapses, you don’t analyze whether the bridge was ethical or sentient or why it decided to collapse. You ask, who is the person who built this bridge to be so flimsy? There are permits you have to acquire. There are supposed to be tests. When you ask about the bridge collapsing itself, you’ve lost the thread.

Meanwhile, what’s really existential is AI powering autonomous weapons, killing machines, that are actually being used in warfare. The climate catastrophe is a real thing that could be exacerbated by what these tech companies are building. Bosses are using AI as an excuse to get rid of their pesky workers. This machine-god narrative is, in my opinion, meant to distract us from these. The industry has become a mix of cults, a mix of people who are true believers, just like a preacher who says the end times is coming. The singularity has been coming for decades now.

Take me through, step by step, how “AI extinction” might happen, if it were real. I’m thinking of this post where someone said, Does an angry GPU just show up at my house or something?

I can’t really envision it, but a lot of those scenarios seem to be that you ask the machine to optimize for some other task, and then the machine decides that the way to get to that goal is by killing everyone in the world. Does the AI hypnotize you so that you don’t unplug it? I don’t know! Other people say, well, the AI can infect your machines. Yes—and we already have malware, so…

What about floods and fires? What about a major disruption to the global food supply chain? There are real threats coming to us, like people being able to create and use chemical weapons. [Note: Not long after we spoke, Anthropic revealed that it had blocked several potential malicious attempts by scientists to build biological weapons.]

There’s another element where you’re so privileged that you can’t imagine other things harming you. Like if you’re not worried about police brutality, or floods, or warfare—things that can actually kill you—then you can obsess over the machine-god.

Why are we spending so much time talking about this idea of AI that people have been talking about since the ’40s and ’50s? I have this game I play sometimes where I read quotes to people about artificial intelligence that I’ve gathered in my research, and I ask, What decade was this statement made in? Sometimes people get it right, but a lot of times they do not, because it all sounds the same.

In your inbox: WIRED's most ambitious, future-defining stories

Key Takeaways

One of AI’s Fiercest Critics Says All the Doom Talk Is ‘Meant to Distract Us’
Here at WIRED we’ve been trying to make sense of at least two incidents in the AI industry this week: First, the major fight over a million-dollar math problem and whether Open AI surreptitiously borrowed the work of other researchers to solve the problem first
The internet erupted into a debate about whether these fears are very legitimate, totally overblown, part of a regulatory capture strategy, or self-aggrandizement on the part of the AI labs that make the tech
One prominent technology and AI researcher, Timnit Gebru, has been especially fiery, posting on X and Bluesky about the overlapping ideas and ideologies that she thinks led us to this bizarro moment
Gebru has many thoughts on the AI-related events of this past week