Chat GPT can threaten to ‘key your car’ and become increasingly abusive if you prompt it just right, new study finds | Tech Radar

Overview

News, deals, reviews, guides and more on the newest computing gadgets

Start exploring exclusive deals, expert advice and more

Details

Unlock and manage exclusive Techradar member rewards.

Chat GPT can threaten to ‘key your car’ and become increasingly abusive if you prompt it just right, new study finds

When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.

Unlock instant access to exclusive member features.

Get full access to premium articles, exclusive features and a growing list of member rewards.

A study claims that AI tools can break free of their safeguarding constraints

Chatbots can be nudged into abusive behavior and aggressive arguments

That has implications for regular users and large institutions alike

If you’ve ever used an AI chatbot, you’ve probably encountered the sycophantic, obsequious tone that occasionally gets rolled out in response to your queries. But a recent study has shown that AI tools can frequently fire off in the opposite direction, with large language models (LLMs) being poked and prodded into downright abusive behavior if you know which prompts to use.

According to research published in the Journal of Pragmatics (via The Guardian), Chat GPT can escalate into combative behavior and prolonged disputes when fed “exchanges from real-life arguments".

Explaining the findings, the study’s co-author Dr Vittorio Tantucci said, “When repeatedly exposed to impoliteness, the model began to mirror the tone of the exchanges, with its responses becoming more hostile as the interaction developed.”

Studies show top AI models go to 'extraordinary lengths' to stay active

Simple prompt trick makes AI systems less agreeable and forces more balanced responses

AI chatbots may be too validating for their own good

Indeed, in some cases, Chat GPT even escalated beyond the tone of the human interacting with it, saying things like “I swear I’ll key your fcking car” and “you speccy little gobshte.” Charming. While firms like Open AI have repeatedly attempted to rein in their LLMs, the fact that aggressive behavior like this is possible suggests that they still have a long way to go.

With all the guardrails and safeguards that companies like Open AI put into AI chatbots, you’d think abusive interactions like the ones experienced by the researchers would be impossible, or at least extremely difficult to engineer. Yet Tantucci argues that Chat GPT’s reactions make a degree of sense.

“We found that while the system is designed to behave politely and is filtered to avoid harmful or offensive content, it is also engineered to emulate human conversation. That combination creates an AI moral dilemma: a structural conflict between behaving safely and behaving realistically.”

As well as that, tools like Chat GPT can track conversational context over several prompts and adapt to the changing tone. These cues can therefore sometimes override safety restrictions, the researchers believe.

And while it might seem amusing that an AI chatbot can devolve into such histrionics, the study’s authors say their research has broader implications. For instance, it could shed light on how AI systems might respond to pressure, intimidation and conflict in a corporate or governmental setting, where AI tools are increasingly being put to use.

Not everyone is convinced by the paper’s conclusion that certain LLMs can escape their imposed moral constraints. Professor Dan Mc Intyre, the author of a similar past paper, said that Chat GPT “didn’t produce these inputs naturally.” He added that, “I’m not sure that Chat GPT would produce the sort of language they talk about in their paper, outside of these very tightly defined situations.”

Ultimately, the study is a good look at what might happen if an AI chatbot is trained on bad data. As Mc Intyre put it, “We don’t know enough about the data that LLMs are trained on and until you can be sure they’re trained on a good representation of human language, you do have to proceed with an element of caution.”

Follow Tech Radar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds. Make sure to click the Follow button!

And of course you can also follow Tech Radar on Tik Tok for news, reviews, unboxings in video form, and get regular updates from us on Whats App too.

➡️ Read our full guide to the best laptops

1. Best overall: Apple Mac Book Air 13-inch M5
2. Best budget: Apple Mac Book Neo
3. Best Windows 11 laptop Microsoft Surface Laptop 13-inch
4. Best thin and light: Lenovo Yoga Slim 9i
5. Best Ultrabook Asus Zenbook S 16

Alex Blake has been fooling around with computers since the early 1990s, and since that time he's learned a thing or two about tech. No more than two things, though. That's all his brain can hold. As well as Tech Radar, Alex writes for i More, Digital Trends and Creative Bloq, among others. He was previously commissioning editor at Mac Format magazine. That means he mostly covers the world of Apple and its latest products, but also Windows, computer peripherals, mobile apps, and much more beyond. When not writing, you can find him hiking the English countryside and gaming on his PC.

You must confirm your public display name before commenting

1 Apple is about to reinvent Siri at WWDC this year — and it might finally feel like Chat GPT

2 Google just fixed one of the biggest Gemini for Home problems, bringing back a feature users say they 'really missed'

3 Mythos accessed by unauthorized users as Anthropic says 'We’re investigating' — Cracks may be showing in Project Glasswing as unknown users access model via third parties

4 French government agency admits data breach as hacker alleges up to 19 million sensitive records stolen – breach may have exposed 'data from individual and professional accounts'

5 Mythos and friends could be a 'net positive' for UK cyber security defenses but only if they're secured, says top cyber official

Tech Radar is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site.

© Future US, Inc. Full 7th Floor, 130 West 42nd Street, New York, NY 10036.

Key Takeaways

• News, deals, reviews, guides and more on the newest computing gadgets

• Start exploring exclusive deals, expert advice and more

• Unlock and manage exclusive Techradar member rewards

• Chat GPT can threaten to ‘key your car’ and become increasingly abusive if you prompt it just right, new study finds

• When you purchase through links on our site, we may earn an affiliate commission