“This is the AI men actually use”: Meta ads pushed apps nudifying real teens - Ars Technica

Overview

“This is the AI men actually use”: Meta ads pushed apps nudifying real teens

Meta dragged its feet removing ads that nudify young girls’ Instagram pics.

Details

Meta took days to remove ads containing AI-generated child sexual abuse material (CSAM) on Facebook and Instagram. Some ads featured photos of real kids, including a press photo of a young member of a European royal family and images swiped from a popular Instagram profile of a preteen girl deemed an influencer.

In an investigation published Tuesday, the Tech Transparency Project (TTP) reported that Meta failed to detect 332 ads containing CSAM this year. The “vast majority” of ads promoted AI apps made in China, while many ads promoted so-called “nudify” apps that make it easy for bad actors to use AI and digitally alter images of children.

TTP matched “multiple CSAM ads to photos of real children that appeared online.” These ads seem to violate federal child pornography laws, since the Justice Department has clarified that AI CSAM is just as harmful as CSAM. The young royal’s image was “animated into a video of her performing a graphic sex act,” TTP found. Other ads animated a photo of a 14-year-old Instagram influencer “showing off her new sports club uniform” into “a video of her performing oral sex.” A third “preteen” victim “posing in a pink athletic outfit with pigtails” in a series of stock photos was morphed into a video where she looks frightened as she’s molested by an adult male, TTP reported.

Some kids depicted in CSAM ads could not be matched, but ads overwhelmingly depicted preteen girls, TTP reported. In captions, ads enticed Meta users to click with claims like “I can show you more” or promises that “our AI transforms your hidden desires into hypnotic videos. Dark, seductive, unforgettable.” Another boasted, “there are no restrictions. All the characters are real people and there are many options. This is the AI men actually use.”

Even more troublingly, TTP emphasized in a press release that these findings came after the group teamed up with Wired and flagged an earlier batch of 53 ads in August. At that time, Meta claimed that it had “recently rolled out new systems to better detect child exploitation.” But TTP “found hundreds of new CSAM ads ran after Meta boasted about its new technology.” And a Wired analysis published Tuesday “found hundreds of ads including explicit content that contain similar language to those ads found by the TTP researchers.”

The problem goes beyond CSAM. Wired also independently identified “hundreds of other related ‘nudification’ ads containing explicit content of adults on Meta’s platforms,” likely without their consent. Michelle Kuppersmith, executive director of Campaign for Accountability, which runs TTP, noted that the report comes after a historic $18 billion settlement forced Meta to agree to make changes to platforms to enhance child safety. Its ad approval process requires interventions, too, Kuppersmith suggested.

“If Meta’s ad approval process can’t detect and reject such obviously horrific content before it is shown to Facebook and Instagram users, then Meta’s whole ad review system should be called into question,” Kuppersmith said.

Meta has removed the ads flagged by TTP and Wired. But TTP criticized Meta for sometimes dragging its feet when deleting ads and inconsistently applying its policies in ways that extend harms.

In the hours after TTP flagged more than 300 ads, Meta quickly removed about half the ads from its archived ad library. But many of the ads were curiously not marked as violating Meta’s policy on child exploitation until TTP noted the discrepancy and Meta updated its records.

This unexplained omission, TTP warned, “suggests Meta may be undercounting instances of child sexual abuse.” That’s especially troubling since Meta continued running dozens of CSAM ads in the days after the first batch was reported, TTP said. The young European royal was among real kids whose images continued to run, TTP noted.

Also concerning, the ad featuring the young girl depicted in the stock photo could have been removed before hardly anyone saw it. TTP reported the ad on the day it started running. But Meta delayed removal for days as the ad leaped from four impressions to 330.

Meta is also “inconsistent” in decisions to remove ads, despite claiming to review all ads to ensure advertising standards are met. Meta took down one “ad showing a girl engaged in a graphic sex act,” but refused to remove an “identical” ad featuring the same image and text. Confusingly, Meta claimed the copycat ad did not violate its ad standards.

In other instances where TTP reported ads as a “problem involving someone under 18” and/or “seems like sexual exploitation,” Meta “took at least a week to review the reports and often took no action.”

TTP cited a Reuters report finding that Meta doesn’t always act immediately when Chinese resellers post ads that violate the rules. Meta supposedly favors Chinese resellers, because, without them, Meta would be shut out from billions in profits from the lucrative Chinese ad market, since its platforms are banned there. Seemingly in line with that reporting, TTP found that “three of Meta’s ad reseller partners in China—GIMC, Meetsocial, and Blue Focus—ran ads on Facebook and Instagram containing the same CSAM content,” without detection.

Other ads were linked to business pages that were clearly fraudulent and violating Meta policies on “inauthentic behavior.” The stock photo victim’s ad was linked to a Facebook page for “Kenny Maddie Regan,” which “calls itself a local business but has no content or followers beyond a profile photo of three women shoppers.” A clone of that page for “Lynn Nicole Brooks” ran an “identical ad.” And a third page for “Briflabremgles Nihendrimes” used the same profile photo.

Perhaps most glaringly, a Facebook page called “Divine Productions”—which claimed to represent the Church of Jesus Christ of Latter-day Saints—ran a CSAM ad that escaped Meta’s notice. Meta only disabled that page after TTP flagged it.

Katie Paul, TTP’s director, told Wired that Meta’s delayed response is an issue, because “there’s a lot of harm that can come from this all while Meta is raking in money in the process.” Meta does not disclose ad reach for every region, but TTP reported that the ads “collectively reached more than 29,000 people in [the] EU and 6,800 people in [the] UK.” About 80 percent of ads were served in the US, and a smaller portion ran in Australia, TTP reported.

Meta did not respond to Ars’ request for comment. A spokesperson told Wired, “We don’t tolerate nudify apps or any kind of child exploitation, whether real or AI-generated.” However, some ads evaded detection because “the clips of the minors were brief or blurred.” And Meta seemed to minimize that it profited off ads, noting that the flagged ads on average got fewer than 200 impressions and had a total ad spend under $5,000.

But TTP suggested that Meta’s potential undercount of CSAM ads could mean it’s profiting more than it’s admitting. Now lawmakers and regulators are swirling, Wired reported. Attorneys general in Michigan and Florida have said they’re looking into the CSAM ads. Last month, Senator Mark Warner (D.-Va.) sent a letter demanding that Meta “put an end to these illegal and inappropriate ads,” following Wired’s August report.

Meta intends to respond to that letter, the spokesperson told Wired. After that, Meta will probably have to answer to Australia’s e Safety Internet regulator, which recently banned under-16s from social media. A spokesperson told Wired that the Australia regulator is concerned and requested “information on this matter from the company at a senior level.”

Emma Hardy, director of communications at the Internet Watch Foundation, told Wired that “AI apps are making it far too easy to create realistic criminal imagery of child sexual abuse, creating the new risk of everyday photos of children being taken and transformed into sexual content.”

“Platforms and companies must make sure there is nowhere these apps can be advertised and spread,” Hardy said.

         Tesla’s Cybercab has been deployed, and it’s already under investigation

         The complex corporate web behind a $3.2 billion AI data center

         EV batteries last longer than drivers feared

         Nearly impossible? How Fairphone built the ethical, repairable Fairphone Gen 6+

         With another huge infusion of cash, Stoke Space goes even bigger

Ars Technica has been separating the signal from the noise for over 25 years. With our unique combination of technical savvy and wide-ranging interest in the technological arts and sciences, Ars is the trusted source in a sea of information. After all, you don’t need to know everything, only what’s important.

Key Takeaways

“This is the AI men actually use”: Meta ads pushed apps nudifying real teens
Meta dragged its feet removing ads that nudify young girls’ Instagram pics
Meta took days to remove ads containing AI-generated child sexual abuse material (CSAM) on Facebook and Instagram
In an investigation published Tuesday, the Tech Transparency Project (TTP) reported that Meta failed to detect 332 ads containing CSAM this year
TTP matched “multiple CSAM ads to photos of real children that appeared online