Elon Musk’s x AI used child porn to train Grok models, lawsuit says - Ars Technica
Overview
Elon Musk’s x AI used child porn to train Grok models, lawsuit saysvar abtest_2169474 = new ABTest(2169474, 'click');
x AI accused of training Grok on real and AI-generated child pornography.
Details
x AI has now been accused of training Grok on child sex abuse materials (CSAM), as regulators and courts continue to probe how far the problem goes, and some Grok users have been arrested.
In a complaint filed on Wednesday, a plaintiff known as Jane Doe explained that she was preschool-age in the early 2000s when adult men repeatedly raped her to create CSAM to sell to pedophiles online. Since then, Doe’s images have been hashed by groups like the National Center for Missing and Exploited Children (NCMEC) and the Canadian Centre for Child Protection (CCCP).
For her safety, Doe has opted to receive alerts from the US Department of Justice Victim Notification System any time she may be a victim in a new criminal investigation. Although she has received countless alerts, she was shocked when the CCCP notified her that it had identified AI-generated CSAM on x AI that depicted her. This re-traumatized Doe, whose complaint alleged that messages were found on online forums “between offenders chatting about creating AI generated CSAM of Plaintiff and other similarly situated known, legacy, victims of CSAM.”
Now, Doe fears that x AI has not only made it easier to make more violative images of the most distressing time in her life, but also that x AI allegedly has stored the images that Grok generates and uses those outputs to further train Grok. Because of this, she believes that Grok has been trained on both the initial set of images that have haunted her for more than 20 years and the more recent AI-generated ones.
This is the first case to accuse x AI of training on CSAM, and the complaint does not go into great detail on that claim. Previously, Ars reported on a controversial dataset that was later scrubbed after researchers found CSAM in the training data, but there’s no indication x AI trained on that data. In a press release from lawyers representing Doe, it’s explained that Doe’s images were included in a CSAM Hash List maintained by NCMEC, and “that same material” allegedly “was part of the dataset x AI used to build Grok’s image and video generating capabilities.” The complaint similarly only alleged that “CSAM depicting Plaintiff with its longstanding well-known hash values has been used as a part of the dataset used by x AI.”
More detail is shared on claims alleging that Grok trains on AI CSAM, however.
“Because Grok’s terms treat public X posts and Grok’s own outputs as training data by default, publicly posting an image does not just expose it to viewers, but also feeds [it] directly into the pipeline x AI uses to train and improve its model and thereby generate further images,” Doe’s complaint said.
The lawsuit further noted that while x AI filters out violent content in Grok outputs to exclude it from training data, x AI’s terms notably do not specify if CSAM, non-consensual intimate imagery (NCII), or NSFW material are “excluded categories.”
It seems to follow then that “because full removal of a training example’s influence from an already-trained model is technically difficult and not something that x AI has publicly claimed to have done, any CSAM ingested into training before takedown likely continued to shape the model’s outputs even after the original images were removed from public view,” Doe’s complaint said.
In her proposed class action, Doe seeks to put an end to Grok’s harmful outputs. If approved, the class would represent every victim whose childhood images have been used to generate Grok CSAM.
The lawsuit accuses X of violating federal child pornography laws, as well as Masha’s Law, both of which give CSAM survivors a right to sue on claims of production, possession, and distribution. A lawyer for Doe, Margaret E. Mabie, suggested in the press release that all three are crimes and “x AI did all three.”
If Doe wins, x AI could owe money damages to every victim who can prove that Grok generated CSAM based on their real photos. Doe’s also asked the court to order x AI to destroy all Grok-generated CSAM that it may be storing on its servers and using to train Grok models. Additionally, she wants x AI to block Grok from ever generating CSAM, which her complaint suggested would require blocking Grok from any sexualized outputs, including NCII and NSFW “bikini pics” that Musk has promoted.
Sarah London, one of Doe’s lawyers, said in the press release that Doe “has lived for nearly two decades knowing that images of the worst thing that ever happened to her are circulating among predators online, and that they can resurface at any moment. x AI must be held responsible for knowingly training its models on images of the horrific abuse she suffered, and on the abuse images of every other survivor in this class.”
-
New Twitter launches, says Musk's X gave up the name -
AI agents meant to replace Meta workers made “large-scale, disruptive actions” -
Claude, Codex, and Hermes installed unowned code inside corporate networks -
Florida Catholics slap down state AG by rejecting religious vaccine exemptions -
RIP, Tim Curry: Ars remembers his top 10 iconic performances
Ars Technica has been separating the signal from the noise for over 25 years. With our unique combination of technical savvy and wide-ranging interest in the technological arts and sciences, Ars is the trusted source in a sea of information. After all, you don’t need to know everything, only what’s important.
Key Takeaways
-
Elon Musk’s x AI used child porn to train Grok models, lawsuit saysvar abtest_2169474 = new ABTest(2169474, 'click');
-
x AI accused of training Grok on real and AI-generated child pornography
-
x AI has now been accused of training Grok on child sex abuse materials (CSAM), as regulators and courts continue to probe how far the problem goes, and some Grok users have been arrested
-
In a complaint filed on Wednesday, a plaintiff known as Jane Doe explained that she was preschool-age in the early 2000s when adult men repeatedly raped her to create CSAM to sell to pedophiles online
-
For her safety, Doe has opted to receive alerts from the US Department of Justice Victim Notification System any time she may be a victim in a new criminal investigation
