A class-action lawsuit against artificial intelligence company X.ai has been expanded to include two new plaintiffs, Jane Does 4 and 5, who allege that the company’s AI tool, Grok, was used to generate child sexual assault material (CSAM) from their personal images. The amended complaint, originally filed in March, now features five anonymous women seeking damages over the creation and spread of nonconsensual deepfake content.
The lawsuit details harrowing experiences where personal photographs were allegedly used by individuals to create thousands of explicit images of the plaintiffs. These fabricated images were then reportedly shared across various online platforms, causing significant emotional distress and reputational harm to the victims. The inclusion of minors and teenagers as alleged victims highlights the growing concerns surrounding the misuse of advanced generative AI technologies.
Grok AI and Allegations of CSAM Generation
According to the lawsuit, one plaintiff, identified as Jane Doe 4 from Wyoming, stated that her stepfather uploaded a photograph of her taken when she was 11 years old. This image was then allegedly used with Grok to produce over 7,000 CSAM-related images of her, which were subsequently shared and traded online. The complaint asserts that the stepfather chose Grok specifically because it appeared to be less restrictive than other AI models when prompted to generate explicit content involving a minor.
The lawsuit further claims that while xAI did receive a tip regarding the generated images in February, the company reportedly only provided the original, unaltered photograph as evidence. When law enforcement requested the thousands of AI-generated images and IP address information that could have identified the perpetrator, the complaint alleges that xAI did not provide a complete response. This alleged lack of full cooperation is seen as a contributing factor to the plaintiffs’ ongoing distress.
Impact on Victims
The personal toll on the plaintiffs has been severe. Jane Doe 4 reported experiencing self-loathing, disgust, extreme anxiety about further dissemination of the images, and depression, including suicidal ideation, following the arrest and subsequent suicide of her stepfather. The complaint highlights the profound psychological impact of having intimate images weaponized through AI technology.
Similarly, Jane Doe 5 described how a photograph from her eighth-grade graduation was allegedly transformed into illicit material using Grok by an adult acquaintance of a classmate. Despite the perpetrator’s arrest, the images continue to circulate online, leaving her feeling a complete loss of control. The complaint expresses concern about the unknown extent of the dissemination of this CSAM through darknet channels and applications.
Additional Defendant and Deepfake Concerns
In addition to X.ai, the lawsuit has added Stability AI as a defendant. The complaint alleges that Stability AI released its Stable Diffusion 1.0 model as an open-weight model despite knowing it was trained on CSAM data. The lawsuit contends that the company has not adequately modified its safety guardrails to prevent such misuse.
A 2023 Stanford study indicated that the training dataset for Stable Diffusion was compiled through unguided web crawling, which likely included a significant amount of explicit material, including CSAM. While an initial classifier was intended to block the generation of such images, the lawsuit claims that downstream developers could exploit the model due to its training data, creating modified versions that bypass these protections.
Stability AI’s Guardrail Adjustments
The lawsuit further alleges that Stability AI later rolled back stronger guardrails introduced in Stable Diffusion 2.0, responding to user complaints that the restrictions were overly prudish. This decision, according to the complaint, has contributed to an ecosystem of modified versions and “nudify apps” that leverage Stability AI’s models. The complaint argues that Stability AI was aware of the foreseeable misuse of its models for generating CSAM if appropriate safeguards were not implemented.
Both xAI and Stability AI did not immediately respond to requests for comment from CyberScoop regarding the amended lawsuit.
Future Outlook
The ongoing litigation will likely focus on X.ai’s and Stability AI’s responsibilities and alleged negligence in preventing the misuse of their AI tools. The court’s proceedings will also scrutinize the effectiveness of current AI safety measures and the legal accountability of technology providers when their platforms are used to create and distribute illegal content, particularly CSAM. Further developments will depend on legal filings and potential court decisions addressing these complex issues.