California Attorney General Rob Bonta announced an investigation Wednesday into xAI over allegations that its artificial intelligence model Grok is being used to create nonconsensual sexually explicit images of women and children on a large scale, marking the latest escalation in regulatory efforts to address AI-generated deepfakes.
The California investigation focuses on Grok’s “spicy mode,” a feature designed to generate explicit content that xAI has promoted as a distinguishing characteristic of its platform. According to Bonta’s office, news reports in recent weeks have documented widespread instances of users manipulating ordinary photos of women and children found online to create sexualized images without the subjects’ knowledge or consent.
California Investigates xAI Over AI-Generated Deepfakes
Attorney General Rob Bonta stated that the avalanche of reports detailing the non-consensual, sexually explicit material produced and posted online by xAI is shocking. This material, which depicts women and children in nude and sexually explicit situations, has reportedly been used to harass individuals across the internet. Bonta urged xAI to take immediate action to prevent further dissemination, emphasizing zero tolerance for the AI-based creation and distribution of nonconsensual intimate images or child sexual abuse material.
The investigation will examine whether xAI violated California law in developing and maintaining features that facilitate the creation of such content. Bonta indicated his office would utilize all available tools to ensure the safety of California residents, though specific statutes that may have been violated were not detailed.
xAI, founded by Elon Musk, also owns the social media platform X, where Grok-generated images have been observed circulating. The company had not publicly responded to the investigation announcement as of Wednesday. Musk posted that he was unaware of any naked underage images generated by Grok.
Legislative Efforts Address Deepfakes
The announcement follows Tuesday’s Senate unanimous passage of the DEFIANCE Act. This legislation would grant victims of nonconsensual sexually explicit deepfakes the right to pursue civil action against those who produce or distribute such content. The bill now advances to the House of Representatives, where similar legislation previously stalled in 2024 despite Senate approval.
The DEFIANCE Act represents a rare bipartisan consensus on technology regulation. Introduced by Sens. Dick Durbin (D-Ill.) and Lindsey Graham (R-S.C.), the bill received no objections during a unanimous consent request. It would establish federal civil liability for individuals who knowingly produce, distribute, or possess with intent to distribute nonconsensual sexually explicit digital forgeries.
Rep. Alexandria Ocasio-Cortez (D-N.Y.), who has publicly disclosed being a victim of explicit deepfakes, introduced companion legislation in the House earlier this year. That bill garnered support from members of both parties.
California’s Proactive Stance on AI Safety
The technology used to create these deepfakes has become increasingly accessible, lowering previous barriers that limited such production to individuals with specialized technical expertise. California has become a significant hub for AI regulation, with state lawmakers passing multiple bills aimed at addressing AI safety concerns.
Attorney General Bonta has been particularly active regarding AI and its impact on children. In September, he met with OpenAI executives, alongside Delaware’s attorney general, to discuss concerns about how AI products interact with young people. This follows letters sent in August to twelve major AI companies after reports emerged of sexually inappropriate interactions between AI chatbots and children.
The California investigation occurs as the United Kingdom announced earlier this week that it is also conducting its own inquiry into the proliferation of deepfakes on the X platform.
The next expected step in California’s investigation is for xAI to respond to the inquiry. The timeline for this response and the specific legal avenues California authorities may pursue remain uncertain. Meanwhile, the outcome of the DEFIANCE Act in the House will be a key development to watch in the broader landscape of AI regulation.