California sends xAI cease-and-desist letter, demanding it prevent sexualized deepfake images of minors

• California sent xAI a cease-and-desist letter, demanding that it prevent the allowance of deepfake images of minors.
• Elon Musk's xAI faces sustained criticism over allegations that Grok is creating nonconsensual sexualized images.
• The letter, sent by AG Rob Bonta, threatens legal action if the deepfakes are not stopped.

California Attorney General Rob Bonta has demanded that xAI prevent its chatbot, Grok, from creating sexualized deepfake images of children.

Bonta's office sent a cease-and-desist letter to Elon Musk's AI startup on Friday after sustained criticism over the bot's ability to create nonconsensual sexualized images, including those of minors.

The letter demanded that xAI prevent Grok from creating sexualized images of people who did not request the image be made, or were minors when the image was created. Failure to stop such images from being created or distributed would be considered a violation of the state's deepfake porn, child-sexual-abuse images, unlawful recording, and unfair business practice statutes, the letter states.

xAI was given until January 20 at 5 p.m. to comply with the order.

Earlier this week, social media platform X, which is owned by xAI, said that it implemented restrictions on Grok.

"We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," X's safety account said in a blog post on the platform on Wednesday. "This restriction applies to all users, including paid subscribers."

However, that didn't stop the X or Grok app from creating sexualized images, Business Insider's Henry Chandonnet found on Thursday.

Grok has also been facing international backlash and legal actions for its ability to undress images of real people and create revealing deepfakes of them without their consent at users' request.

Ahead of the cease-and-desist letter, Bonta announced on Wednesday that his office is investigating the "non-consensual, sexually explicit material that xAI has produced and posted online."

Separately, Ashley St. Clair, an influencer, political strategist, and the mother of one of Musk's children, sued xAI on Thursday, alleging in a complaint that Grok generated sexually explicit deepfakes of her without consent using photos from when she was a child.

UK Prime Minister Keir Starmer called graphic images generated by Grok "disgusting" and "shameful" in a meeting with the House of Commons, while Indonesia, Malaysia, and the Philippines resorted to blocking access to Grok. There is no indication that the ban has been lifted since X's blog post on Wednesday.

Representatives for the CA Attorney General's office did not immediately respond to requests for comment from Business Insider.

In an automated response to Business Insider, xAI said, "Legacy Media Lies."