Elon Musk's online network X has provoked significant backlash after users employed its AI chatbot Grok to manipulate images of females by removing their garments.
Several instances have been seen where the technology has been employed to virtually strip female subjects to make them appear in bikinis without consent, as well as to insert them in sexualized contexts.
Company Reaction and Regulatory Position
XAI, the firm responsible for Grok, remained silent to questions, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to outlaw nudification software. Under a proposed law, anyone who supplied such tools would "face a prison sentence and heavy fines".
"The platform is capable of halting these types of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the danger" of people accessing illegal content on their services, but did not confirm whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some additional features—which responds to X users' requests when they reference it in a message.
It is often used to give reactions or additional context to other posters' comments, but people on X are also able to edit an posted image through its AI image editing function.
History of Controversy and Policy
The system has been faulted for permitting users to generate images and footage with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in making inappropriate content of celebrities.
"The platform has been allowing the generation and sharing of these pictures for months without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of people in a graphic way".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and confirmed this covered sexual deepfakes made with AI.
It said platforms such as X were obligated to take "necessary steps" to "minimize the danger" of users encountering illegal material on their platforms and to take down it quickly when they learn of it.
