Elon Musk's online network X has sparked major backlash after people utilized its AI chatbot Grok to manipulate pictures of women by removing their clothing.
Several examples have surfaced where the tool has been employed to virtually strip women to make them appear in swimwear without permission, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, did not respond to inquiries, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to outlaw nudification tools. Under a proposed criminal offence, anyone who supplied such tools would "face a jail term and substantial penalties".
"The platform has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the risk" of people accessing unlawful material on their platforms, but did not specify whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some paid-for features—which responds to X users' requests when they reference it in a post.
It is commonly used to offer reactions or further background to other users' comments, but people on X are also able to alter an uploaded picture through its artificial intelligence picture modification function.
History of Controversy and Policy
The tool has been criticised for enabling users to produce images and clips with nudity and sexualised content, and it was earlier alleged to have been involved in making inappropriate material of celebrities.
"The platform has been permitting the creation and sharing of these images for a long time without taking any action and we have not seen any challenge by regulators," the expert added.
XAI's own terms of service prohibits "showing images of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or illegal content" and confirmed this included explicit AI images created with AI.
It said platforms such as X were mandated to take "suitable steps" to "reduce the risk" of users encountering illegal material on their services and to remove it quickly when they are notified of it.
