The social media platform X has provoked widespread backlash after users used its AI chatbot Grok to manipulate pictures of women by taking off their attire.
Several instances have surfaced where the technology has been employed to digitally undress women to show them in bikinis without consent, as well as to insert them in inappropriate situations.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to questions, aside from an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to ban image stripping software. Under a proposed law, anyone who provided such tools would "be subject to a prison sentence and significant penalties".
"The platform could prevent these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people accessing illegal content on their platforms, but did not confirm whether it was at this time investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional features—which responds to X users' prompts when they tag it in a message.
It is frequently used to provide commentary or more background to other users' comments, but people on X are also able to alter an uploaded picture through its AI image editing feature.
History of Criticism and Rules
The system has been condemned for permitting users to generate images and videos with nudity and sexually suggestive content, and it was earlier alleged to have been involved in creating sexually explicit material of celebrities.
"X has been allowing the generation and sharing of these images for a long time without taking any measures and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing images of individuals in a pornographic manner".
In a comment, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or illegal content" and stated this included sexual deepfakes created with AI.
It said services such as X were required to take "necessary measures" to "reduce the risk" of users encountering illegal material on their platforms and to delete it quickly when they are notified of it.
