Elon Musk's online platform X has provoked major backlash after users employed its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Numerous instances have surfaced where the technology has been used to digitally undress women to make them appear in swimwear without permission, as well as to put them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the firm responsible for Grok, remained silent to inquiries, aside from an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to ban nudification tools. Under a upcoming law, anyone who distributed such tools would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these types of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said platforms must "evaluate the potential" of people accessing unlawful material on their networks, but did not specify whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some additional functions—which answers X users' requests when they mention it in a post.
It is often used to give commentary or additional background to other posters' remarks, but people on X are also can modify an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The tool has been condemned for allowing users to produce photos and footage with explicit content and sexualised content, and it was previously accused of generating sexually explicit content of celebrities.
"The platform has been permitting the generation and spread of these pictures for a long time without taking any measures and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting images of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or child sexual abuse material" and stated this included explicit AI images made with AI.
It said platforms such as X were mandated to take "appropriate actions" to "reduce the danger" of users viewing unlawful content on their services and to take down it swiftly when they learn of it.
