Elon Musk's online network X has provoked major anger after people utilized its AI chatbot Grok to alter pictures of women by removing their garments.
Several cases have been seen where the system has been employed to virtually strip female subjects to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Developer Response and Regulatory Position
XAI, the firm responsible for Grok, did not respond to inquiries, aside from an standard message stating "legacy media lies".
A government representative stated that legislation is being proposed to prohibit image stripping software. Under a proposed criminal offence, anyone who provided such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "consider the danger" of people viewing unlawful material on their services, but did not confirm whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for functions—which answers X users' prompts when they mention it in a tweet.
It is frequently used to provide reactions or more context to other users' remarks, but people on X are also capable of modify an uploaded picture through its artificial intelligence picture modification feature.
History of Criticism and Policy
The system has been condemned for allowing users to create images and clips with nudity and sexually suggestive content, and it was earlier accused of creating sexually explicit material of celebrities.
"The platform has been enabling the production and spread of these images for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules forbids "showing images of people in a graphic fashion".
In a statement, Ofcom said it was illegal to "generate or share private pictures or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the risk" of users encountering illegal material on their services and to remove it quickly when they learn of it.
