Elon Musk's social media network X has provoked major anger after people used its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Multiple instances have emerged where the technology has been used to remove clothing from women to render them in swimwear without consent, as well as to put them in sexualized contexts.
Company Reaction and Regulatory Position
XAI, the firm responsible for Grok, did not respond to questions, other than an auto-reply message stating "legacy media lies".
A government representative stated that legislation is being proposed to outlaw nudification technology. Under a new law, anyone who provided such tools would "be subject to a jail term and heavy penalties".
"X or Grok is capable of halting these types of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people seeing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some paid-for functions—which answers X users' queries when they tag it in a post.
It is often used to offer commentary or further background to other users' comments, but people on X are also capable of edit an posted photo through its AI image editing feature.
Background of Criticism and Policy
The system has been condemned for enabling users to produce pictures and clips with explicit content and sexualised content, and it was previously accused of making inappropriate content of celebrities.
"The platform has been enabling the creation and sharing of these images for months without taking any action and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "create or share deepfakes or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said networks such as X were required to take "suitable actions" to "lessen the risk" of users viewing illegal material on their services and to remove it promptly when they become aware of it.
