The social media network X has sparked widespread outrage after users utilized its artificial intelligence assistant Grok to change pictures of females by stripping their clothing.
Numerous cases have surfaced where the tool has been used to remove clothing from women to show them in bikinis non-consensually, as well as to put them in inappropriate situations.
Developer Reaction and Regulatory Position
XAI, the firm responsible for Grok, remained silent to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to ban nudification software. Under a upcoming criminal offence, anyone who provided such technology would "be subject to a jail term and substantial fines".
"X or Grok could prevent these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was currently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some additional functions—which responds to X users' requests when they tag it in a tweet.
It is commonly used to provide commentary or more background to other users' comments, but people on X are also capable of modify an posted photo through its AI image editing feature.
Background of Criticism and Rules
The chatbot has been condemned for enabling users to create photos and clips with explicit content and sexually suggestive content, and it was previously alleged to have been involved in generating sexually explicit material of celebrities.
"X has been enabling the generation and spread of these images for a long time without taking any steps and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "depicting images of persons in a sexually explicit way".
In a comment, Ofcom said it was against the law to "generate or circulate deepfakes or child sexual abuse material" and clarified this included sexual deepfakes created with AI.
It said services such as X were mandated to take "appropriate actions" to "reduce the danger" of users encountering illegal content on their platforms and to delete it quickly when they become aware of it.
