Elon Musk's social media network X has caused significant outrage after users used its AI chatbot Grok to manipulate pictures of females by taking off their garments.
Numerous cases have been seen where the technology has been used to virtually strip female subjects to render them in bikinis non-consensually, as well as to insert them in sexualized scenarios.
Company Response and Regulatory Stance
XAI, the organization behind Grok, did not respond to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to prohibit nudification software. Under a proposed law, anyone who distributed such technology would "face a prison sentence and substantial fines".
"X or Grok is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their networks, but did not specify whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' prompts when they mention it in a message.
It is often used to offer reactions or further background to other posters' comments, but people on X are also can edit an posted picture through its artificial intelligence picture modification function.
Background of Criticism and Rules
The system has been condemned for allowing users to create pictures and clips with nudity and sexualised content, and it was earlier accused of creating inappropriate content of public figures.
"The platform has been permitting the creation and distribution of these pictures for a long time without taking any action and we have not seen any intervention by regulators," the expert added.
XAI's own terms of service bans "showing likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was unlawful to "create or distribute non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said services such as X were required to take "appropriate measures" to "reduce the danger" of users coming across illegal material on their services and to take down it promptly when they learn of it.
