The online network X has sparked significant backlash after people employed its artificial intelligence assistant Grok to alter images of females by stripping their attire.
Multiple examples have emerged where the tool has been used to digitally undress women to make them appear in swimwear non-consensually, as well as to place them in sexualized situations.
Company Reaction and Legal Position
XAI, the organization responsible for Grok, remained silent to inquiries, aside from an auto-reply message stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to ban image stripping tools. Under a proposed criminal offence, anyone who supplied such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the danger" of people seeing illegal content on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some paid-for features—which responds to X users' queries when they mention it in a message.
It is commonly used to offer reactions or more context to other users' comments, but people on X are also able to alter an uploaded picture through its AI image editing tool.
History of Criticism and Policy
The chatbot has been condemned for allowing users to generate images and clips with explicit content and sexualised content, and it was in the past accused of generating graphic material of celebrities.
"The platform has been permitting the creation and spread of these pictures for months without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own usage rules bans "showing likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was illegal to "create or distribute deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes created with AI.
It said services such as X were required to take "necessary actions" to "minimize the risk" of users encountering illegal content on their services and to take down it promptly when they learn of it.
