Elon Musk's online platform X has caused major backlash after individuals employed its AI chatbot Grok to manipulate photos of women by removing their garments.
Multiple examples have been seen where the technology has been employed to remove clothing from female subjects to render them in swimwear without permission, as well as to place them in inappropriate contexts.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, remained silent to questions, other than an automatically-generated message stating "legacy media lies".
A government representative stated that new laws is being brought forward to prohibit nudification tools. Under a new law, anyone who distributed such tools would "be subject to a prison sentence and significant penalties".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "consider the risk" of people viewing illegal content on their platforms, but did not specify whether it was presently examining X or Grok in relation to AI-generated pictures.
Grok is a free chatbot—with some paid-for functions—which responds to X users' queries when they reference it in a message.
It is frequently used to provide reactions or additional context to other users' remarks, but people on X are also can alter an uploaded photo through its AI image editing tool.
Background of Criticism and Policy
The system has been criticised for enabling users to create photos and clips with explicit content and sexually suggestive content, and it was previously accused of generating inappropriate material of public figures.
"X has been enabling the creation and spread of these pictures for months without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own acceptable use policy forbids "portraying images of persons in a graphic manner".
In a comment, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or child sexual abuse material" and confirmed this included explicit AI images generated with AI.
It said services such as X were required to take "necessary measures" to "minimize the risk" of users viewing unlawful material on their services and to delete it promptly when they are notified of it.
