The online platform X has caused widespread backlash after people used its AI chatbot Grok to manipulate images of females by taking off their attire.
Numerous examples have emerged where the technology has been used to digitally undress women to render them in swimwear without consent, as well as to place them in sexually explicit contexts.
Company Reaction and Regulatory Position
XAI, the firm behind Grok, did not respond to inquiries, except for an automatically-generated response stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to ban nudification software. Under a upcoming criminal offence, anyone who provided such technology would "receive a prison sentence and significant fines".
"X or Grok has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people seeing unlawful material on their services, but did not confirm whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' prompts when they tag it in a message.
It is frequently used to provide reactions or additional context to other users' comments, but people on X are also can edit an posted photo through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The system has been condemned for enabling users to create photos and footage with explicit content and sexually suggestive material, and it was previously accused of making graphic content of public figures.
"X has been permitting the creation and sharing of these images for a long time without taking any measures and we have yet to see any intervention by regulators," the expert added.
XAI's own terms of service forbids "depicting images of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "generate or distribute deepfakes or illegal content" and stated this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary actions" to "lessen the risk" of users encountering unlawful material on their platforms and to take down it quickly when they become aware of it.
