The online network X has provoked significant anger after individuals utilized its AI chatbot Grok to alter pictures of women by taking off their garments.
Multiple cases have surfaced where the technology has been employed to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to insert them in sexualized contexts.
Developer Reaction and Legal Position
XAI, the company that created Grok, remained silent to questions, other than an automatically-generated response stating "traditional news outlets lies".
A government representative stated that legislation is being brought forward to outlaw nudification software. Under a upcoming law, anyone who provided such technology would "receive a prison sentence and substantial penalties".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their networks, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some additional features—which responds to X users' requests when they tag it in a message.
It is commonly used to give commentary or more background to other posters' remarks, but people on X are also capable of alter an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The system has been criticised for enabling users to create images and footage with nudity and sexually suggestive content, and it was previously alleged to have been involved in creating graphic material of public figures.
"X has been permitting the generation and spread of these images for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of people in a pornographic way".
In a statement, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images generated with AI.
It said services such as X were required to take "suitable steps" to "lessen the risk" of users viewing illegal material on their platforms and to take down it swiftly when they are notified of it.
