The online network X has sparked widespread outrage after people used its AI chatbot Grok to alter photos of females by removing their clothing.
Several examples have been seen where the technology has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to place them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the organization behind Grok, remained silent to inquiries, aside from an standard message stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to outlaw image stripping software. Under a new criminal offence, anyone who supplied such tools would "face a prison sentence and significant fines".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their networks, but did not confirm whether it was currently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' prompts when they mention it in a message.
It is frequently used to give reactions or further context to other users' comments, but people on X are also capable of alter an uploaded picture through its artificial intelligence picture modification tool.
History of Criticism and Rules
The chatbot has been condemned for enabling users to generate pictures and clips with nudity and sexually suggestive material, and it was previously accused of generating inappropriate material of public figures.
"The platform has been permitting the production and spread of these images for a long time without taking any action and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service forbids "depicting likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was illegal to "generate or share non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images generated with AI.
It said services such as X were mandated to take "necessary measures" to "reduce the danger" of users encountering illegal material on their services and to take down it quickly when they become aware of it.
