Elon Musk's online network X has sparked widespread backlash after users utilized its artificial intelligence assistant Grok to manipulate photos of females by taking off their garments.
Numerous examples have been seen where the system has been used to virtually strip women to show them in bikinis non-consensually, as well as to put them in inappropriate scenarios.
Company Response and Legal Position
XAI, the organization responsible for Grok, remained silent to requests for comment, except for an standard message stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to outlaw image stripping software. Under a new law, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the risk" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some premium features—which responds to X users' requests when they tag it in a tweet.
It is frequently used to offer commentary or further background to other posters' comments, but people on X are also capable of modify an uploaded image through its AI image editing feature.
History of Controversy and Rules
The system has been condemned for permitting users to create images and videos with nudity and sexualised material, and it was previously accused of creating sexually explicit content of celebrities.
"X has been allowing the generation and spread of these pictures for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing images of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said platforms such as X were required to take "appropriate actions" to "lessen the danger" of users coming across unlawful content on their services and to take down it swiftly when they become aware of it.
