How AI-generated sexual images cause real harm, even though we know they are ‘fake’
Many women have experienced severe distress as Grok, the AI chatbot on social media site X, removed clothing from their images to show them in bikinis, in sexual positions or covered in blood and bruises. Grok, like other AI tools, has also reportedly been used to generate child sexual abuse material.
In response, the UK government has announced it will bring forward the implementation of a law, passed in June 2025, banning the creation of non-consensual AI-generated intimate images. Following bans in Malaysia and Indonesia, Grok has now been updated to no longer create sexualised images of real people in places where it is illegal, which will include the UK.
Read more: What can technology do to stop AI-generated sexualised images?
X’s owner, Elon Musk, has claimed the UK government wants “any excuse for censorship”. The media regulator, Ofcom, is also conducting an investigation into whether X’s activities broke UK law.
Some X users have minimised the harm these “undressed” and “nudified” images cause, describing them as “fake”, “fictional”, “very realistic art at most” and “no more real than a Tom & Jerry cartoon”.
You might think that AI-generated and edited images only cause harm through deception – fake images mislead us about real events. But how can images that everyone knows aren’t real cause harm?
The sexualised content of undressed images is not real, even if they are based on genuine photos. But these images are highly realistic. This, along with the misogyny motivating their creation, is sufficient to cause significant psychological distress to victims.
How ‘undressed’ images harm
MP Jess Asato and other victims report an uncanny feeling at seeing undressed images of themselves: “While of course I know it’s AI, viscerally inside it’s very, very realistic and so it’s really difficult to see pictures of me like that,” Asato told the BBC.
Research in philosophy and psychology can help explain this experience. Think about looking down from a tall building. You know you are completely safe, but might still feel terrified of the drop. Or you watch a horror film, then feel on edge all night. Here, your emotions are “recalcitrant”: you feel strong emotions that clash with what you believe to be true.
Seeing oneself digitally undressed generates powerful recalcitrant emotions. People strongly identify with their digital appearance. And a “nudifed” image really looks like the subject’s body, given it is based on a real picture of them.
So, while knowing these images are fake, their realism manipulates the victim’s emotions. They can feel alienated, dehumanised, humiliated and violated – as if they were real intimate images shared. This effect may worsen as AI-generated videos provide ever more realistic sexual content of users.
Research shows that the nonconsensual sharing of nude or sexual images is “associated with significant psychological consequences, often comparable to those experienced by victims of sexual violence”.
Besides the psychological impact of undressed images, users also feel horrified at the very real motivations behind them. Someone felt entitled to sexualise your photo, directing Grok to strip away clothing and reduce you to a body without consent. Publicly bombarding women with these images exerts control over how they present themselves online.
Sexual deepfake videos and undressed images – whether of celebrities, politicians or members of the public – target women for humiliation. The misogynistic mindset behind these images is real and familiar, even if their content is not.
Virtual harms
The distress caused by “undressed” images resembles another prevalent form of digital misogyny: the assault and harassment of women in virtual worlds. Many women in online virtual reality environments report their avatars being assaulted by other users – a common issue in video games that is worsened as virtual reality (VR) headsets present an immersive experience of being assaulted.
Whistleblowers have claimed Meta has suppressed the lack of child safety on its VR platform, with girls as young as nine frequently harassed and propositioned by adult men. Meta denies these allegations. A company spokesperson told the Washington Post that Meta’s VR platform has safety features to protect young people, including default settings that allow teen users to communicate only with people they know.
Virtual assault is also often dismissed as “not real”, even though it can cause similar trauma to physical assault. The realistic appearance of virtual reality, strong identification with one’s avatar and the misogynistic motivations behind virtual assault enable it to cause serious psychological harm, even though there is no physical contact.
These cases show how misogyny has evolved with technology. Users can now create and participate in realistic representations of harm: undressed images, deepfake videos, virtual assault and the abuse of chatbots and sex dolls based on real people.
These forms of media cause significant distress, but are slow to be regulated as they don’t physically harm victims. Banning social media like X isn’t the solution. We need proactive regulation that anticipates and prohibits these digital harms, rather than enacting laws only once the damage is done.
Victims undressed by Grok or assaulted in virtual worlds are not being “too sensitive”. It is a mistake to dismiss the real psychological impact of this media just because the images themselves are fake.
Alex Fisher has received research funding from The Society for Applied Philosophy, the Leverhulme Trust, the Aristotelian Society, and the Royal Institute of Philosophy.