'Politicians Must Try Harder To Stamp Out Abhorrent Deepfake Trend – Before It Gets Too Big'

The start of 2026 has brought with it a grim reminder of ever-evolving online threats. Recent weeks have seen the rapid growth of a vile new practice – the generation of non-consensual, sexualised images of women and girls by Elon Musk’s AI tool, Grok.

This technology – and the people behind it – have enabled the proliferation of degrading images of real women, whose photos have been undressed and distorted via user queries to an AI chatbot. Individuals have been empowered by this technology to humiliate, shame and intimidate women. More disturbing yet, children have been sexualised by the bot, as Grok has altered their pose and clothing to create thousands of explicit images, many of which meet a legal threshold for child sexual abuse material (CSAM).

This is far from an isolated trend. The harrowing reality is that CSAM online is a growing problem. NSPCC data shows a 93% increase in cases in the last five years, while the Internet Watch Foundation found a record high level of CSAM online last year. As AI tools become ever smarter, we can only expect the problem of explicit deepfake images of real children to grow.

The consequences of this aren’t limited to the online world. CSAM online increases demand for its creation, putting children in danger. And seeing this kind of content become more normalised on a mainstream platform sends a terrible message to boys and young men about healthy relationship behaviour in the real world too. Apps that facilitate the “nudification” of real women and girls set an awful precedent for attitudes on everything from objectification to consent.

The moral imperative of tackling the abhorrent trend of non-consensual image generation presents two duties to those of us who seek to keep women and children safe online.

The first is to the regulator, Ofcom, to meet the urgency and severity of the moment. For all its limitations, the Online Safety Act (OSA) is clear that sites must take down sexual images of children, with Ofcom empowered to issue penalties for platforms like X that are clearly failing their moderation duties. Probes into X announced this week by Ofcom are to be welcomed and will be an important test of Ofcom’s ability to effectively implement the OSA in a way that prioritises women and children’s safety, over managing relations with regulated platforms.

The second challenge is for politicians to meet their moral responsibility to curb dangerous new technology. Developments on X over recent weeks have shown the pace at which online risks are evolving. The OSA laid down important requirements on platforms when it came to moderation, but it is already frustratingly out of date. With generative AI some way off at the time of its conception, it failed to fully address the possibility of platforms actively creating such harmful material themselves. With 13% of the public regularly using generative AI tools, we can only expect risks related to emerging technologies to increase.

We must confront the uncomfortable truth that, so far, politicians have failed to design and implement protections that can fully keep up with technology. The OSA was a landmark step in government recognising its responsibility to keep children safe online, as in any other sphere. But we cannot meet today’s risks replicating a legislative process that began in 2017, led to legislation in 2023, and largely took until 2025 to come into force.

That means finding legislative approaches whose speed and consequences are consistent with the pace and grim severity of the threats at hand. We must grapple with the way we consider harm and accountability online.

Grok does not create CSAM with malicious intent – as a large language model it cannot possess intent. But the harm it causes is the same, and we must consider how we hold the human beings behind it responsible for their failure to ensure its safety, as we would with those who make any dangerous product.

And we must learn from regulatory approaches in other areas where risks are fast moving and informational asymmetries high, including considering whether greater personal legal responsibility should be held by executives of corporations that are found to have exposed our children to online threats or, as in the case of Grok, created those threats.

As part of our strategy to tackle violence against women and girls, the government rightly announced the criminalisation of apps that allow users to produce non-consensual nude images. We cannot have two-tier justice when it comes to the sexual abuse images. Whether facilitated by Big Tech or produced by a lone wolf – those responsible for these vile images should face the full force of the law.

We mustn’t pretend that keeping up with silicone valley will be easy. But on an issue as fundamental as keeping women and children safe from abuse, we have a moral obligation to meet this challenge with the urgency it deserves.