Charity Reports Children Creating AI-Generated Child Abuse Images

Children are using AI to create inappropriate images of their peers, warns a UK charity, the UK Safer Internet Centre (UKSIC). While reports from schools are currently minimal, the center stresses the urgent need for preventative measures.

The UKSIC emphasizes the importance of educating children about the gravity of their actions, as any creation, possession, or distribution of such material is illegal, and could lead to unintended widespread sharing or even blackmail.

A study by RM Technology involving 1,000 pupils found that nearly a third use AI for unsuitable online content, revealing a knowledge gap between students and teachers and complicating online safety efforts.

There’s a debate among teachers regarding who should educate children about online harms, but UKSIC advocates for a joint effort from schools, parents, and governments.

UKSIC director David Wright and Marie Collins Foundation CEO, Victoria Green, express concern over the potential long-term harm and criminal abuse, urging immediate action to address the misuse of new technologies like AI.

An app in Spain has raised alarms by enabling the creation of fake nude images of minors, which spread online without the victims' consent.

These "declothing" apps, often part of social media platforms, are becoming increasingly sophisticated, posing a growing threat according to cyber expert Javaad Malik. Despite their alarming use in malicious activities like revenge porn, the makers have yet to face legal consequences.