Markets 
It is no longer safe to share images and videos of students online, experts have warned, as the eSafety regulator moves to combat a spike in AI-generated deepfakes targeting children and teachers.
Images taken from official school social media feeds are increasingly being scraped and run through generative AI to produce sexualised, mocking or damaging depictions of victims.
The escalating threat has prompted the eSafety commissioner to release a new advisory, urging schools to reconsider their posting habits and establish action plans to respond when images are weaponised.
Between January and March this year, the national eSafety regulator received more than 100 reports of anonymous accounts targeting schools and school staff, including AI-generated dance videos, face swaps, memes and fabricated stories about principals and teachers. Almost all involved imagery harvested from school social media accounts or websites.
eSafety Commissioner Julie Inman Grant said a significant number of cases involved AI-generated deepfake child sexual exploitation material originating from school settings.
“Behind each of these reports are real people – teachers, school staff and students – who are facing humiliation, reputation damage and distress,” said Inman Grant.
Other manipulations include sexualised AI imagery of teachers kissing and AI dance videos that alter the body image of a teacher.
While social media may be banned for Australian children under the age of 16, the Herald has observed a sharp increase in schools sharing images of their students and teachers to boost enrolments and build community.
One northern beaches public primary school parent, who wished to stay anonymous to protect the identity of her child, said the department’s consent process “doesn’t allow parents to make sensible distinctions”.
“At our school, if you want to refuse permission for your child’s image to appear on social media, you also have to refuse permission for them to appear in the school newsletter,” she said.
“I suspect many families have simply given consent because the alternative is having their children excluded or missing out on recognition, not because they’re genuinely comfortable with their children’s images being shared on social media.”
A spokesperson for the NSW Education Department said it took online safety and student wellbeing seriously and would carefully consider the eSafety commissioner’s advice. It said parents who consent to a school publishing images of their children could withdraw that consent at any time.
Yet internationally recognised child sexual exploitation experts warn that in the age of AI, there is simply no safe way to share images of children and staff.
“I can see the conundrum for schools but, generally, we are in a situation where there is no risk-free way to share images and videos of kids online, and particularly kids in school uniform,” said UNSW professor Michael Salter.
“Unfortunately, children in uniforms intersects with some of the worst online appetites.”
Curtin University professor of internet studies Tama Leaver said schools that use social media presence to build community find themselves in a tricky contradiction.
“We live in a country with the world’s first under‑16 social media ban,” Leaver said. “Having children on social media when they have no control or say in that is actually a mixed message at best.
“Either we’re in a risk-mitigation setting where we have to take this seriously, or we’re not. It would be safer to not have kids’ images online at all.”
Leaver and Salter said many parents were signing broad, once-a-year consent forms, leaving families blind to how their child’s image could be used.
“My honest advice is: unfortunately it’s a more complicated world now. We can’t tick a box once in February and hope that it’s all for the best for the rest of the year. We do need to have more conversations about where these images are going, especially if they are being made public,” Leaver said.
‘Unfortunately children in uniforms intersects with some of the worst online appetites.’
UNSW professor Michael Salter
In Australia, it is illegal to create and share non-consensual explicit deepfakes, punishable by up to seven years in prison. The eSafety world-leading industry codes and standards force AI-generated undressing apps out of the Australian market.
However, Inman Grant said not all AI material is covered by these laws.
“School communities should be aware that not all harmful material can be addressed under the Online Safety Act, especially material that targets adults. This is one of the reasons why it is so important to carefully assess the risks when posting imagery online – imagery that may later be digitally altered without consent.”
Before posting, she said schools should ensure there is clear consent, understand who can access the content they share, and regularly review privacy and security settings.
A spokesperson for Independent Schools NSW advised schools to prioritise less identifiable images of students, such as pictures taken from behind and at a distance, limiting close-up photos, and obtaining specific consents for images that will be publicly available and therefore at greater risk of misuse.
The NSW Independent Education Union secretary Carol Matthews said the union had observed an increasing number of deepfake matters in the last 18 months.
“Where staff have experienced AI-generated anonymous videos and images, it is extremely stressful and upsetting,” she said. “A number of staff within one school may be impacted at the same time. Everyone deserves a safe workplace.”
Start the day with a summary of the day’s most important and interesting stories, analysis and insights. Sign up for our Morning Edition newsletter.
Emily Kowal is a Walkley award-winning education reporter for The Sydney Morning Herald.Connect via email.From our partners
Read the full article here
