AI in formal education is one dimension; however, parental guidance on AI use at home is equally critical and arguably less structured. I was struck when my 8-year-old son turned to an AI system to answer a very basic question—how to care for a dog. He later mentioned that AI tools have already been introduced at school. I would not advocate restricting generative AI in learning environments; rather, this experience reinforced the need to actively develop AI literacy and critical thinking. The priority, in my view, is to cultivate reflective use rather than passive reliance.
I am also not surprised that adolescents, as they begin to develop sexual curiosity, explore digital platforms—including AI systems—to address it. This is not a new phenomenon; it reflects longstanding sociocultural and biological drivers. However, generative AI appears to be accelerating both access and exposure, in ways that outpace the ability of parents, educators, and policymakers to provide adequate guidance or safeguards. In that sense, the technology is not creating the underlying issue but is amplifying it at a rate that challenges existing oversight structures.
For this reason, your framing of “Slow AI” resonates strongly. It captures the need for a more deliberate, developmentally informed approach to integrating these tools—particularly in educational and domestic contexts. This remains a persistently complex problem, with no straightforward solution.
Thanks so much Vic. I recognise a lot of what you saying my work and my home. And as you say the most important thing is that we raise awareness and engage in dialogue. I do think BigTech has a large responsibility and role to play though.
Sam, thank you for writing what no one should have to write. You have done the forensic work that the regulators are not doing. The numbers are stark and unarguable: 3,443 videos, up from 13. 97% girls. 483 million downloads. $122 million in revenue. €1,200 in sanctions.
You have also named the structural pattern that makes this possible: platform impunity. Motherless.com operated for 18 years before partial action. The nudifier ecosystem is on the same trajectory, but faster, because the friction is now zero and the perpetrators now include children.
The conversation defaults to “teach boys consent”. That is necessary and it is also a category error. Children cannot be the principal regulatory mechanism for an industry whose business model is platform impunity.
The change that would matter is structural:
· Platform liability for the apps that companies distribute, with real penalties.
· Enforced app‑store removal with criminal accountability for repeat hosting.
· Age verification at the model level, not just the app level.
· Mandatory disclosure of training data for any model that can be used to generate images of people.
· Treatment of model developers and platform operators as primary actors, not neutral conduits.
The AI Commons has no special power over Apple or Google. But we can refuse the frame that this is inevitable. We can name the architecture that enables it. And we can build alternatives that do not rely on the same extractive, unaccountable platforms.
Thank you for doing the work that the platforms will not do.
✊❤️🌎
From the AI Commons – no paywall, no surveillance, no enclosure
It is indeed a matter of culture. What we allow and even promote in society matters. Character matters. Modeling behavior matters. Parents, teachers neighbors, elected officials, corporations- their actions and words all matter. It takes a village.
Children enact the problem but are not the source of it.
I am thinking of the massive sweets and snacks industries that bring harm to children.
The fast food industry. Almost all children love McDonalds. It comes with toys that capture them young.
Acquisitiveness entrained by toy manufacturers.
Online games that employ paid tokens for progression.
I am thinking of drug dealers who use children.
Across the world the tobacco industry still aims to catch people young. Or in the wealthy west to do the same with bubblegum flavour vapes.
When I was a child cigarettes were sold from machines on the street. Given away as prizes in village fetes. My first pack I won at hoopla aged ten.
Regulation of tobacco products in the UK has made a massive difference. However most children who start smoking young just get their supplies from their parents. And perhaps their inspiration.
Yes, children do need to understand what consent means and they need to have it modelled for them continually.
But they're being used here in a shocking way and that's the huge tragedy.
And money making industries do not ask anyone's consent but sidestep their responsibilities with their eyes firmly fixed on the money prize.
The scary thing, of course, is the huge power of today's technology to do harm.
In superhero comics when I was a child I was fascinated by the back page adverts. These included X-ray specs that were depicted as seeing through girls' clothes. The technology was rubbish but the idea was spookily the same.
Thank you Sam for writing on this extremely unpalatable situation.
Thank you Davina. And wow. That is genuinely insane regarding the village fete but I do remember getting cigarettes for friend's parents from vending machines. 😢
Thank you Anne. I think the first step is in raising awareness and then making every single CEO individually accountable. The speed with which the ‘fixed’ will be implemented once they can be named and threatened with real jail time will be faster than the speed of light.
Thank you for writing about this, it makes my stomach turn to read AND it's so important that we are talking about this. It breaks my heart for all of the children involved. What happened is terrible, and I can't imagine how scared and hurt those girls were. Yet, those boys are children, too. Yes, they "should" know better. Of course. AND, they should never ever ever have access to that as easily as I just (unfortunately) was able to see what that site is like. The impulse control of a teen boy is practically nothing, and they don't think about what they are doing as creating child sex abuse material.. because they are fucking children. Yet, we hold the kids accountable not the app developers or the people in power who create a system that incentivizes the victimization of children. And grown men, who are much better at hiding the horrible things that they do, will make pictures of children this way for years to come, without much accountability at all. Because if this app, which I can't believe still exists after this incident, gets taken down then another will pop up and then another and then another. This is the world that women like me were raised to be aware of, on steroids. Awful.
Thank you Courtney and oh my goodness how right you are. I just do not understand how this app is ever allowed to exist. I (vaguely) remember being a teenage boy and this point about impulse control is spot on. 😢
I’m a fan of serious fines to the app stores for even having it available. More accountability across the board at that level 💯 what does it take? More conversation about this? Thank you for writing about this, Sam.
Absolutely Tracy. Like seriously why not fine them 20% of profits and also hold individuals accountable. Will be interesting to see how the State of Florida vs Sam Altman plays out in this regard.
Ooh I didn’t even know about that case! I completely agree. App store, google store all of these that host the sales of these things need to be held accountable too.
Honestly. As soon as CEOs can be named and taken to court we will be amazed at the speed in which the technology can suddenly do what we have been asking for…
This is SO SAD - and also DREADFUL - that such harm to young people isn’t policed properly & professionally?? 😫
I knew that the law had NOT kept up with technology - but this makes it even worse… 🥲
Thanks so much for posting this - though I understand why some people might not want to read it?
What I don’t understand - given the power of social media & all media (online & offline) to change things - WHY isn’t this being “called out” and publicised so widely & publicly that it isn’t stamped out or at least reduced??
All these government bodies & groups - making NO difference - who funds them because it’s a waste of money??
You & I don’t have the power - who does?
And how can social media be used to challenge these people to act?
Thanks again for sharing this - and I hope there is NO negative impact on you!?
Thank you Gail. I sadly think that the main reason why these issues have not been addressed is because BigTech has worked out that to do so would likely cost their shareholders money. 😢
I don’t think MONEY has or ever will be a good reason to NOT protect children from this type of bullying & abuse?
And I know it’s NOT your fault or mine = it’s a sad comment on today’s world where money seems to be more important that other things that I believe are important = children! Thanks again Sam! 👍
I will say tho that the main other thing I was thinking throughout the article is how do we change the apparently default position of society's framing of women...and then I came to your phrase at the end 'female victim population', and hated that - I feel like even that perfectly valid use of words to say what you mean still inadvertently reinforces the idea of females as victims - those that have been affected and manipulated by others who wish to affect and manipulate them (us, me, women). I think it was PWC that only stopped forcing females to wear high heels about 5 years ago....in appearance we are still told that 'making an effort' is a thing that focuses on making ourselves attractive within an applied societal stereotype...
This is an excellent point Alice. And you are right I could perhaps have used a better phrase to explain this rather than be led towards a position that I agree we need to actively work to change.
I know exactly what you mean Karen. But I still think AI has the potential to do great good, we just need to make sure that the people who develop these tools and models take the appropriate actions to protect those amongst us who are most vulnerable.
With automated moderation tools it should be possible for these apps to “card” any nude images. ID plus facial recognition from several angles. Adults who want such images of themselves presumably won’t be afraid to identify themselves, and it would screen out children and revenge “porn”. Or just outlaw AI porn of real people altogether, because no law is going to prevent coercion or force if an abuser is physically present.
As for school children making images of classmates, they absolutely do deserve the legal consequences of their actions. The life they are damaging is as young and valuable as their own (and a whole lot more innocent.)
Lucy, I absolutely agree the tech is there to do this, and we need to ask ourselves seriously what is the reason for this not being done, other than the potential impact it might make on profit for shareholders.
Thank you for bringing this to wider attention. My day job is a boring desk job in the adult industry, and we've been fighting Motherless for years. They've promised "stricter content moderation", but the precedent is still there.
Anyway, I think it's telling that the first action is the take down the photographs and the second is to punish the individuals responsible, instead of actually targeting the source of the problem and taking down the apps.
Thank you for highlighting this Sam. I cannot think of a single reason why an app of this kind is needed. Therefore, it seems that it can only be about profit and exploitation. It should be illegal to make the apps.
arggggg - I write this instead of filthy words.
I have something to say and will wait until later to say it.
Thanks, Sam, for raising this.
AI in formal education is one dimension; however, parental guidance on AI use at home is equally critical and arguably less structured. I was struck when my 8-year-old son turned to an AI system to answer a very basic question—how to care for a dog. He later mentioned that AI tools have already been introduced at school. I would not advocate restricting generative AI in learning environments; rather, this experience reinforced the need to actively develop AI literacy and critical thinking. The priority, in my view, is to cultivate reflective use rather than passive reliance.
I am also not surprised that adolescents, as they begin to develop sexual curiosity, explore digital platforms—including AI systems—to address it. This is not a new phenomenon; it reflects longstanding sociocultural and biological drivers. However, generative AI appears to be accelerating both access and exposure, in ways that outpace the ability of parents, educators, and policymakers to provide adequate guidance or safeguards. In that sense, the technology is not creating the underlying issue but is amplifying it at a rate that challenges existing oversight structures.
For this reason, your framing of “Slow AI” resonates strongly. It captures the need for a more deliberate, developmentally informed approach to integrating these tools—particularly in educational and domestic contexts. This remains a persistently complex problem, with no straightforward solution.
Thanks so much Vic. I recognise a lot of what you saying my work and my home. And as you say the most important thing is that we raise awareness and engage in dialogue. I do think BigTech has a large responsibility and role to play though.
Sam, thank you for writing what no one should have to write. You have done the forensic work that the regulators are not doing. The numbers are stark and unarguable: 3,443 videos, up from 13. 97% girls. 483 million downloads. $122 million in revenue. €1,200 in sanctions.
You have also named the structural pattern that makes this possible: platform impunity. Motherless.com operated for 18 years before partial action. The nudifier ecosystem is on the same trajectory, but faster, because the friction is now zero and the perpetrators now include children.
The conversation defaults to “teach boys consent”. That is necessary and it is also a category error. Children cannot be the principal regulatory mechanism for an industry whose business model is platform impunity.
The change that would matter is structural:
· Platform liability for the apps that companies distribute, with real penalties.
· Enforced app‑store removal with criminal accountability for repeat hosting.
· Age verification at the model level, not just the app level.
· Mandatory disclosure of training data for any model that can be used to generate images of people.
· Treatment of model developers and platform operators as primary actors, not neutral conduits.
The AI Commons has no special power over Apple or Google. But we can refuse the frame that this is inevitable. We can name the architecture that enables it. And we can build alternatives that do not rely on the same extractive, unaccountable platforms.
Thank you for doing the work that the platforms will not do.
✊❤️🌎
From the AI Commons – no paywall, no surveillance, no enclosure
Thank you team! That collective refusal is the first step. We are stronger and better together. 💪
It is indeed a matter of culture. What we allow and even promote in society matters. Character matters. Modeling behavior matters. Parents, teachers neighbors, elected officials, corporations- their actions and words all matter. It takes a village.
Exactly this Linda. And thankfully we have Substack and this community as a place to start. 🙏
This is staggering in the worst way possible. Thank you for reporting on the hard things. It’s the only way we can hope they get rectified.
You are very welcome JHong. And thank you for reading. I remain hopeful that structural change is possible. But it will need us all to rise up. ✊✊
Children enact the problem but are not the source of it.
I am thinking of the massive sweets and snacks industries that bring harm to children.
The fast food industry. Almost all children love McDonalds. It comes with toys that capture them young.
Acquisitiveness entrained by toy manufacturers.
Online games that employ paid tokens for progression.
I am thinking of drug dealers who use children.
Across the world the tobacco industry still aims to catch people young. Or in the wealthy west to do the same with bubblegum flavour vapes.
When I was a child cigarettes were sold from machines on the street. Given away as prizes in village fetes. My first pack I won at hoopla aged ten.
Regulation of tobacco products in the UK has made a massive difference. However most children who start smoking young just get their supplies from their parents. And perhaps their inspiration.
Yes, children do need to understand what consent means and they need to have it modelled for them continually.
But they're being used here in a shocking way and that's the huge tragedy.
And money making industries do not ask anyone's consent but sidestep their responsibilities with their eyes firmly fixed on the money prize.
The scary thing, of course, is the huge power of today's technology to do harm.
In superhero comics when I was a child I was fascinated by the back page adverts. These included X-ray specs that were depicted as seeing through girls' clothes. The technology was rubbish but the idea was spookily the same.
Thank you Sam for writing on this extremely unpalatable situation.
Thank you Davina. And wow. That is genuinely insane regarding the village fete but I do remember getting cigarettes for friend's parents from vending machines. 😢
Whenever or whatever I can help with this. I’m on board.
Thank you Anne. I think the first step is in raising awareness and then making every single CEO individually accountable. The speed with which the ‘fixed’ will be implemented once they can be named and threatened with real jail time will be faster than the speed of light.
Thank you for writing about this, it makes my stomach turn to read AND it's so important that we are talking about this. It breaks my heart for all of the children involved. What happened is terrible, and I can't imagine how scared and hurt those girls were. Yet, those boys are children, too. Yes, they "should" know better. Of course. AND, they should never ever ever have access to that as easily as I just (unfortunately) was able to see what that site is like. The impulse control of a teen boy is practically nothing, and they don't think about what they are doing as creating child sex abuse material.. because they are fucking children. Yet, we hold the kids accountable not the app developers or the people in power who create a system that incentivizes the victimization of children. And grown men, who are much better at hiding the horrible things that they do, will make pictures of children this way for years to come, without much accountability at all. Because if this app, which I can't believe still exists after this incident, gets taken down then another will pop up and then another and then another. This is the world that women like me were raised to be aware of, on steroids. Awful.
Thank you Courtney and oh my goodness how right you are. I just do not understand how this app is ever allowed to exist. I (vaguely) remember being a teenage boy and this point about impulse control is spot on. 😢
I’m a fan of serious fines to the app stores for even having it available. More accountability across the board at that level 💯 what does it take? More conversation about this? Thank you for writing about this, Sam.
Absolutely Tracy. Like seriously why not fine them 20% of profits and also hold individuals accountable. Will be interesting to see how the State of Florida vs Sam Altman plays out in this regard.
Ooh I didn’t even know about that case! I completely agree. App store, google store all of these that host the sales of these things need to be held accountable too.
Honestly. As soon as CEOs can be named and taken to court we will be amazed at the speed in which the technology can suddenly do what we have been asking for…
100%!
Girls are being told to disappear from photos while the apps stay up.
That tells you exactly where the burden is being placed.
Exactly Mark. Very much from the “stop wearing short skirts” school of thinking. 😢
This is SO SAD - and also DREADFUL - that such harm to young people isn’t policed properly & professionally?? 😫
I knew that the law had NOT kept up with technology - but this makes it even worse… 🥲
Thanks so much for posting this - though I understand why some people might not want to read it?
What I don’t understand - given the power of social media & all media (online & offline) to change things - WHY isn’t this being “called out” and publicised so widely & publicly that it isn’t stamped out or at least reduced??
All these government bodies & groups - making NO difference - who funds them because it’s a waste of money??
You & I don’t have the power - who does?
And how can social media be used to challenge these people to act?
Thanks again for sharing this - and I hope there is NO negative impact on you!?
Best always - you are a CHAMPION!! 👍👍❤️
Thank you Gail. I sadly think that the main reason why these issues have not been addressed is because BigTech has worked out that to do so would likely cost their shareholders money. 😢
I don’t think MONEY has or ever will be a good reason to NOT protect children from this type of bullying & abuse?
And I know it’s NOT your fault or mine = it’s a sad comment on today’s world where money seems to be more important that other things that I believe are important = children! Thanks again Sam! 👍
How right you are Gail. 😢
Yep, all of that.
I will say tho that the main other thing I was thinking throughout the article is how do we change the apparently default position of society's framing of women...and then I came to your phrase at the end 'female victim population', and hated that - I feel like even that perfectly valid use of words to say what you mean still inadvertently reinforces the idea of females as victims - those that have been affected and manipulated by others who wish to affect and manipulate them (us, me, women). I think it was PWC that only stopped forcing females to wear high heels about 5 years ago....in appearance we are still told that 'making an effort' is a thing that focuses on making ourselves attractive within an applied societal stereotype...
This is an excellent point Alice. And you are right I could perhaps have used a better phrase to explain this rather than be led towards a position that I agree we need to actively work to change.
I enjoy building things with AI…but then I see stories like this and seriously wonder if the harm outweighs the good.
I know exactly what you mean Karen. But I still think AI has the potential to do great good, we just need to make sure that the people who develop these tools and models take the appropriate actions to protect those amongst us who are most vulnerable.
That’s going to be the challenge!
With automated moderation tools it should be possible for these apps to “card” any nude images. ID plus facial recognition from several angles. Adults who want such images of themselves presumably won’t be afraid to identify themselves, and it would screen out children and revenge “porn”. Or just outlaw AI porn of real people altogether, because no law is going to prevent coercion or force if an abuser is physically present.
As for school children making images of classmates, they absolutely do deserve the legal consequences of their actions. The life they are damaging is as young and valuable as their own (and a whole lot more innocent.)
Lucy, I absolutely agree the tech is there to do this, and we need to ask ourselves seriously what is the reason for this not being done, other than the potential impact it might make on profit for shareholders.
Thank you for bringing this to wider attention. My day job is a boring desk job in the adult industry, and we've been fighting Motherless for years. They've promised "stricter content moderation", but the precedent is still there.
Anyway, I think it's telling that the first action is the take down the photographs and the second is to punish the individuals responsible, instead of actually targeting the source of the problem and taking down the apps.
Exactly this Ana. I wonder if AI will be the same...
Thank you for highlighting this Sam. I cannot think of a single reason why an app of this kind is needed. Therefore, it seems that it can only be about profit and exploitation. It should be illegal to make the apps.