![Pixel 9a vs. Pixel 8a: The A-series is all grown up [Video]](https://i0.wp.com/9to5google.com/wp-content/uploads/sites/4/2025/04/Pixel-8a-and-Pixel-9a-wide.jpg?resize=1200%2C628&quality=82&strip=all&ssl=1){kind=tracking}
Naughty AIs Are Spilling Their Users' Super Personal Chats Onto the Open Web
Steady Drip A new report from the security firm UpGuard reveals that some NSFW chatbot sites are oozing explicit user chat contents into the open web — and what those chats contain can be disturbing. According to Wired, UpGuard's investigation focused on 400 "exposed" AI services, all of which were built on an open-source AI protocol called llama.cpp. Researchers from UpGuard were able to determine that 117 IP addresses connected to these poorly built services were, in fact, leaking user prompts into the digital wild, and that three such systems were leaking extensive, sexually explicit prompt data into the digital […]
Steady Drip
A new report from the security firm UpGuard reveals that some NSFW chatbot sites are oozing explicit user chat contents into the open web — and what those chats contain can be disturbing.
According to Wired, UpGuard's investigation focused on 400 "exposed" AI services, all of which were built on an open-source AI protocol called llama.cpp. Researchers from UpGuard were able to determine that 117 IP addresses connected to these poorly built services were leaking user prompts into the digital wild, and that three such systems were leaking data from extensive, sexually explicit interactions with erotic chatbots.
Over 24 hours, UpGuard collected nearly 1,000 leaked user chats. Disturbingly, five of them centered on child sexual abuse scenarios.
Though UpGuard couldn't determine which specific AI sites or services were leaking the exposed prompts, according to Wired, they were able to make out that the sites are host to roleplay-oriented chatbots designed to embody specific "characters."
UpGuard's report is alarming on multiple levels. On its face, that sexually intimate user interactions with AI chatbots are drifting out into the open web is a massive privacy issue. At the same time, those privacy gaps reveal some of the darkest underbellies of generative AI, where people use unregulated AI services to engage in horrifyingly abusive — and alarmingly accessible — sexual fantasies.
Large language models "are being used to mass-produce and then lower the barrier to entry to interacting with fantasies of child sexual abuse," Greg Pollock, the VP of product at UpGuard and the author of the report, told Wired.
Darkest Corners
To Pollock's point, this is far from the first time that generative AI has been used to create or consume child pornography or engage in pedophilic fantasies.
Last week, Wired reported that yet another exposure incident at a South Korean AI image generator startup revealed a horrifying trove of AI-generated sexual deepfakes, including sexual synthetic images of celebrities de-aged to look like children (the company deleted the whole website as a result.)
And regarding character-based chatbot services specifically, MIT Technology Review recently found that a chatbot provider called Botify AI was similarly hosting chatbot versions of sexualized, de-aged celebrity women, while a Futurism investigation last year found that the chatbot startup Character.AI — which is currently fighting two child welfare lawsuits — was hosting chatbots expressly designed to groom underage users.
More on AI and abuse: AI Startup Deletes Entire Website After Researcher Finds Something Disgusting There
The post Naughty AIs Are Spilling Their Users' Super Personal Chats Onto the Open Web appeared first on Futurism.
