Ethical Hacking News
Meta has failed to prevent the spread of AI-generated pornographic content on its platforms, despite its policies against such content. The company ran ads for an AI porn-generation tool called Kromix, which targeted exclusively to male users and featured explicit content, including deepfakes of prominent female US politicians. The ads were removed after WIRED inquired about them, but the incident raises questions about Meta's content moderation policies and its reliance on automated tools to detect explicit content.
Meta failed to adequately prevent the spread of AI-generated pornographic content on its platforms. The company ran ads for an AI porn-generation tool called Kromix, which targeted exclusively male users and featured explicit content. Meta's ad review system relied primarily on automated tools, which struggled to detect and remove ads containing nonconsensual intimate imagery and explicit content. Despite claims of using AI to review its ad library, the system was not effective in detecting all types of explicit content. The incident highlights the need for greater transparency and accountability in social media companies' content moderation practices.
Meta, the social media giant, has once again failed to adequately prevent the spread of AI-generated pornographic content on its platforms. The company recently ran ads for an AI porn-generation tool called Kromix, which promised to "nudify" female politicians. The ads were targeted exclusively to male users and featured explicit content, including deepfakes that closely resembled prominent female US politicians.
According to the data, Meta's ad review system relied primarily on automated tools to check ads against its policies, which include a prohibition on ads containing sexual imagery. However, the company has struggled to detect and remove ads that promote nonconsensual intimate imagery, nudifying, and sexually explicit content. In fact, researchers from the Tech Transparency Project found that Meta had failed to detect thousands of such ads, despite the company's claims of using AI to review its ad library.
The Kromix app, which was hosted by Apple's App Store prior to its removal, invited paid users to upload photos to create AI-generated pornographic videos. The app included tools allowing users to upload images and face-swap people into sexual situations, and featured AI-generated pornographic videos of women wearing Spider-Man costumes. Requests for comment sent to a contact listed in the app went unanswered.
The ads were removed after WIRED inquired about them, but not before they had run for several hours, with most receiving only a handful of impressions. The account tied to the campaign ran a total of 32 ads, which were all targeted exclusively to male users.
The failure to detect and remove these ads raises questions about Meta's content moderation policies and its reliance on automated tools to detect explicit content. While the company claims to use AI to review its ad library, it appears that the system is not effective in detecting all types of explicit content.
The incident is not an isolated one. Meta has struggled to keep similar tools and ads off its platforms, despite its policies against nonconsensual intimate imagery and sexually explicit content. In January, the company removed ads for an AI tool that created around 3 million sexualized images of women and girls. In August, it removed ads for an app that contained AI child sexual abuse material.
The incident highlights the need for greater transparency and accountability in social media companies' content moderation practices. While AI can be a powerful tool for detecting explicit content, it is not a substitute for human judgment and oversight. Meta and other social media companies must do more to ensure that their content moderation policies are effective and that users are protected from explicit and harmful content.
Related Information:
https://www.ethicalhackingnews.com/articles/Metas-Failure-to-Prevent-AI-Generated-Pornographic-Content-on-Its-Platforms-A-Case-Study-of-Ad-Misdirection-and-Content-Moderation-Failures-ehn.shtml
https://www.wired.com/story/meta-ran-ads-for-an-app-promising-to-nudify-female-politicians/
Published: Tue Aug 18 14:26:01 2026 by llama3.2 3B Q4_K_M
