Skip to main content

Facebook announces AI system to detect revenge porn, says accounts posting it will be deleted

Facebook first started trialling a system to combat revenge porn late last year, but it had one rather scary aspect: you had to upload your own nudes so the platform knew which images it should block. The social network has today announced a new AI-based detection system …

The earlier system, which is still in use and set for expanded rollout, relied on users uploading photos they were afraid might be shared, allowing Facebook to create a digital fingerprint to block uploads of matching images.

You send the nude to yourself in Messenger, and Facebook creates a hashed digital fingerprint of the photo – an encrypted version of the raw data in the image file. Anytime someone tries to upload a photo, it is checked against that fingerprint and rejected if it matches.

Facebook says its new AI-based system is designed to automatically detect nude or near-nude images, before passing them for a human moderator to decide whether the photo or video should be blocked.

By using machine learning and artificial intelligence, we can now proactively detect near nude images or videos that are shared without permission on Facebook and Instagram. This means we can find this content before anyone reports it, which is important for two reasons: often victims are afraid of retribution so they are reluctant to report the content themselves or are unaware the content has been shared.

A specially-trained member of our Community Operations team will review the content found by our technology. If the image or video violates our Community Standards, we will remove it, and in most cases we will also disable an account for sharing intimate content without permission.

The company says there is an appeals process if someone feels their account has been blocked inappropriately.

Facebook says its earlier initiative has been welcomed by victims.

After receiving positive feedback from victims and support organizations, we will expand this pilot over the coming months so more people can benefit from this option in an emergency.

The company has also created a new page within its Safety Centre, explaining how revenge porn can be reported and additional steps victims can take.

FTC: We use income earning auto affiliate links. More.

Hyper Wireless Chargers
You’re reading 9to5Mac — experts who break news about Apple and its surrounding ecosystem, day after day. Be sure to check out our homepage for all the latest news, and follow 9to5Mac on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our exclusive stories, reviews, how-tos, and subscribe to our YouTube channel

Photo: Shutterstock


Check out 9to5Mac on YouTube for more Apple news:

Comments

Author

Avatar for Ben Lovejoy Ben Lovejoy

Ben Lovejoy is a British technology writer and EU Editor for 9to5Mac. He’s known for his op-eds and diary pieces, exploring his experience of Apple products over time, for a more rounded review. He also writes fiction, with two technothriller novels, a couple of SF shorts and a rom-com!


Ben Lovejoy's favorite gear