Skip to main content

Facebook improves its AI that generates photo descriptions on iOS for visually impaired users

Facebook today announced a major update to its Automatic Alt Text (AAT) system, which automatically generates photo descriptions for people with visual impairments. Based on AI, the system can now recognize 10 times more objects and scenes in photos shared on Facebook and also Instagram, providing detailed descriptions for screen readers like Apple’s VoiceOver.

According to Facebook, the artificial intelligence technology developed by the company can now identify activities, landmarks, types of animals, and various other details in the photos. This will make it easier for blind or visually impaired people to understand the context of a photo shared on social networks.

Instead of just describing what’s in the photo, the updated AAT system can also specify the composition of the image, such as saying that there are two people in the center of the photo and three others in the sides instead of just saying that there are five people in the photo.

Or, instead of simply describing a lovely landscape with “May be a house and a mountain,” we can highlight that the mountain is the primary object in a scene based on how large it appears in comparison with the house at its base.

Taken together, these advancements help users who are blind or visually impaired better understand what’s in photos posted by their family and friends — and in their own photos — by providing more (and more detailed) information.

As pointed out by CNET, the new AAT system will be available on Facebook for photos in News Feed, Profiles, and Groups, while on Instagram the generated descriptions will be available for photos in Feed, Explore, and Profile. The feature is compatible with 45 different languages.

To access the detailed photo descriptions on iOS, users must use the new custom actions through VoiceOver. On Android, you just have to long-press the image. The update is still being rolled out for users.

FTC: We use income earning auto affiliate links. More.

You’re reading 9to5Mac — experts who break news about Apple and its surrounding ecosystem, day after day. Be sure to check out our homepage for all the latest news, and follow 9to5Mac on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our exclusive stories, reviews, how-tos, and subscribe to our YouTube channel



Avatar for Filipe Espósito Filipe Espósito

Filipe Espósito is a Brazilian tech Journalist who started covering Apple news on iHelp BR with some exclusive scoops — including the reveal of the new Apple Watch Series 5 models in titanium and ceramic. He joined 9to5Mac to share even more tech news around the world.