Facebook’s iOS app uses AI and VoiceOver to describe the contents of photos to blind users
At a time when so many Facebook posts comprise a photo and a brief comment, there’s one group of people who get rather left out of the picture: those who are blind and partially sighted. That’s a problem Facebook is fixing, starting from today. The iOS app now uses artificial intelligence to figure out the content of photos, and Apple’s VoiceOver feature to read aloud a description of them.
The Verge got a demo of the feature, which Facebook calls ‘automatic alt text.’
Automatic alt text, which is coming to iOS today and later to Android and the web, recognizes objects in photos using machine learning […] While still in its early stages, the technology can reliably identify concepts in categories including transportation (“car,” “boat,” “airplane”), nature (“snow,” “ocean,” “sunset”), sports (“basketball court”), and food (“sushi”). The technology can also describe people (“baby,” “smiling,” beard”), and identify a selfie.
Facebook said that there were two approaches it could have taken to the problem, and it chose the one it believed would be most successful …