Skip to main content

Apple working on advanced 3D cameras with object and gesture recognition

This is not the first time an Apple patent has surfaced relating to three-dimensional camera technologies. A previous patent highlighted advanced 3D object recognition and verification. A new patent—published today by the United States Patent & Trademark Office and detailed by PatentlyApple—shows Apple is continuing to work on 3D camera technologies that could land in future iOS devices. Apple’s patent described a 3D imagining camera that uses advanced microlenses, depth-detection, chrominance, and luminance sensors. The camera could recognize facial expressions and gestures while creating 3D models of scanned objects. PatentlyApple explained:

Apple has invented a killer 3D imaging camera that will apply to both still photography and video. The new cameras in development will utilize new depth-detection sensors such as LIDAR, RADAR and Laser that will create stereo disparity maps in creating 3D imagery. Additionally, the cameras will use advanced chrominance and luminance Sensors for superior color accuracy. And if that wasn’t enough, the new cameras will not only include facial recognition but also facial gesturing recognition.

According to Apple, current 3D imaging devices cannot pull detailed/accurate geometric data of 3D objects and are “incapable of extracting depth information.” Apple’s patent outlines various methods of using several dedicated 3D cameras. As usual, Apple mentioned the camera could be built into “digital cameras, personal computers, personal digital assistants (PDAs), mobile telephones, a standalone camera,” or any device capable of processing the image data.

As for what this might mean for the end user, the report described two intriguing applications for the technology, such as facial and gesture recognition and 3D models created from video of rotating objects.

Facial gestures may include, but are not limited to, smiling, grimacing, frowning, winking, and so on and so forth. In one embodiment, this may be accomplished by detecting the orientation of various facial muscles using surface geometry data, such as the mouth, eyes, nose, forehead, cheeks, and so on, and correlating the detected orientations with various gestures.

The camera could also create a 3D digital model of an object by taking video or photographs of it while it is rotating. The image data can even be captured by rotating the device around the object. PatentlyApple described another embodiment of the feature that would allow users to create a 3D model of their home by shooting video while walking through it:

a user may take video of a home while walking through the home and the image sensing device could use the calculated depth and surface detail information to create a three-dimensional model of the home. The depth and surface detail information of multiple photographs or video stills may then be matched to construct a seamless composite three-dimensional model that combines the surface detail and depth from each of the photos or video.

PatentlyApple offered more information about a different Apple patent today that details another 3D avatar creation app. The report described it as “like Photo Booth on steroids and on the other hand it’s definitely being aimed at the 3D video game market.” You can get all the details here.

Related articles

FTC: We use income earning auto affiliate links. More.

You’re reading 9to5Mac — experts who break news about Apple and its surrounding ecosystem, day after day. Be sure to check out our homepage for all the latest news, and follow 9to5Mac on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our exclusive stories, reviews, how-tos, and subscribe to our YouTube channel



Avatar for Jordan Kahn Jordan Kahn

Jordan writes about all things Apple as Senior Editor of 9to5Mac, & contributes to 9to5Google, 9to5Toys, & He also co-authors 9to5Mac’s Logic Pros series.