Skip to main content

Latest ARKit demo shows off 3D finger painting through augmented reality

We’ve already seen dozens of examples of Apple’s new ARKit framework in use from navigation enhancements to the Falcon 9 landing, and now developer Osama Abdel-Karim has shown off augmented reality finger printing with a 3D twist.

The latest ARKit demo makes it possible to simply draw in the air with your finger as if you were holding a pen, and ink realistically appears through software as if it were on the surface. After your illustration is complete, the demo optionally lets you add an effect that brings the AR drawing to life as a 3D object.

The 3D finger painting demo uses a combination of Apple’s ARKit framework and an iOS 11 tool called Vision that tracks the fingertip through software to create the realistic finger painting effect. The developer writes more about the process:

One of the cool libraries that Apple introduced in iOS 11 is Vision Framework. It provides some computer vision techniques in a pretty handy and efficient way. In particular, we are going to use the object tracking technique. Object tracking works as follows: First, we provide it with an image and coordinates of a square within the image boundaries for the object we want to track. After that we call some function to initialize tracking. Finally, we feed in a new image in which the position of that object changed and the analysis result of the previous operation. Given that, it will return for us the object’s new location.

A full detailed write-up including source code further explains how the demonstration was created. See it in action below:

Apple first unveiled ARKit as part of iOS 11 for iPhone and iPad back in June at WWDC 2017. So far we’ve only seen hobby projects of how the framework can be used in the real world. When iOS 11 ships later this fall, we’re expecting to see tons of ARKit applications from launch partners including IKEA.

FTC: We use income earning auto affiliate links. More.

You’re reading 9to5Mac — experts who break news about Apple and its surrounding ecosystem, day after day. Be sure to check out our homepage for all the latest news, and follow 9to5Mac on Twitter, Facebook, and LinkedIn to stay in the loop. Don’t know where to start? Check out our exclusive stories, reviews, how-tos, and subscribe to our YouTube channel



Avatar for Zac Hall Zac Hall

Zac covers Apple news for 9to5Mac and hosts the 9to5Mac Happy Hour and 9to5Mac Watch Time podcasts.