At ZAUBAR we’re deeply passionate about creating AR experiences which are meaningful. In this post, I argue that a state of Deep Tracking must be reached in order for immersive content to be truly meaningful.

‍ZAUBAR’s definition of AR

Augmented reality (AR) adds virtual elements to our environment to create a deeper comprehension of reality, at best to create a sense of higher meaning and truth.

Any AR is computer-based, augmenting our audio-visual experience through high-resolution screens and binaural sounds. Most importantly, any AR is only as good as the tracking it’s based on. At its core, tracking is the computer’s ability to learn and know of reality. The better the tracking is, the more meaningful the augmented reality experiences are.

Over the years, since Ivan Sutherland’s world’s first AR device the ‘ultimate display’, tracking technology has remarkably improved.

Ivan Sutherland’s ‘Sword of Damocles’, 1966. A quirky name coined due to the contraption’s appearance.

Back in the 60s, one of Sutherland’s foundational innovations was the use of headtracking. This type of tracking allowed the head-mounted display to align content accurately to the viewer’s gaze. While the tracking was simple and content was limited to wireframe boxes, it was one of the first steps on AR’s path to Deep Tracking. [1]

Wireframe boxes displayed in Sutherland’s head-mounted display. See video here.

I’d like to argue that Deep Tracking denotes the moment when multiple advanced tracking technologies, linked together, enable a leap in the user’s knowledge of their environment through augmented reality.

At ZAUBAR we’re chasing to enable these leaps.

Deep Tracking

In our work,, we’re combining a variety of tracking methods to create deeply meaningful experiences. Among the computer’s most important insights into its environment is the recognition of earth and people. In terms of augmented reality development this is commonly referred to as human segmentation and ground tracking.

Algorithms detect humans, their shape and the computer’s orientation relative to the world.

In terms of augmented reality experiences, the computer now knows about the existence of people and where they are in the world – at least in the small area covered by the computer’s camera.

But there is still one last step missing to make a profound experience possible. Right now the computer only knows that there’s a human standing somewhere.

In order to create meaningful augmented reality, Deep Tracking needs to be achieved. Here’s where location tracking comes into play.

Today’s techniques are much more precise than traditional GPS and inform the computer of its hyper accurate location on planet earth. State-of-the-art location tracking allows the computer, in our case the phone, to exactly know on what street it is, in which direction it’s pointed, where it’s heading – down to the centimetre.

With centimetre-accuracy, the phone has precise 3-dimensional contextual knowledge of a city, here San Francisco.

Given this accuracy, this allows for very unique augmented reality experiences.

Although, technically, most people refer to this type of AR as location-based, we’re more thinking of it as meaningful augmented reality.

Now that the computer has deep knowledge of the world, it is in turn able to teach us about it.

Here at ZAUBAR, we’re very intrigued by the powers of revealing hidden stories and timelines. We think that we’ve found one way to make AR meaningful by displaying virtual content in a most contextual way.

[1] Rheingold, H. (1992). Virtual Reality, Simon & Schuster, New York, N.Y.