Augmented reality (AR) has been featured in popular entertainment since well before practical AR technology was available to consumers. Now, as more and more AR products come to market, it’s worth taking a look at both the predictive and, at times, fanciful ways filmmakers represent augmented reality on screen. Though the HUD (Head-Up Display) is by far the most commonly used representation of AR, it certainly isn’t the only one.
Augmented reality is any piece of technology that, through its use, augments the perception of reality by adding information to the real world. While something like The Matrix alters the realities of the humans trapped within it, it doesn’t add to their reality — it replaces it. As such, it’s virtual reality — which has its own place in popular entertainment.
Let’s start at the beginning…
One of the first movies to utilize a simple, innocuous heads-up display was an interesting one, and it actually came out two years before the term “augmented reality” was coined by Boeing researcher Tom Caudell. In They Live, Roddy Piper’s character John Nada finds a pair of sunglasses that reveal hidden aspects of his reality — they allow him to see through the disguises of aliens living among us. While the tech behind the sunglasses isn’t belabored, they’re very much a HUD that gives their wearer additional information about what he’s seeing.
“Nosedive,” the season 3 premiere episode of Black Mirror, showed a simpler version. While mostly a cautionary tale about the consequences of social media — not too far removed from the MeowMeowBeenz episode of Community (ep. 508 “App Development and Condiments”) — it also tackles another interesting element of augmented-reality futurism with the augmented lens. Far less obvious than even Nada’s sunglasses, this posited tech holds the entirety of your visual augmented-reality interface within a set of contact lenses. And while this feels far more like the stuff of science fiction, and less like something we’ll be able to achieve any time soon, actual futurists disagree. In his book “Physics of the Future,” Michio Kaku addresses the likelihood of an internet-connected HUD housed within a contact lens as a real possibility. Researchers have already managed to install LEDs into a contact lens, and are also considering the possibility of one day using nano-lasers to beam light directly into your retina for your brain to reassemble as high-resolution images. So let’s keep our fingers (and possibly our eyes) crossed for those.
Moving away from the wearables for a moment, what’s also interesting about They Live — other than everything — is that Nada’s sunglasses don’t actually augment his reality, they remove augmentation from his reality by peeling away the layers of deception surrounding the aliens. This leads to a really intriguing implication — there are, in fact, competing forms of AR in this movie. Nada’s sunglasses are only half of the equation. The other half is the technology the aliens use to augment reality for everyone else on Earth. Nada’s glasses are simply a means by which he can counteract a much more powerful and all-encompassing technology. As augmented reality technology grows, will this sort of mass alteration become a possibility? Will people have the option to cloak what they don’t want to see?
As we see augmented reality come into the mainstream and alter the very way we see the world around us, will there be people like Nada who would rather see things as they are?
THE TACTICAL HUD
A much more common reflection of AR in cinema deals with HUDs designed explicitly for tactical use in military applications. Even Luke Skywalker flirts with augmented-reality targeting tech during his Death Star run, only to retract his targeting scope and rely on the Force instead… Of course, that worked out pretty well for Luke since he was uniquely attuned to a metaphysical presence tying all of reality together. For the rest of us, there’s AR.
But, let’s shift focus to some augmented characters who don’t have the choice to abandon their HUDs completely.
Arnold Schwarzenegger’s Terminator, for instance, doesn’t wear a visor or a helmet — his HUD is built directly into his head. In a movie (multiple movies, actually) that doesn’t spend too much time in direct, first-person character POVs, we know that each time we cut to such a point of view, we’re going to see that HUD display. And because we learn early on that his HUD is focused primarily on finding and assessing targets and weighing threat levels, tension builds each time we slip into that red-saturated point of view
Robocop has a similar set-up in his display, though in this case his is a retractable (if not removable) visor, giving him the option of seeing the world raw when he wants to. The key difference between Robocop and the Terminator — other than cyborg vs robot — is that while The Terminator trusts his HUD, Robocop has a tendency to distrust his as he struggles with his new state of being. Though the functionality remains very similar — target, assess, destroy.
In both The Terminator and Robocop, the HUD serves two primary purposes — 1) to raise tensions and 2) to remind us that these men aren’t quite human. While those are a large part of the threat in the former, they serve as a key source of internal conflict in the latter. It’s that second part that bears attention. Even as we focus on consumer AR products that we may see in the next 5, 10, or 15 years, robotics is also advancing. It stands to reason that if we ever get to the point where robots also become ubiquitous that some form of HUD will be incorporated into their visual components.
Then we have Iron Man, neither robot nor cyborg — though his cardiologist may disagree. He indeed has an advantage over both the Terminator and Robocop in that he can step easily from one state to the other by donning his Iron Man suit. At the same time, his HUD conveys far more information than his predecessors’. Yes, he has targeting and tracking capabilities, as well as scanning and threat assessment, but he also has the added benefit of a voice interface to assist him with a second set of eyes on his readouts and the ability to help him multitask.
As we see in the first Avengers, Tony Stark’s HUD isn’t exclusively focused on tactics and attacks — it also has diagnostic and scientific capabilities. As such, he isn’t limited to shooting his way through problems, he can also aid in the repair of a giant, flying aircraft carrier engine mid-flight. I suspect both the Terminator and Robocop would be fairly useless in that scenario — even if their intentions were pure… and they somehow got a hold of some rocket boots. But that’s another element that sets Stark’s HUD apart: His suit serves as a personal conveyance. Iron Man is a suit of armor, yes, but it’s also a vehicle, and Tony’s HUD has a flood of information utilizing the entirety of his field of vision.
Already in the move toward augmented vehicles, we’re seeing increased functionality in the front windshield and dashboard — whether that’s the simple inclusion of apps or a visible overlay outlining the path to a given destination. When we hit a peak for self-driving cars, it’s reasonable to assume the windshield will be converted almost entirely into a fully functioning AR interface for passengers. This would certainly make for a more interesting and diverse slate of travel games, not to mention greatly improving the tedium of rush hour.
Of course, the AR of the Marvel Cinematic Universe is not limited to Tony Stark’s suit. We also get a heaping helping of interactive holographic displays that Tony uses to view files, design suits, and engineer solutions to huge problems. It’s a creative person’s dream — a limitless 3D whiteboard to play around with ideas on a much bigger scale than would otherwise be possible without making a huge mess.
And this particular movie universe isn’t the first to explore such appealingly futuristic interfaces. Minority Report, if you recall, processed its pre-crime reports through an expansive holographic screen that Tom Cruise’s John Anderton manipulated with a pair of specialized gloves. Now this sort of display is being used more and more in science fiction, but at the time it was a breath of fresh air.
HOW CLOSE ARE WE?
Despite a somewhat limited imagination regarding the breadth of ways in which AR might be utilized, filmmakers have done a fair job of hewing closely to realistic futurism. In fact, this may not be all that different from the way the various Star Trek series succeeded in predicting approximations of smart phones, iPads, and even automatic doors. What today’s filmmakers have done far more successfully is make a lot of this tech appealing.
The AR future they’ve depicted is closer to the present than ever before.