Augmented reality goes mainstream with mobile applications
Competition between Apple, Google, and third-party vendors will push augmented reality programs onto smartphones faster than anyone has predicted.
Chances are your first introduction to augmented reality (AR) was in the 1986 movie Top Gun where our fighter pilot heroes fly planes with heads-up displays that let them monitor their planes and weapons while watching their flight. Now, augmented reality is moving from planes, high-end cars, and military helmet mounted displays to more consumer and personal settings. Indeed, AR is on the verge of becoming commonplace and commercially successful.
Augmented reality interlaces our vision with digital information. Robert Rice, chairman of the Augmented Reality Consortium defines AR as "any media that is specific to your location and the context of what you are doing (or want to do) and augments or enhances your specific reality."
So what does that mean? It means, with new applications like Google Goggles, you can use your phone's built-in camera to 'search' your enviroment by taking photos of your surroundings and Google will report back with information about what you're looking at. While Google is getting the headlines, especially now that it's toying not only with its own mobile device operating system, Android, but its own mobile phone, the Nexus One, they're not the only ones playing with augmented reality.
For example, with Layar 3.0, an Android mobile-phone application, you can also have your mobile camera 'look' at your surroundings and Layar will supplement what you're viewing with information on your display about what you're seeing.
The smartphones of 2009 can do this because they've grown more powerful, have faster broadband connections, and have extra features that enable them to run augmented reality applications. For example, it's not that easy to identify a location just from a camera view, but when you add in Global Positioning System (GPS), compasses, and hybrid positioning systems, which use Wi-Fi access points and cell towers for location, it becomes possible to create location-based augmented reality applications.
Joe Madden, an ABI Research analyst, sees "hand-held platforms transforming the Augmented Reality ecosystem, with revenue associated with Augmented Reality growing from about $6 million in 2008 to more than $350 million in 2014." The iPhone, with its current, November 2009, 3% market-share but with 500% year-over-year growth, according to Gartner, appears to be leading the way with augmented reality location-based services.
These predictions were all made before Google jumped in with its Goggles and, now, it seems, with its own phone. Competition between Apple, Google, and third-party vendors should push augmented reality programs on all our smartphones at faster clip than anyone has predicted.
How augmented reality works
So how is augmented reality able to 'augment' our reality? Dan Greenberg, an independent AR analyst, explains that augmented reality applications do this by "registering" an image. This is the process by which the augmented reality application figures out exactly what it is looking at. Today, this works by a combination of techniques. These include image recognition -- that tall structure sure looks like the Eiffel Tower -- and location-based services (LBS), which are supported by the device. For example, the compass in the Apple iPhone 3Gs helps by plotting out not only the device's location but also its orientation.
Blair MacIntyre, a professor at Georgia Tech's Augmented Environment Labs explains, "The key to many AR applications is that the information is registered (i.e., aligned) with the view of the physical world. To overlay graphics (or sound) correctly on the view of the world, the computer (or phone) needs to know precisely where the world is in relation to the camera. The more accurately the position and orientation of the device is known, the more accurately the graphics and the physical world can be combined."
That's easier said than done, continued MacIntyre, "On a smart phone, you can try and use the GPS and other sensors (e.g., compass, accelerometers) to estimate the position and orientation of the phone relative to the world; the problem is that the limited accuracy of those sensors restricts the capabilities of these applications."
For example, said MacIntyre, "I have been showing a video I made of one of the 'information browsing' systems on the iPhone, where I look around the courtyard in front of my building, and none of the labels align with the businesses they are referring to. This is not a bug in the application; it's a limitation of the accuracy of the GPS, and is common to all GPS-based AR applications on mobile phones today."
Sean Everett, co-founder of the mobile application marketing firm Evolyte agrees. "The iPhone's GPS and location tracking software is currently not powerful enough to pin-point locations down to a foot or so. If it was, you could imagine walking past a retail store and your phone buzzing you that there was a pair of jeans for sale in your size 6.5 feet inside the door and 8 feet to the right, bottom shelf, 3rd from the bottom. Or you could imagine real-time reservations popping up requesting you to eat at a restaurant you were walking by (say goodbye to the hostess)."
Google thinks it has an answer -- a new barcode system called Moseycode. With it, anyone can place Moseycodes on whatever they want and a Moseycode aware application can then 'read' its information. Once this 'bridge' between reality and its corresponding Web-based information is made, it would be simple to find Everett's jeans.
There are no standards yet for augmented reality applications. The closest we currently have is the W3C's draft Geolocation API Specification. So, for now, the AR applications tend to be one-offs, but Google's move may quickly standardize how they work.
Google isn't the only one trying to bring rhyme and reason to this quickly expanding field. Augmented reality company Mobilizy is attempting to bring order to AR browsers with an HTML-style standard language for augmented reality called ARML (Augmented Reality Mark-up Language). As Rice, chairman of the AR Consortium, observed "Companies are beginning to turn away from the novelty and focus on building an industry."
Augmented reality today
At this point, augmented reality applications are works in process. You can find the most AR applications for advanced smartphones like the HTC T-Mobile G1 handset, which runs Google's Android OS, and the iPhone. Perhaps the most advanced of these applications is SPRXmobile's Layar Reality Browser 3.0. This application superimposes an AR browser over the phone's camera view and presents you with layers of information about the immediate area. You can pan your cell phone camera around and see nearby buildings and spaces tagged with information drawn from Web sources such as Flickr, Wikipedia, and Yelp.
Layar faces competition not just from Google Goggles but from Mobilizy, which has a similar program Wikitude, an AR browser that's works on both Android and Nokia's Symbian platform.
Besides the general purpose augmented reality browsers, there are more specific-purpose applications like AcrossAir's Nearest Tube, and Nearest Subway. These iPhone applications can help you find the nearest subway stations in London and New York City respectively. Similar navigation programs for other cities are coming.
The general purpose augmented reality programs are also being used as the foundation for other applications. For example, with Layar you can take a virtual Beatles tour around London's Abbey Road neighborhood. And other similar tourist applications are already appearing.
Another use of augmented reality is as guides for the blind. vOICe for Android is exploring what can be done with an AR application that tells an Android device user about his or her environment.
And, you don't have to depend on companies for information. While Moseycode is still coming together, users will be able to label whatever they want with Moseycode labels. Tonchidot, a Japanese company, has created the Sekai Camera -- a social tagging service for both Android phones and the iPhone -- that allows users to walk around and see notes from their friends about a location. For example, great pizza here!
Or, to solve a problem near and dear to me, there's Intridea's Car Finder (iTunes link), which uses augmented reality and the iPhone 3Gs location-based service to help you find your car in the parking lot.
This just touches the surface of what can be done with consumer mobile augmented reality applications. There will be more, many more -- an AR guide to your college (or business) campus? Why not? -- and at this stage it's hard to predict exactly what other forms augmented reality will take. It's safe to say though that we're going to see a wide variety of applications.
While augmented reality for the military and industry has been around for years, and we don't even notice such AR enhancements as the yards for a first down markers on television football broadcasts, personal mobile AR is still in its infancy. That said, everything seems to be coming together to make AR the next big thing in personal technology.
The devices have the LBS hardware they need to register their locations; their processors and broadband connections are fast enough to handle the data requirements; and the software infrastructure is coming together quickly. Will 2010 be the year of augmented reality? Maybe not. But, AR is on the verge of going mainstream, and it will be fascinating to see just how it will change our lives.