It's a really exciting time to be a geek. In just a few years we've gone from punching buttons to issuing commands via voice. I talk to a speaker in my kitchen (the Amazon Echo), to my TV through a number of devices, to my phone of course, and to my computer, and it feels very natural to do so.
Sometimes I just wave my hands to get things done thanks to devices like Microsoft Kinect and Leap Motion. I feel like these experiences aren't yet as dialed-in as voice commands, but they're coming along.
What's next? Well if Tobii has their way, it might be eye-tracking. Yesterday the company announced its new IS4 eye-tracking module. This isn't something you're going to purchase directly as a consumer; Tobii is pitching the tech to manufacturers of laptops, tablets and other devices.
IS4 is the 6th generation of eye-tracking tech from Tobii and it is 'cost optimized' (aka cheap enough) and has low enough power consumption to be used in a wide variety of consumer devices. In other words, they can put it in a phone or tablet without sucking your battery dry.
So what does this mean for future devices? Time for a thought experiment.
I have friends who play a lot of space exploration game Elite: Dangerous. Some of them are hardcore enough that they've set up head tracking systems like the TrackIR to help them control the game. By moving their heads they can look around the inside of their space ship's cockpit, bringing off screen control panels onto the screen without letting go of their flight controls. From all reports TrackIR works pretty well but it's expensive. What if your PC had the IS4 system built in? Developers could support head tracking using it.
Head tracking is also useful with the many VR and AR products poised to arrive in consumer hands in the years to come. But eye tracking offers even more for VR applications. It enables "foveated rendering" which means that the application/device you're using focuses its graphics processing power on the subject you're looking at, and objects in your peripheral vision are rendered at lower resolutions. For example, if you're in a VR world and there's a car and a person in front of you, when you look at the person, IS4 reports that to the visor and the person is rendered at maximum resolution but the car starts being rendered at a lower resolution. When you then shift your focus to the car, the reverse is true. The idea is that you can squeeze better graphics out of your hardware if it doesn't have to render the entire screen at the same resolution.
So what else? Well as I said this is the 6th generation of product from Tobii. Their website lists a selection of apps that work with their older hardware and what the hardware enables in that app. For instance in the game Son of Nor:
Son of Nor is about magic, and now it actually feels like magic to play, thanks to its new eye tracking feature, Natural Targeting. Use your gaze to decide where you want to do something, instead of being forced to always center your screen on the area you want to affect. Now you can look at rocks, terrain or enemies anywhere on the screen, then press the mouse button to telekinetically lift a rock, terraform or throw something at your enemy, without having to aim with the mouse. These features make for a much more natural way to interact with the game – and turn the gaming experience into a much richer and more compelling one.
That sounds interesting but not like a killer app, and I'm not sure we have an eye tracking killer app yet. This is still a very new technology and to some extent I feel like we have a chicken and egg situation. Once a lot of developers start playing with the technology I'm sure they'll come up with all kinds of interesting ideas. The trick for Tobii is going to be getting OEM's interested in including the technology in the first place; without a lot of compelling software support it could just seem like an extra expense to hardware manufacturers.