How lip reading could help improve online security

A new study shows that visual speech patterns could be the basis for a more secure method of verifying our identities online

Picture of a bench shaped like human lips outside of Baci restaurant along the Yarra RiverImage credit: flickr/Andy Blackledge (license)
The solution to the hacking problem?

The recent hacking of celebrity iCloud accounts reminds us all, once again, of the need for more secure methods of authentication for our online accounts. While 2-factor authentication is becoming increasingly popular, researchers are looking into other, still more secure methods of verifying a user. Biometric-based forms of authentication, such as fingerprints and iris scans, are potentially more secure than password-based methods, but they usually require additional hardware which hinders their widespread implementation.

Now comes a new approach from Ahmad Basheer Hassanat, a researcher in Jordan, who has shown that authentication based on lip reading is a viable and attractive alternative method of authentication. Hassanat’s proof of concept, Visual Passwords Using Automatic Lip Reading, was recently published in the International Journal of Sciences: Basic and Applied Research. In addition to providing the benefits of a biometric-based approach (since it’s based on who you are), authentication based on visual speech patterns would only require a device to have a camera (not even a microphone), and no other special hardware.

The method that Hassanat outlines is a two-stage approach:

Stage one is a training stage, in which a video recording is made of a user speaking a number of different words. Each word generates a “feature vector” which is a matrix of data from each frame in the video. Data collected includes measurements of the height and width of the mouth, the number of visible teeth, the amount of visible red color (i.e., the tongue), how these change from frame to frame, etc. Audio plays no role in this method, so, in theory, the user could just mouth the words. A “visual password” (VP) is chosen based on one or more of the target words.

Stage two is verification, when a user wants to log into the system. A video is recorded of the user saying a password and, again, a matrix of data is generated. These data then get compared to the reference data generated in the training stage and the user is authenticated or rejected based on how “close” their data is to that of a valid user.

Hassanat tested his approach using 20 randomly selected subjects (10 male and 10 female). After a training stage, he had users, some valid, some impostors, attempt to be verified in a number of different ways. Sometimes, the users actually knew the VP and sometimes they didn’t. Sometimes the VP was a single word and sometimes it was multiple words. 

Overall, the success rates (correctly verifying a valid user, correctly rejecting an invalid user) were pretty promising. Even when imposters knew the VP, the system worked (i.e., blocked their access) at least 80% of the time. When imposters didn’t know the VP, and the VP was two words, the system worked 92% of the time. A three or four (or longer) word VP, one would expect, should be even more effective.

Hassanat did find, surprisingly, that the success rate of the system decreased when users only whispered or mouthed the words. So, even though audio doesn’t play a role in this method, there’s something about actually putting voice behind a word that affects facial movements. Also, again not surprisingly, Hassanat found that facial hair made it harder for the system to properly authenticate or reject a user.

This is a potentially promising way to enhance the security of our online accounts. Who knows? Maybe in a few years we’ll all be speaking to our phones or computers to login into an account. We make look silly talking to our devices, but it might be worth it to keep our data more secure.

Read more of Phil Johnson's #Tech blog and follow the latest IT news at ITworld. Follow Phil on Twitter at @itwphiljohnson. For the latest IT news, analysis and how-tos, follow ITworld on Twitter and Facebook.

ITWorld DealPost: The best in tech deals and discounts.
Shop Tech Products at Amazon