The system, which is designed to translate both spoken words and written text on road signs and other instructions that are vital gibberish to monolingual travelers, has to be trained for an hour or so to get the intonations and vocal qualities of one speaker correct, according to Microsoft Principal Researcher Frank Soong of Microsoft's Beijing research operation, which developed the latest speech app.
In demonstrations at TechFest, Soong translated the recorded words of Microsoft honcho Craig Mundie into Mandarin without losing the recognizable qualities of Mundie's voice.
He also translated Microsoft Research boss Rick Rashid from English into Spanish, Italian and Mandarin.
The ability to handle many languages on many devices for spoken words, digital content and text on paper, signs or other physical media is part of an overall effort at Microsoft Research to mix physical and virtual worlds to make each simpler.
Other demonstrations showed the potential benefit of controlling a PC using arm gestures picked up by the web cam/motion-capture capability the Xbox Kinect uses for gaming, for example.
Using a concept called "sensor fusion" Microsoft Researchers demonstrated the ability to combine location data from a mobile phone and a Kinect web cam to allow computers attached to the Kinect to apply graphics to the actual location of the phone – to make it a ping-pong paddle or presentation pointer without the need to use a specialized devices.
Communicating in multiple media and several languages with both other humans and machines is a "tectonic shift in our relationship with technology," according to a Microsoft Research blog entry.
Augmented reality and controls that cross both real and virtual worlds are technically difficult, expensive to develop and only theoretically profitable in the future of whatever vendor builds a really effective one.
Microsoft, always a fast-follower rather than an innovator of new technology, has consistently poured far more time and money into technology that can do more than wreck a nice beach. Its work has helped develop a few decent product features and raise the average technical capability for both itself and its competitors by making abstruse technology common in Windows.
It has rarely hit a financial or technological home run despite success in multi-touch computer screens, high-performing flight simulators and other demanding technologies.