Posted February 17, 2011 - 10:51 pm
If you've been following the Jeopardy-IBM Watson faceoff this week, then you have witnessed a breakthrough in analytics and in new architectures for mining and analyzing diverse types of information in a single application. Watson and its successors may usher in a new approach to computing, combining as it does, so many disparate techniques to create a "thinking" machine. IBM has combined deep NLP with machine learning, a voting algorithm, a method of interpreting the questions and assessing them by formulating parallel hypotheses, and Hadoop and UIMA for preprocessing, as well as the usual search, fuzzy matching software and of course an in-memory caching system to save time in retrieval. To me, the strength of this system is the combination of all of these, and it is remarkable in that it doesn't rely on just one. In Watson, the whole is greater than the sum of its parts.