February 12, 2007, 8:16 AM — Xerox Corp. research subsidiary the Palo Alto Research Center has struck a licensing deal with a high-profile startup in the hopes of building a search engine that could one day rival Google Inc.
Powerset Inc. in San Francisco is developing a search engine based on natural language processing with the help of PARC, which has been working on technology in this area for 30 years, said Powerset founder and CEO Barney Pell. The search engine is expected to go live by the end of the year.
Powerset, which has raised US$12.5 million in funding from various venture capital firms and angel investors, has been negotiating with PARC to use the technology the research firm developed since September 2005, a mere month after Powerset was launched and a month before the company was incorporated in October, Pell said.
The startup even managed to win over top talent from PARC to join its team. Ron Kaplan, who led the PARC team that developed the natural language processing technology Powerset is licensing, is joining the company as its chief technology and scientific officer.
In addition to the licenses, Powerset also holds the patents to the technology, Pell said. In return, PARC receives equity in Powerset and royalties on company revenue. Powerset also is funding the natural language processing research team's efforts at PARC.
Pell described the difference between how a search engine powered by natural language processing technology and search engines available from Google Inc., Yahoo Inc. and others that depend on keywords work. He said the way many of the top search engines today index Web content is in keywords, but they don't have any idea what those words mean or how they relate to each other.
A search engine based on natural language, however, can accept queries written as people normally speak -- such as, "What company did IBM acquire in 1996?" Pell said. The results of the search should directly answer that question without giving a Web user every reference to the words "acquire," "IBM" and "1996" that have been indexed.
It's true the major Web search engines such as Google do question-and-answer type searches today, Pell said, but they are still mainly based on keywords.
Of course, researchers have been working for three decades to come up with successful natural language processing technology, and it has been no easy task, something that Pell himself acknowledges.
"Enabling computers to extract meaning and relationships in text ... is an incredibly hard problem," he said.
That said, to assume Powerset's search engine will work without a hitch is not necessarily a safe bet. However, Pell said that there have been recent breakthroughs at PARC in this area, and the software that Powerset has licensed should provide some of the highest-quality natural language processing-based search available.
Powerset is not the only company attempting to perfect natural language processing-based Web search. Hakia Inc. also is developing a search engine based on natural language processing. A beta of that engine can be found here. The Brainboost search engine, which is now a part of Answers.com, also is based on natural language processing.
