October 27, 2009, 10:15 AM — According to Gartner, by 2011, IT will allocate 75% of information management resources to integrate and analyze a blend of traditionally structured and diverse data types, whereas today 75% of resources are focused only on structured data1. The next generation of information access solutions will reach out to the entire enterprise, bring together all of the information and make it universally accessible.
The Status Quo
Today’s mainstay Business Intelligence (BI) tools are extremely good at tracking raw transactional numbers like sales figures and profit margins. What they fail to adequately address are the root causes, or drivers, of trends in those numbers. Moreover, they are typically able to tell what happened – but not explain why (unless it is evident in some other numeric data), let alone alert the business as a change emerges. Savvy business executives will gain advantage by moving beyond simply knowing what happened to preemptively understanding the forces acting on their business. What is causing a delay of payment from our largest customer? Why are sales in the southwest region down? How is user sentiment impacting our newest product?
Answering these types of questions with the average BI tool is challenging: at best, it takes a great deal of time to gain even one additional level of insight. The cost of these investigations is often high. Large numbers of IT staff must collaborate to extract, transform and load the data into a warehouse, update data dictionaries and then reconfigure the layers of OLAP, summarization, reporting and dashboarding. Despite these efforts and a slew of recent corporate acquisitions, many questions remain beyond the reach of such systems.
To provide greater value, information access tools must evolve in two ways. They must enable users to answer deeper, sometimes “fuzzier” questions about the enterprise. Then they must make it possible for general business users to easily obtain information.
Unified information Access (UIA) is an emerging class of information-centric infrastructure that helps you quickly access and analyze information from any source, including core applications and databases and external content from the web. It is optimized to work across both structured and unstructured data sources, enriching the information for better decision making.
Answering Deeper Business Questions
Deeper questions require more thought than usual. In the enterprise, more thought translates to more content. So the first challenge is to gain access to more information – including unstructured content like emails, documents and PDFs – breaking down digital silos throughout the enterprise and integrating the content together. The downside is that this is a challenging activity because mapping unstructured data into structured storage can be exhausting. The usual ETL tools do not apply; data warehouses and marts are designed to consume pre-integrated, de-normalized, structured data. UIA overcomes these challenges by inferring structure and relationships in unstructured content and using that to link it back, at query time, to structured data.
For example: an analyst staring at payment histories may never discover the reason behind an obvious anomaly. This is because he can’t seamlessly navigate to the relevant contracts and see variations in contract terms - highlighted in such a way that the root cause becomes obvious. (More on that later…)
UIA offers capabilities like entity extraction and fuzzy search, long staples of unstructured search engines, to put these types of discovery activities entirely within reach.
One of the most valuable repositories in the typical enterprise is electronic mail. How much of the knowledge in your company is contained in these files? Email is semi-structured, so it offers a variety of unique insights. Your CRM system can tell you that your sales team in the southwest region isn’t on track to achieve their goal this quarter, but can it tell you why? A combination of entity extraction, automated sentiment analysis and social network analysis might just turn up the problem account, internal resource or impossible customer requirement. The structured data leads the way, identifying the transactional problem, complemented by the unstructured data, which fills in the underlying cause.
In e-Commerce, social networks, and anywhere user content is generated, breaking the structured/unstructured silo can create real competitive advantage. UIA lets users correlate what users say (in text) with what they do (in transactions) to provide deep insight that can change average order value, average revenue per user (ARPU) and dramatically reduce customer defections, commonly referred to as churn.
Providing Widespread Access to Business Insight
BI requires users to query systems using one clearly defined question, plus an understanding of how the data model in use could possibly answer it. In today’s ultra-competitive business climate, where innovation is critical, the question is not always clear. UIA’s flexible query capabilities allow users to ask open-ended questions without understanding the data model. Instead of trying to hunt down disparate pieces of information, or rely on highly trained IT experts, the average business manager or analyst is presented with potential answers through the ability to drill through logical and associative patterns in the content to modify the set of results and their display.
Conclusion
Today, BI is accessible by the few and is supported by a legion of analysts and IT resources to make it work. This is the “BI pyramid”. Unified Information Access brings together structured and unstructured data and makes querying with the precision of SQL and the “fuzziness” of search a reality. UIA’s ability to help the user analyze and navigate disparate results will make information more accessible, inverting the pyramid to bring BI to the masses and creating a legion of knowledge workers, consuming information as they need it, supported by a few IT staff and commodity IT resources.
1. Gartner, Dec 2007, “Predictions 2008 – Information Infrastructure Will Master Information Chaos”
