July 29, 2011, 6:10 AM — The growing need for companies to manage surging volumes of structured and unstructured data is continuing to propel enterprise use of open source Apache Hadoop software.
However, instead of replacing existing technologies, Hadoop appears to be finding more of a place working alongside conventional relational database management system platforms, according to a new report from Ventana Research.
Hadoop is designed to help companies manage and process petabytes of data. Much of the technology's appeal lies in its ability to break up very large data sets into smaller data blocks that are then distributed across a cluster of commodity hardware for faster processing.
Early users of the technology including Facebook , Amazon, eBay and Yahoo have been using Hadoop to store and analyze petabytes worth of unstructured data not easily handled by conventional RDBM systems.
Ventana's report, which is based on a survey of more than 160 companies, shows that a growing number of enterprises have begun harnessing Hadoop for the same kind of reasons.
More than half of all enterprises looking to glean business insights by analyzing very large volumes of structured and unstructured data are said to have begun using Hadoop to help them with the task.
Most are using Hadoop to add new capabilities rather than to replace existing technologies, said David Menninger, author of the Ventana report.
Ventana's research shows that a majority of companies that are using Hadoop are using it mainly to collect and analyze huge volumes of unstructured and machine-generated data such as log and event data, search engine data, and text and multimedia data from social media sites.
The technology is much less likely to be used for analyzing conventional structured data such as transaction data, customer data and call records data, where traditional RDBS tools still appear to have the edge, Menninger said.
People are using Hadoop because it enables new kinds of data analytics capabilities, he said. "In two-thirds of the cases we found that people are using Hadoop for advanced analytics, and for types of analysis that they were not doing before."
Hadoop appears to be delivering new capabilities especially on the operations side of the business, Menninger said. In most cases, Hadoop is being used by business units such as sales and marketing rather than by groups such as human resources and finance, he said.