EMC Greenplum tackles big data with Hadoop distribution

By Thor Olavsrud, CIO |  Big Data, EMC Greenplum, Hadoop

In a further sign that Big Data is beginning to hit its stride, EMC Greenplum on Monday debuted a new Apache Hadoop distribution that it hopes will soon become the de facto standard, pushing aside current leaders like Cloudera and Hortonworks.

"The bet that we're making at Greenplum is that the future of our Big Data business is Hadoop, so it's important that we have our own distribution,"

Josh Klahr

vice president of product management

EMC Greenplum.

"Virtually every customer and prospect we talk to is doing something with Hadoop," says Josh Klahr, vice president of product management at EMC Greenplum. "It ranges from CIOs that have heard of Hadoop and spun up a Big Data team to figure out what to do with it to folks that are being a little more thoughtful about it and have figured out a use case."

"We're seeing rapid adoption," he adds. "The Hadoop business is growing 60 to 70% a year. We see this as a real sea change."

Since its emergence as an Apache Lucene subproject in 2006, Apache Hadoop has quickly become the preferred solution for Big Data applications with massive repositories of unstructured data. Hadoop has many things to recommend it: It's flexible, scalable, built on commodity hardware and fault-tolerant.

However, there are hurdles to implementing Hadoop in the enterprise. One critical hurdle is the lack of useful interfaces and high-level tooling for Business Intelligence and datamining. That's where Greenplum sees its opportunity, Klahr says.

Greenplum Bets Hadoop Is the Future of Big Data

"The bet that we're making at Greenplum is that the future of our Big Data business is Hadoop, so it's important that we have our own distribution," Klahr explains. "There are some things we're adding into the distribution that aren't yet supported by Apache. We think that the Hadoop market is going to be so big that we wanted to have our own distribution."

Some elements of EMC Greenplum's distribution, Pivotal HD, may never find their way back into the Apache project. The core of Pivotal HD is the marriage of Greenplum's massively parallel processing (MPP) database technology with the Apache Hadoop framework, a technology called HAWQ. HAWQ is essentially a fully functional, high-performance relational database that runs in Hadoop and which speaks SQL natively.

"Our plan is to actively contribute certain elements back to Apache, but we're going to have HAWQ as a proprietary service that we won't be open sourcing," Klahr says.

Klahr says that HAWQ delivers performance improvements of 50X to 500X when compared with existing SQL-like services (like Hive) on top of Hadoop.

HAWQ Connects Data Workers, Data Tools to Hadoop Repositories


Originally published on CIO |  Click here to read the original story.
Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Answers - Powered by ITworld

Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Ask a Question
randomness