Hurdles for Hadoop

By Jesse Davis, director, research and development, Progress DataDirect, Network World |  Big Data, Hadoop

In another example, Chevron is utilizing big data analytics to get more barrels of oil out of its drilling productions. Sensor technology has advanced a great deal enabling exploration ships are capable of getting much higher resolution scans of the ocean floor.

The newest opportunity for big data in big oil is compiling and reducing the scan data from all the different ships to get a more complete view of the best places to start drilling. Data scientists are able to use systems like Hadoop to store and reduce this data quickly, which means that Chevron and other oil companies using this technique can get more barrels of oil into the market sooner than was previously possible. In addition to finding new oil sites, Chevron is also analyzing data taken from existing oil platforms to get the most effective and efficient production, saving millions of dollars in operating costs.

Developing trends

As we forge deeper into the world of Big Data, the trends that develop from big data use also impact Hadoop. These include the ability to handle a huge amount of information from a mixed bag of data sources, juggle multiple features in one space and consolidate them into something manageable.

As previously mentioned, Hadoop allows companies to capture all available data and save it to answer questions that may surface in the future. When these questions are identified, companies can then reduce existing data to both ask these questions and answer them effectively. The big players in the space are pushing an approach in which you can take your reduced data, move it to the data warehouse and ask questions there. This will allow you to utilize your existing data warehouse and store all of your refined data from your Hadoop systems in one central place, easily accessible to members of your organization. Given that, it's likely that we'll see trends around pairing Hadoop with existing data warehousing and analytics infrastructure in this pipeline format.

But the future of Hadoop extends beyond the data warehouse. The three V's of big data -- volume, velocity and variety -- are last year's problems. the three V's are all issues that we have been working to solve, but they are only the starting place for big data, not the end.

As technology evolves, we will continue to see rapid adoption, and pairing of disparate data will give us conclusions we never thought possible. This will pave the way for the three I's of big data: intelligence, insight and innovation.

Originally published on Network World |  Click here to read the original story.
Join us:






Answers - Powered by ITworld

ITworld Answers helps you solve problems and share expertise. Ask a question or take a crack at answering the new questions below.

Ask a Question