Disruptive change in the database world: Get ready

Databases are on the verge of an epochal shift

By Billy Bosworth, VP and general manager at Quest Software, Network World |  Data Center, database, Hadoop

There is a lot of debate over what to call these things, but I think NoSQL conveys the point that this is not a relational database, nor does it have things like the transactional consistency we take for granted in a relational model. (There's no reason that, in theory, they cannot use SQL, but the name caught on, so we're going to run with it.)  What they can do is scale out like nobody's business without the complexities of data modeling and schema management, which most developers just love.

MapReduce and Hadoop

NoSQL databases strive to meet the scale demands for large-scale Web applications.  However, they don't do much to address the business need for analyzing the resulting masses of data.  One solution is to go back to the future and create a brute-force way of scaling and processing.  Google's answer was MapReduce, which most of the world now knows in its open source form, Hadoop. 

Apache Hadoop can churn through petabytes of data across thousands of nodes running nothing more than commodity hardware. The reduction in storage costs alone can be astounding, but there are definitely tradeoffs.  Setting up Hadoop is not trivial, and your army of resources who know SQL will have to learn about things like HIVE and PIG (I know... it is hard to write that with a straight face) to write MapReduce jobs to handle your business needs.  While many identify these problems as a step backwards, it does appear that Hadoop is here to stay. 

Specialized Databases

These aren't new, but they are getting a lot of attention, and for good reason. Relational databases have become like the Borg, assimilating various technologies along the way to become all things to all applications.  They're now big and complicated.  Specialized database advocates argue that, rather than trying to solve every problem with one solution, it is better to build the right specific technology to solve a specific problem. Doing so, they say, dramatically decreases cost and increases performance.

The most common specialized database today focuses on solving the "data warehousing" problem, which is read-intensive requests going against massive amounts of data.  Another specialization is occurring around online transaction processing systems that require very high transaction rates on relatively small data sets, and we expect to see more and more breakthroughs as in-memory and solid state disk technologies change the economics of delivering high IO rates. 

Your Career Amidst the Chaos


Originally published on Network World |  Click here to read the original story.
Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Answers - Powered by ITworld

ITworld Answers helps you solve problems and share expertise. Ask a question or take a crack at answering the new questions below.

Join us:
Facebook

Twitter

Pinterest

Tumblr

LinkedIn

Google+

Ask a Question
randomness