November 11, 2010, 9:03 AM — Major market shifts in the database world don't happen often. When they do, they're massive, creating an impact that can last 10 to 20 years. When I entered the job market, it was right at the tail end of the last major shift from the mainframe to client/server. Our CIO announced his (aggressive) vision of the client/server future and I noticed three primary responses from my more seasoned colleagues. They either ridiculed it and ignored it; did a little nail biting, wondering what it all means; or got on board as fast as they could.
The new databases of today have the potential to be every bit as disruptive as the client/server wave, and the human reactions I saw nearly 20 years ago haven't changed all that much. These disruptors bring change, but while it sounds cliche, the truth is that with this change comes tremendous opportunity.
Let's try and classify some of these changes into a few big buckets so we can get a feel for how they could affect the data center as well as individuals from a career perspective.
For a whole lot of reasons that are beyond the scope of this article, modern applications are being asked to scale and perform at radically higher rates than ever before. When someone builds these modern applications, did you ever wonder what manages the data behind them?
Here's a hint: It's often not a relational database, and the reason is something called the CAP theorem.
The CAP theorem -- first articulated by Eric Brewer in 2000 -- essentially explains what the Rolling Stones knew long ago: "You can't always get what you want." CAP is an acronym for consistency, availability and partition tolerance (scale out). To drastically oversimplify, it says: "Pick any two, but you can't have all three."
Relational databases are fantastic at consistency and availability. But partition tolerance? Not so much. That's a problem, because what's often needed most in these modern applications is the ability to scale out across many database servers, even at the expense of something else. That something is transactional consistency. So what if your status doesn't immediately appear on all your friends' walls? Not a huge deal. But if that application is running slow because it can't scale -- that's a show stopper. This is where the NoSQL database comes into play.