March 10, 2011, 1:59 PM — If cloud computing is becoming a commodity -- It is. That was the whole point. -- what will be the next big thing to hit IT?
Big Data. It's driving mergers, acquisitions, investments on Wall Street. IBM just launched a Boot Camp-style training class in what Big Data is and what to do with it.
Big Data, which came out of academics and high-performance computing as way to describe mondo data sets -- originally things like the multi-terabyte piles of stuff Ph.D.s used to analyze weather patterns, sky maps and the possibility that the next time they fired two invisible particles in opposite directions around a giant loop that the universe would end when they met.
For corporate IT it's not nearly as exciting. Mostly what people mean are giant piles of largely irrelevant data people who often claim to be concerned about their privacy force on the universe by posting it on social networks with no limits on who can see it.
Some of that is useful from a marketing point of view. Some of the useful Big Data comes from engineering, mining/geological or logistical databases that track vast numbers of events or data instances and match patterns among them.
Much of the rest, the parts growing most quickly and most wastefully, is marketing-related information scraped, bought or sifted from social networking sites, discussion forums and often-false, usually faulty customer-comment and customer-information forms sent in by people who might or might not be customers, but would like their questions answered, please.
IBM's Big Data business manager Rod Smith told InfoWorld it was another big step up in volume to "help people sift through data where maybe 90 percent of it is not very useful."