Given that the primary goal behind any big data project today is a better understanding of your customers-how they interact with your company, its products and what they want going forward&mash;the skills of a Ph.D. statistician doing regression analysis is just a subset of the skills a full-blown data scientist will be expected to know, says Herain Oberoi, a director in the Business Platform Group at Microsoft.
"The title is definitely new. The data scientist role is not," Oberoi says. "It's part of a continuum. What's happened in the past few years is new technologies like Hadoop, that enables cheap distributed processing and improved capabilities and the ability to do things like statistical programming, [have] become easier, so the bar from getting insights from new types of data has come down."
This means specialists skills are no longer needed to glean specialist insights, at least in the discovery and modeling phases of finding the little nuggets of knowledge that lead to innovative products and services. Those nuggets exist in the massive data streams and data sets now open for examination, says Paul Barth, co-founder and managing partner of big data consultancy New Vantage Partners.
Analysis: Desperately Seeking Data Scientists
"It's going to be a lot different compared to today, where you throw your questions over a wall and wait six weeks for an answer and then have to say, 'No, that's not what I asked,'" Barth says.
Big data analysts, who are the forerunners of and most likely candidates for the data scientist title today, let companies ask and answer questions in quick succession, significantly shortening the mean time-to-answer and thus bringing the power of Moore's Law and analytics to the average business user.
"What kind of person does all this?" Thomas Davenport and D.J. Patil ask in their Harvard Business Review article. "What abilities make a data scientist successful? Think of him or her as a hybrid of data hacker, analyst, communicator and trusted adviser. The combination is extremely powerful-and rare."
Allen Bernard is a Columbus, Ohio, writer. He has covered IT management and the integration of technology into the enterprise since 2000. You can reach Bernard via email or follow him on Twitter @allen_bernard1. Follow everything from CIO.com on Twitter @CIOonline, on Facebook, and on Google +.
Read more about data management in CIO's Data Management Drilldown.