Ultimately, however, predictive analytics is forcing a showdown between data-driven and intuition-based decision making, says Eric Siegel, president of the analytics training firm and conference organizer Prediction Impact Inc. "That's the big ideological battle. It's a religious debate."
Data: Getting to good enough
On the technology side, both building the model and preparing the data can be stumbling points. Predictive analytics is an art as well as a science, and it takes time and effort to build that first model and get the data right, says Abbott. "But once you build the first one, the next one is much less expensive to model" -- assuming you're using the same data. Analysts building a new, entirely different model that uses different data might find that project just as time consuming as the first. Nonetheless, he says, "The more experience one gains, the faster the process becomes."
Data preparation issues can quickly derail a project, says Siegel. "The software vendors skip that point because all of the data in the demo has already been put into the correct format. They don't get into it because it's the biggest obstacle on the technical side of project execution -- and it can't be automated. It's a programming job."
When the Magic's Perez got started in 2010 he grossly miscalculated the time it would take to prepare the data. "We didn't set the right expectations. All of us were thinking that it would be easier than it was," he says. Pulling together data from Ticketmaster, concession vendors and other business partners into a data warehouse took much longer than anticipated. "We went almost the entire season without a fully functional data warehouse. The biggest thing we learned was that this really requires patience," he says.
"Everyone is embarrassed about the quality of their data," says Elder, but waiting until all of the data quality issues are cleaned up is also a mistake. Usually, he says, the data that really matters is in pretty good shape. "I urge people to go ahead and make a salad and see what you can get," he says.
Blue Health Intelligence (BHI) had no issues with the patient health care data coming from its 39 Blue Cross Blue Shield affiliates -- but with seven years of data about 110 million members, they had a lot of it. "Health care is way behind in analytics because of the complexity of our data," says Swati Abbot, CEO. "People tend to run after the data and not know why they need it." Clinical insights must come first, she says. "Then the math takes over."
BHI developed models to predict which of its highest risk members were most likely to be hospitalized, who had an avoidable risk, who was most likely to respond to intervention and which actions were most likely to work in each case.