March 11, 2012, 8:45 AM — The Ultra Fast Broadband and Rural Broadband Initiative schemes could allow New Zealand companies to move, aggregate and analyse relatively large amounts of data and hence improve business efficiency and effectiveness through predictive modelling. But a shortage of analysis skills - and slowness in change of attitude - may limit the take-up of such technology.
This is the view of panellists from users, industry analysts and data analysts at an event hosted by sister publications CIO and Reseller News and sponsored by EMC in Wellington last week to discuss the use of 'big data' and the relevance of the emerging fibre networks.
Because New Zealand doesn't have operations the size of Google or WalMart, it's often assumed that talk of 'big data' is irrelevant to our market, says Ullrich Loeffler, IDC country manager in New Zealand. However, big data is "more of a concept" than an absolute size measure, he says. It implies treatment of data in new ways. A variety of unstructured, semi-structured and structured data in relatively high volume is brought together and analysed in real-time as it flows in, to derive commercial value from it.
The essence of big data, says Michael Whitehead, CEO of data analysis company Wherescape, is that data is the source for the organisation's operation and improvement, rather than a by-product of that operation.
Most New Zealand organisations have yet to grasp the value of fully analysing the data from their past and current operations to make predictions and govern their future operations, says EMC country manager Phil Patton. We're moving to the point where the technology is available so a corner dairy in New Zealand can mash together data on past buying patterns with a weather forecast for a hot day and predict how much extra ice-cream it should order, he says.
New Zealand, with its predominance of small companies is perhaps well-placed because collecting a wide range of data will not result in huge data repositories.
The truly big data of international social media networks can also be valuable and high-capacity communications networks can make it more feasible for this to be grabbed out of the cloud and analysed, to, for example, judge reaction to a new product line by positive and negative comments on Facebook and Twitter.
However, this kind of analysis demands a different set of skills, which belong in the mainstream of the business and are unlikely to be found in the IT department. The expert in the big-data style of manipulation is described as a "data miner" and a "data scientist". Patton describes such a person as "a statistician on steroids". They are rare and there will be a substantial learning curve for New Zealand to cultivate such skills, panellists said.