For years these cloud warehouses were not held back by NAND, SSD, or HHD hardware issues. The hard part was the scalability and manageability of crushing amounts of data. Traditional hierarchic files systems were never intended to store hundreds of millions and billions of files in a single namespace. This old network file system (NFS) architecture forced data to be stored in multiple file systems, sometimes in multiple locations, and in complex directory structures.
Enter object storage, a relatively new way of storing data that organizes it into objects. Each object is made up of bytes, but also includes metadata describing the object. This allows data to be stored more efficiently and organized by metadata for easier retrieval.
Next up, the Storage Networking Industry Association is proposing a new standard called Cloud Data Management Interface. CDMI is a set of protocols defining how companies can safely move data between private and public clouds. The standard is open, allowing for wide adoption and the opportunity to associate more defined rules around object storage data, such as how long data should be retained, how many copies should be kept, and whether those copies need to be distributed geographically.
Enterprise Strategy Group's McClure says CDMI is a driving factor for making business more comfortable about moving data from backroom servers to the cloud. Twenty-two percent of the companies it surveyed that already store data in the cloud said that in the next three years they would move 90 to 100% of their data to the cloud.
While it's a given that storage will continue to drop in terms of price per gigabyte, how fast and how far are always the questions. It's impossible to factor in events such as the 2011 floods in Thailand that halted some manufacturing and significantly raised hard-drive prices. However, as bad as the floods were for the people of Thailand, they had a sobering effect on the hard-drive industry--a time-out as it were. Hard-drive prices have again dropped, but not quite back to pre-flood levels, and they are not in the free fall that they had been in.
According to Gartner analysts John Monroe and Joseph Unsworth, there is "a widespread and ongoing readiness at the executive level to curtail production at any moment..." and "the HDD sector intends to closely and continuously align production with evolving demand..." After the NAND price free fall in the first two quarters of 2012, a similar resolve seems to be developing in that industry. That, however, can change in any competitive market at the drop of a hat.
According to Forward Insights analyst Greg Wong, "Steady 20 to 25% yearly price drops over the next few years" for NAND should be the norm. Substantiating this is Gartner's prediction of a drop from the current 85 cents per gigabyte to around 25 cents per gigabyte by 2016 for NAND. The Gartner numbers also predict a drop from the current 6 cents per GB for 3.5-inch desktop hard drives, to a mere 0.8 cents in 2016. Also, 2.5-inch mobile hard drives should drop from their current 11.5 cents per gigabyte to 3.5 cents per gigabyte.
While mainstream storage technology development is slowing in terms of density and performance, it's important to remember just how far we've come. In his 2006 presentation, "Fifty Years in Hard Drives and the Exciting Road Ahead," Seagate's Mark Kryder pointed out that if automobiles had progressed as far as hard drives in the time period from 1979 to 2006, each vehicle would carry 150,000 people, cost $15, and travel nearly a million miles an hour at 36,000 mpg. That's a heck of an improvement, and it will continue for a decade at least with the present basic technologies. After that, who knows what we wily humans have up our sleeves?
(PCWorld's Melissa Perenson & Tom Spring contributed to this report.)
This story, "Wanted: 40 trillion gigabytes of open storage, stat!" was originally published by PCWorld.