Deduplication: A Quick Fix or The Way Forward for Storage?

Having engulfed the IT team, the data avalanche is set to hit the wider business. Organizations now have to impose restrictions on email inboxes and local storage facilities because of the amount of data being produced by users across the network.

The problem may feel insurmountable; however a key way of reducing data is to examine how much information is simply replicated by multiple users. With standardized operating systems and applications, come thousands of identical files on legions of computers. Add to that identical attachments stored in multiple recipients’ inboxes and it’s easy to see how much duplicate documents add to an organization's storage requirements.

Unless organizations want to risk facing an all-engulfing data avalanche, the amount of data being stored has to be reduced, or managed more efficiently. Vendors have been quick to address this critical pain point; however it’s unclear as to whether these technologies have the capacity to cope with new developments in data such as bigger file sizes, multimedia formats and distributed data. Unless the situation is evaluated now, companies may find themselves left with a quick-fix solution that could quickly leave them in the same position as before.

Deduplication – A Lasting Solution?
Deduplication has quickly risen to the top of the IT agenda as a method to help reduce storage and power costs through streamlining the amount of information needing to be backed up. It also helps to address issues such as business continuity, e-discovery and compliance requests.

Deduplication technologies can take a myriad of forms, but there are several fundamental methodologies:

• Elimination of identical duplicate files across the network
• Incremental backups – finding the differences between today’s and yesterday’s files and only saving the changes
• File compression – further reducing the volume of data stored

These techniques are highly effective at stripping out a huge amount of backed up data that simply isn’t required. The technology can also work across distributed data centers, ensuring one centralized version of a document is backed up, rather than several different versions held on different devices.

However, deduplication really only tackles the initial symptoms of the data mountain and will not be able to match the growing average size of files as video and media files become increasingly popular. Compression is already implemented within these file formats, which will mean a reduction rate at the transmission stage. Effectively storage could get worse, rather than better.

Data Reduction – The Next Generation
Data reduction takes deduplication one step further, moving it from a reactive to a proactive approach to data management.

Sign up for ITworld's Daily newsletter
Follow ITworld on Twitter @IT_world

• Email
• Print
• Share
  • Slashdot
  • Digg
  • Stumble
  • Reddit
  • Newsvine