Storage Tip: When Is a Disaster Not a Disaster?
Send your Storage question to David Hill today!
See other Storage tips from David
What seems to be the problem? Okay, I confess. I chose the title because it was catchy. However, it does describe a very real problem that many organizations face. When do you consider an event a disaster? Why must you distinguish between problems that are disasters and those that are non-disasters, especially since a non-disaster could still be a serious problem?
As we shall see, enterprises must be able to make a clear distinction between types of downtime causing or threatening events for planning and provisioning purposes. The response to each type of situation is different. Understanding when a real disaster occurs is easy (a hurricane such as Katrina that renders a data center inoperable either for an extended period of time or permanently), but is a fire that destroys only a single disk array (albeit one with a key application on it) a disaster?
What do you need to know? The basic distinction is that a non-disaster problem is an operational recovery problem and a disaster is a disaster recovery (DR) problem. (Although it is not necessary, you may want to review the storage tip "Data Protection: Getting Back to First Principles." A disaster occurs when a downtime-threatening or causing event forces you to failover to a remote site, either your own or that of a third party, for an extended period of time. Every other event is an operational problem, i.e., a non-disaster.
What happens if I lose that key disk array at a local site (say to fire or two disk failures) and have to failover to the DR site? Isn't that a disaster? Not if the failure is isolated to a single array and not the whole data center. You could theoretically have placed the mirrored array at the local site. More importantly, your basic management team, policies, and procedures are still in place at the local site. Responsibility has not been transferred to the remote site except for running the mirror copy of the affected array as the production copy.
What are the tradeoffs? Knowing the difference between an operational problem and a disaster is essential as they are treated differently (see Table).
Table: Differences between Operational Recovery and Disaster Recovery
| Operational Recovery | Disaster Recovery | |
| Differentiation | Local failure that is targeted and not systemic | Requires failover to a remote DR site (where the remote site may or may not be run by a third party) |
| Problem Location | The focus is concentrated (typically one application only unless an array has multiple applications on it or a network, which affects all applications, is down) | Typically, all applications are affected due to the unavailability of the original production site |
| Basic mindset | Fire drill | Disaster relief |
| Management control mentality | "Emergency room" | "Control center" |
| Response Team | IT administrators assigned ad hoc based upon the nature of the problem, the skill sets thought to be necessary to resolve the problem, and personnel availability | Designated DR team that will probably consist of the remote IT staff, selected original production site IT staff (if possible), and business users/executives |
| Problem Resolution Strategy | Intensive focus to resolve the problem as quickly as possible to minimize impact upon service level objectives | Apply triage (restore most critical applications first); data preservation (restore operations correctly) may take precedence over data availability (restore operations quickly at the risk of the loss of some data) |
Source: Mesabi Group, July 2006
Essentially, to prevent and recover from operational problems, provisioning is local, such as a tape library and backup/restore software, to enable recovery from a problem with a particular disk array. Personnel to address operational problems are also local at the original production site. The opposite is true for DR. Personnel and equipment are placed at a remote internally-managed DR site on a permanent basis. (Using a third party site is different as you may or may not own/lease the equipment and your permanent employees are not always likely to be on site.)
So distinguishing between a disaster and a non-disaster is easy. Anything that requires you to transfer access to using storage at a remote site is a disaster. Anything else is an operational problem. Making that distinction is important because the resources -- both in terms of equipment and personnel -- that you bring to bear on resolving the problem are different. Don't use a hammer when a screwdriver will do and vice versa.