University flees fire with network in a box -- literally

When fire again threatened to char Santa Barbara, Calif., last month, the IT staff at Fielding Graduate University literally raced flames out of town with its network stashed in a cardboard box safely resting on the back seat of the getaway car.

While it might sound like a dramatic rescue, it was more the result of a disaster-recovery plan, a newly virtualized network and a quick acting IT staff.

And not wanting to settle just for an escape, the staff got operations back on line in a day at an off-site location where they kept the online university functioning through a nine-day ordeal.

"We joked that it was our network-in-a-box, but that is the power of virtualization and a blade environment," said Deby DeWeese, director of network services for the university.

For Fielding, the network is the university as students "attend" classes over the Internet. The school just months prior to the fire had completed consolidating 30 servers running its Windows network onto a virtualized network based on Microsoft's Hyper-V technology and four HP ProLiant BL460c blades that included an HP MSA 1500cs SAN. In all, the network and a data protection system housed 2.4 terabytes of data.

The network-in-a-box version was born May 5, the day the Jesusita Fire broke out.

DeWeese's boss – Dan Sewell, associate provost for research and chief learning officer – returned to the data center that night when the flames came closer to the city. Over the phone from her rural Santa Barbara County home, DeWeese walked Sewell through pulling the blades and unhooking the school's server and disk array that runs its Microsoft Data Protection Manager (DPM) system.

The next day they returned, plugged everything back in and started to work.

But the fire continued to rage and by that afternoon Fielding's operations center was a block from the fire's evacuation zone.
"We could see flames from our office windows and ash was falling from the sky," DeWeese said. "We couldn't keep doing network-in-a-box, we needed to get something up and running somewhere else."

DeWeese and Sewell again did the network-in-a-box drill, while HR made sure staff was safe.

And what happened next was a combination of some nimble negotiating and finding friends in the right places that resulted in the university returning online from a new location in about 25 hours.

Sign up for ITworld's Daily newsletter
Follow ITworld on Twitter @IT_world

• Email
• Print
• Share
  • Slashdot
  • Digg
  • Stumble
  • Reddit
  • Newsvine