Jeremiah Peschka, managing director at IT consultancy Brent Ozar PLF, says clients he advises too often overlook installing disaster recovery plans, much less testing them. "This seems like a really sane and safe way to see if you're protected that doesn't cost a ton," he says. Netflix says in the blog the service could likely be run using Amazon SimpleDB, a relational database, and is small enough that it could be run within AWS's free usage tier, which covers up to 25 SimpleDB machine hours and 1GB of storage.
For users that may be nervous about intentionally failing their systems, Peschka recommends running Chaos Monkey in a scaled testing area that mimics a production environment. Other ways to test DR systems, he says, are traditionally from manually built processes of shutting down servers, but Netflix releasing Chaos Monkey is the first DR testing system he's seen released in an open source fashion.
"One of the biggest stumbling blocks of the cloud is that it can go down," he says. "People need to test this stuff and if you're really concerned about having 24/7 uptime, there's probably even more you can do." For example, highly reliant systems typically use elastic load balancers, which are available from Amazon Web Services, to reconfigure virtual machines away from failed instances. Additional testing to fail an entire availability zone and perhaps even an entire region are being tested by Netflix, the company has hinted at in what it calls "Chaos Gorilla." Netflix says it hopes to open source other tools it uses as well in the future.
Read more about data center in Network World's Data Center section.