How did Amazon have a cloud service outage that was caused by generator failure?


I know that "stuff happens", but when I read Amazon's explanation of why it had service failures during the recent power outages, I was left shaking my head. Apparently, when power went out at their data center and they tried to shift to backup power from diesel generators, they "failed to provide stable voltage as they were brought into service". This seems to me like the sort of thing that you test and make darn certain that everything works. Just checking to make sure that your generators start up is insufficient, in my opinion you have to regularly cycle and test your generators to avoid problems like this. Am I being too hard on Amazon here, or did they really drop the ball on this?

Answer this Question


2 total
Vote Up (10)

I agree, this is not unfamiliarity with some new or cutting edge tech/practices that caused the problem, it was a data center management/design problem.  A data center should have its backup system regularly load tested, including making certain that multiple generators are ably to synch outputs which sounds like may have been the problem.  If Amazon wasn't doing this, it was pretty slack on their part.  

Vote Up (10)

No, you aren't being too hard on them. Your expectations sound quite reasonable. The entire situation underscores the dangers of being too reliant on the cloud. Users have no way of really knowing what's going on at the other end of their connection.

Perhaps it might be time to find another cloud provider? Amazon might have proven itself too unreliable or unprofessional in their practices.

Ask a question

Join Now or Sign In to ask a question.
A year ago VMware laid out an ambitious plan, now it's time to hear the details.
Ryan Carmack, the 9 year-old son of the famed programmer and game designer, has released Pong-clone called Angry Face
It's not surprising that former Microsoft CEO Steve Ballmer abruptly gave up his board seat some six months after leaving the top job, and the move should help cement the regime and strategy of his successor Satya Nadella, according to several industry observers.
Venture capital fundraising has picked up steam in the U.S., with cloud computing, mobile technology and robotics getting solid backing.
A new study reveals that Java developers make the most while JavaScript programmers are the most wanted
Microsoft's Azure cloud computing platform, wobbly for more than a week, is again experiencing outages and interruptions that are impacting multiple products in the U.S. and abroad.
Microsoft's Azure cloud computing platform suffered a series of outages and service disruptions in the past week that affected several products and impacted customers in various parts of the world.
Mark Zuckerberg’s latest app takes onerous Terms of Service to a strange new level
The state of Wyoming is planning to discontinue most of its data center operations and move its physical equipment to commercial co-location facilities.
Cisco Systems will cut as many as 6,000 jobs over the next 12 months, saying it needs to shift resources to growing businesses such as cloud, software and security.
Join us: