IT Horror Stories. Be Afraid. Be Very Afraid.

The visions of axe murderers, zombies, and giant spiders that Halloween conjures are nothing compared to the very real threats to your enterprise. Do you have any idea of the potential nightmares that lay dormant within your own organization? Here are two true tales of IT terror, guaranteed to keep you up at night. Read them if you dare.

Nightmare #1 -- When change becomes your enemy

On a cold, rainy night, a rogue IT administrator in his dimly lit cubicle made a simple decision that halted business for several weeks. This administrator, we'll call him Frank N. Stein, was the last person to approve a change and he made one horrific mistake. He added a single line to the approval that said, "And change the database from 3.2 to 3.3."

That simple statement to upgrade the database to a new version was all it took to unleash pure havoc. How did this happen? Was some malicious evil at work? No, he just innocently thought, "why not get two things done at once?" But the upgrade of the database itself hadn't been approved or tested. People started moving the new version of the database into production and since this hadn't been planned, the appropriate backups were not in place.

In this terrifying scenario, transactions appeared to be running okay and people were able to enter orders, but the database wasn't being updated correctly. The front office sold the goods, the Web staff took the orders, but when the back office tried to process the orders they saw only gibberish -- the names of the customers weren't showing up and order amounts were incorrect. To make matters worse, this occurred just before Christmas!

Fortunately, this company was able to survive the fiasco, but poor Frank was never heard from again.

How can such a hair-raising situation be avoided? Well, you could implement an incredibly draconian measure that disallows any changes unless the CIO and 16 members of the board sign off on the agreement. Of course, that would also bring your business to a screeching halt.

A better option would be to implement a closed-loop change management system that goes through every step of the approval process, checks it and verifies it. This automated system would have prevented unapproved additions to a change process from happening. It would have used a Definitive Software Library (DSL) to control what software is allowed to be implemented in production. The new database version then would have only been installed or upgraded if it had been through testing and approval and added to the "golden images" in the DSL.

Nightmare #2 -- Night of the living dread

Sign up for ITworld's Daily newsletter
Follow ITworld on Twitter @IT_world

• Email
• Print
• Share
  • Slashdot
  • Digg
  • Stumble
  • Reddit
  • Newsvine