On the front line against the next Stuxnet

Greg Schaffer, Acting Deputy Under Secretary, DHS National Protection and Programs Directorate talks at the ICS-CERT Watch floor. This is one of two ops centers that handle security response for industrial control systems. The second is at the National Cybersecurity and Communications Integration Center in Washington DC. The DC center operates 24x7. Not so here at the Idaho Falls facility.

IDG News Service

Something has gone terribly wrong on the plant floor at ACME Specialty Chemical International Inc.

Liquid is overflowing from vats, the power keeps shutting off, and CEO Jeff Hahn has no idea what's going on. Behind him is a computer used to control the factory. Ominously, the cursor moves around on the screen as if it has developed a life of its own. "I have no control of my mouse," says the woman at the terminal.

It turns out that Jeff Hahn is the one to blame. Like many CEOs, he clicks on any interesting link he sees in his email inbox. This time, he clicked on a link sent by hackers working for a rival company, Barney Advanced Domestic Chemical Co.

Fortunately, ACME Chemical isn't real. It's part of a training exercise run by the U.S. Department of Homeland Security (DHS) and Idaho National Laboratory (INL). And Jeff Hahn isn't actually a CEO. He's a training lead at INL, playing his part in a cyberexercise that took place Friday at the lab's training facility in Idaho Falls, Idaho.

People who run industrial systems, like those at ACME Chemical, have traditionally cared about one thing above all others: They want their machines to run without interruption, and nothing -- not even an important security patch or operating system update -- can get in the way. These obscure systems are built by big companies such as Siemens, Honeywell, and Rockwell Automation, but they've kept a low profile.

Last year's Stuxnet worm changed everything, showing that these types of machines can be attacked, and even brought down with a cyberattack.

That's put the DHS-funded INL security programs in the spotlight, because they form the backbone of the government's plan to secure industrial systems. "In many ways, we are connecting equipment that has never been connected before to this global network, and as we do so, we have the potential for problems," said Greg Schaffer, acting deputy undersecretary with the DHS's National Protection and Programs Directorate, speaking at a briefing for reporters at INL. "They are kicking on the doors of these systems, and in some cases there have been intrusions."

There are about 75 people working on the INL programs, known collectively as the Control Systems Security Program. With an annual budget of just over US$25 million, they form the first line of defense against attacks on industrial systems.

Friday's exercise was put on for the benefit of the press. But every month about 40 engineers and computer security professionals are invited to test their skills at these day-long exercises, where members of a hacking group, known as the Red Team, try to break into a test network defended by the Blue Team.

According to Hahn, the good guys usually win, but not easily. The test networks are riddled with holes, none of which are known in advance to Blue Team members, and it's often a scramble to secure the systems before the Red Team maps out the network and disrupts the factory floor.

The control systems program one of the U.S. government's main weapons as it tries to beef up computer security in power plants, at chemical refineries and on factory floors. Companies that make the hardware and software for big industrial machines can come to INL for a hard-nosed security evaluation of their products. It's a good deal for vendors, as part of their testing costs are covered by taxpayers, and it's good for the lab, because its engineers get to learn about security problems that could flare up in the future.

Although INL has been doing this work quietly for close to a decade -- last year it assessed products from 75 vendors -- the publicity around Stuxnet has put it in the spotlight like never before.

The world dodged a bullet with Stuxnet. Although it spread across the globe, it left almost every system it infected operational. It was a cyber sniper-shot aimed at uranium-enriching centrifuges at Iran's Natanz nuclear reactor.

The possibility of a second industrial systems worm has many security experts worried, though. Stuxnet infected tens of thousands of systems, including many that contained Siemens programmable logic controllers. If it had been designed to mess up every Siemens system it infected, instead of damaging only the Natanz centrifuges, it could have caused widespread damage.

Now that Stuxnet has proved that these machines can be hit, another cyber attack on industrial systems is inevitable, according to Michael Assante, CEO of the National Board of Information Security Examiners, and a noted expert on industrial security issues. "It's a matter of time," he said.

But is the U.S. Department of Homeland Security's ICS-CERT (Industrial Control Systems) team, set up at INL to respond to this type of incident, ready for a serious problem? Critics say the DHS was slow to respond to the Stuxnet threat and parsimonious with the information it did share.

DHS officials at the training exercise defended their handling of Stuxnet, but the man in charge of ICS-CERT said there's room for improvement. "I think there's always going to be an evaluation of how much information do we release, when do we release it and how do we release it," said Marty Edwards, the ICS-CERT's director. "So as we continuously evaluate those, and Stuxnet was a very good case study of how we performed, we'll continue to fine-tune the processes to give industry the tools they need to defend these systems."

DHS intentionally released fewer details about the problem than vendors like Symantec, Edwards explained. "We still haven't released broadly the [Stuxnet] technical details, because I still believe that they're sensitive," he said. "You're not going to see us post those kind of details to a completely open, public website because we don't want to encourage the script kiddy or the copycat types."

Just a few blocks from the training facility that was home to Friday's exercise, INL operates a "watch floor" for industrial systems. This is the classified building where phones will start ringing should the next Stuxnet show up, and home to staffers who specialize in IT and industrial systems. It's small -- there were just four analysts there on Thursday -- but it looks like the security operations centers you see big companies such as Cisco and Symantec: people sitting in front of computers, with a big screen showing a real time feed of any situations that need to be handled. When Stuxnet first appeared in July 2010, this is where the U.S. response was mustered. The worm was quickly handed over to a special malware analysis lab, also run by INL in Idaho Falls, where it was dissected by security experts and industrial engineers.

Edwards' boss, Greg Schaffer, says the group "had an appropriate response to what was a complex and new set of circumstances that we had to deal with." And while he believes that the siphoning off of intellectual property is the largest cyber issue facing the U.S. right now, the doomsday possibilities of a well crafted attack on power plants or nuclear facilities makes the kind of work that goes on at Idaho National Labs important.

"This is an issue that is evolving and that could have significant impacts to us," he said. "This program is designed to get us in front of those problems."

Robert McMillan covers computer security and general technology breaking news for The IDG News Service. Follow Robert on Twitter at @bobmcmillan. Robert's e-mail address is robert_mcmillan@idg.com