At this late date, most of us know that viruses and other kinds of malicious programs can hide in e-mail attachments, on USB thumb drives and even behind an innocuous looking link you Twitter or Facebook. We know enough (usually) not to just open everything people send us, or link to.
But what about all the hardware and software we buy and just assume to be reliable? Do you know that your Dell laptop, your Mac Powerbook or that new Cisco router for your company didn't come with malicious software already loaded? Could there be an extra hardware component dedicated to spying on you or your colleagues? Assuming that the device did come "certified pre-owned," as they say, how would you ever know?
If you winced just thinking about that, then you have some appreciation of the complexity of what is often called "supply chain security:" the herculean task of verifying the authenticity and integrity of computer hardware and software. It's an issue that's been lurking on the periphery of the national discussion about cyber security -- too thorny and complex to invite many takers among tech firms or policy makers. But the onslaught of sophisticated cyber espionage against the U.S. and its allies has roused both lawmakers and private sector firms to tackle supply chain security.
Microsoft was among those going public with its concerns over supply chain security. The company helped break up a global botnet known as "Nitol," in part by uncovering efforts by cybercriminals to infiltrate its supply chain in China, planting malicious software on computers during the manufacturing process. The company has since released a number of documents and position papers on threats to global supply chains.
But with more attention to the issue in the media, the question falls to individual companies and organizations (to paraphrase CapitalOne): "What's in your router?" In other words: 'what efforts are you making to verify the integrity of the technology products you buy?' And, if the answer is "nothing," is that a risk your company can continue to take?
To get to the bottom of the issue, ITworld reached out to security experts in the area of supply chain management to get their thoughts on how organizations can begin to understand supply chain security risk, and take steps to address it.
An old problem revisited
Supply chain security isn't a new problem. In fact, sabotage directed at supply lines is as old as warfare itself. As long ago as the Middle Ages, adversaries locked in warfare would look for ways to compromise the munitions that might be used against them - lessening casualties and also instilling fear and uncertainty in their enemy. And the practice is still in use today - with reports out of war-torn Syria that forces loyalty to the government there have been distributing batches of balky ammo to opposition fighters.
With technology, however, sabotage, tampering and other impurities can lurk deep within hardware, firmware or software applications, making it difficult to detect. In one of the most celebrated acts of supply chain tampering, the U.S. Central Intelligence Agency is alleged to have conspired with a Canadian company that made industrial control software to plant a Trojan horse in applications that were known to be the target of Soviet espionage. After the KGB stole the software for use on the USSR's Trans Siberian Pipeline, the faulty software caused a pipeline explosion believed to be the equivalent of three kilotons of TNT.
That may have been a victory for the West in the waning, pre-Internet years of the Cold War, but recent events hit closer to home. In 2009, the Stuxnet worm was loosed on Iran's nuclear enrichment facility at Natanz. The malware - widely believed to have been a creation of the U.S. and its allies - nonetheless proved that it was possible to combine more traditional viruses and worms with specialized attacks on SCADA and ICS products to cripple critical infrastructure. Stuxnet's code - like the doomed pipeline software stolen by the Soviets - manipulated programmable logic controllers (PLCs) that ran Iran's uranium centrifuges, instructing them to spin to destruction, all the while reporting normal operation back to Iranian scientists. Although Siemens, the German firm that made the PLCs used at Natanz, has always denied cooperating with the U.S. and its allies on Stuxnet, the Iranian government believes the company played a role in the attack, as do other informed observers.
Stuxnet may have been "ours," but the next 'Stuxnet' may not. And it's almost certain that Stuxnets, in some form, are coming our way. In April, 2012, for example, the U.S. Department of Homeland Security's Industrial Control System CERT (ICS-CERT) warned natural gas pipeline firms (PDF) in the U.S. about a campaign of targeted, "spear phishing" attacks that used malicious email attachments and web sites to try to get a foothold on the firm's' IT infrastructure. That's just one of a number of warnings related to what ICS-CERT believes is a widespread campaign (PDF) against critical infrastructure in the U.S.
Paul Nicholas, the head of Microsoft's Global Security Strategy and Diplomacy Team, said that supply chain security started to take on new importance within the last five years. "People started to realize that their information and communications technology supply chain was global," he said. "Governments started to worry about that (and) wonder 'I am reliant on this IT. How well do I understand the risks?"
On Capitol Hill, those questions found predictable expression. The House Intelligence Committee, for example, conducted hearings this year into allegations that telecommunications vendors ZTE and Huawei were conspiring with the Chinese military to steal intellectual property from U.S. and Western firms. Despite any hard evidence to support those claims, the Committee issued a report in October calling the firms evasive and untrustworthy, and warning U.S. firms to beware.
The Senate has acted, also, in a more nuanced way: adding language to the National Defense Authorization (NDA) act (S. 3254) (PDF) calling for "improvements of security, quality and competition in computer software procured by the Department of Defense." Part of that is for the DOD to require government software development and maintenance organizations and contractors to create a secure software coding plan that includes verifiable processes and practices and to comply with approved secure coding standards issued by the DOD, submit to inspection and appraisals. A version of that bill passed the Senate 98-0 in December.
Knock-offs, backdoors and other headaches
That's all well and good, but what does it mean for businesses and other organizations worried less about geopolitical scorekeeping, and more about the integrity of new equipment they buy - and the hundreds or thousands of devices that have already been deployed on their networks?
Nicholas said that companies lack even a common language to talk about supply chain risks and threats. Some organizations follow a similar tack as the House of Representatives: focusing on where software came from. Others are concerned about control and the integrity of the technology they are receiving. Still others focus on lifecycle management: sourcing, developing and deploying hardware and software in a safe way, he said.
The truth is that organizations must hit on each of those.
"The supply chain challenge is broad and very complex," said Andrew Howard, a Research Scientist at Georgia Tech's Research Institute. "You almost have to work backwards, starting with where you purchase device, to the manufacture of device to the supplier who provided them with materials, to the configuration of device," he said.
Cyber espionage aside, many companies are concerned that hardware devices they buy online are authentic, rather than an inferior knock off. And, even with authentic devices, organizations are concerned about undocumented back doors or security vulnerabilities that could compromise their security. "They want to know 'if I configure this as the manufacturer has specified, are there any vulnerabilities that I can detect?"
That's a good investment of time and energy, says Chris Wysopal, the co-founder and Chief Technology Officer at Veracode. "There is this Spy vs. Spy aspect, where people are looking for back doors in hardware," he said. "That's technically possible, but it's very rare - especially when compared with the prevalence of security vulnerabilities in software and firmware.
In fact, an in-depth security audit of Huawei gear by the German security researcher Felix "FX" Lindner, presented last month at the Hack in the Box security conference in Kuala Lampur, didn't reveal any stealthy back doors for the PLA. But Lindner did find plenty of critical and remotely exploitable software holes (PDF). These pose a much higher risk considering that "everyone in the world" can exploit the vulnerabilities, not just the manufacturer of the device, Wysopal said.
Which isn't to say that backdoors aren't common - they are. But they're not (just) in gear from China. Reputable vendors such as Siemens and others have been caught outfitting their remotely deployed products with administrative back doors that make it easier for the company's engineers to support customer devices in the field, Wysopal notes.
And the story isn't much better when you look at the software applications that companies buy - or develop - to run on that hardware. Veracode, which does application security testing, finds that around 8 in 10 applications sent to it for testing fail on their first submission.
"To me, that says that 20% of those applications are tested before they come to Veracode," Wysopal said. "In other words, most of the (applications) haven't had any security review at all."
There are many reasons for that - from pressure on development teams to hit delivery dates, to the modular nature of most modern application development, which has lead to a heavy reliance on open source and third party software components - many of which are, themselves, vulnerable, Wysopal said.
"Developers just ignore the code that they didn't write. Their attitude is 'someone else took care of the problem, so I don't have to," he said.
Simple steps to a secure supply chain
What's a security-conscious organization to do?
Nicholas of Microsoft said that organizations should take a risk-based approach to supply chain security and not make broad-brush decisions based on crude metrics like geography. Above all else, companies should look for transparency from their software and hardware suppliers.
"It comes down to having a good set of controls," he said. "Some of it is buying from trusted vendors and resellers. Some of it is understanding the processes that go into the products and services you're building into your procurement process," he said. "You need to build the basic foundation that will allow you to understand legitimate vendors and products, then build internal control practices that can identify when something is wrong," he said.
In 2009, Microsoft teamed with other tech giants including EMC, Juniper Networks, SAP, Symantec and Nokia to form SAFECode, a group focused on software assurance. That group has published guidance on identifying and responding to risks in the supply chain. Other for-profit, non-profit and government groups have also weighed in with advice on securing supply chains. The U.S. National Institute for Standards and Technology (NIST) published draft guidance in March (PDF) for federal agencies interested in security supply chains. In the EU, the European Network and Information Security Agency (ENISA) has also published guidance calling for more secure trust models, better vetting of hardware and software and improved technology for detecting malicious or fraudulent software.
No surprise: in many cases there are only dotted lines (at best) connecting recommendations to actual practices at this point. That's one reason that the DOD used its research and development arm, DARPA, to launch a research program dubbed VET, to help determine what constitutes clean and verified versus "malicious" software and how can organizations quickly assess the state of thousands or tens of thousands of IP-enabled devices in their IT environment.
Despite the mind boggling complexity of verifying supply chain security, Howard of Georgia Tech said that, on the practical level, much of it still comes down to relationships. "Know who you're buying from," he said. "You're not going to solve the problem, but you can mitigate your risk by buying from trusted suppliers," he said.
After that, there is plenty of low hanging fruit to be picked: assessments of device security and software security that involve basic device scanning and manipulation.
And, as with so many other topics: the Internet is a great source of information on products. "Information is power," Howard said. Customer forums and other online hangouts can provide information on the products you use. Customers can easily do firmware checks to make sure that the firmware running on their device is up to date and matches what the manufacturer is distributing. Beyond that, the sky is the limit. "Open your router and get the manufacturer and model number for the chips, then Google it to see where they came from," Howard suggested. "It's all about risk mitigation and, with the right technical staff, it's easy to do."