August 19, 2013, 7:00 AM — You spend thousands or even hundreds of thousands of dollars to secure the data stored on the critical databases and application servers your organization relies on. But what if each of those systems secretly harbored a powerful, hardware based back door that would give a remote attacker total control of the system? And what if that backdoor wasn't planted by some shadowy hacker group operating out of the former Soviet republics, but by the multi-billion dollar Western company that sold you the server in the first place?
If that sounds fantastic, I've got one word...err...acronym for you: IPMI, and its turning into the new four letter word in security. IPMI stands for Intelligent Platform Management Interface. It's a powerful protocol that is supported by many late model server hardware from major manufacturers like Dell, HP, Oracle and Lenovo.
At the 100,000-foot level, IPMI can be understood as technology that gives administrators almost total control over remotely deployed servers. IPMI and now-standard hardware called a Baseboard Management Controller (BMC) - let remote administrators monitor the health of servers, deploy (or remove) software, manage hardware peripherals like the keyboard and mouse, reboot the system and update software on it.
You'd think with that kind of power, IPMI would be a fortress: secure against remote hackers and malware based attacks. But you'd be wrong. Instead, researchers who have looked at implementations of IPMI have found just the contrary: that remotely exploitable vulnerabilities in IPMI implementations from major vendors are widespread, potentially giving a remote attacker total control over a vulnerable operating system. The most recent revelation about IPMI insecurity came last week in Washington D.C. at WOOT '13, the 7th annual USENIX Workshop on Offensive Technologies. (Get it? WOOT!) In a presentation there, Anthony Bonkoski, Russ Bielawski and J. Alex Halderman of the University of Michigan presented the findings of research on a common IPMI implementation from the server OEM Supermicro. They found that the IPMI firmware, developed by ATEN Technologies, contained "numerous, textbook security flaws" that included buffer overflow vulnerabilities, privilege escalation vulnerabilities and shell injection. They then demonstrated an attack leveraging one of those: a buffer overﬂow in a web interface used to access the IPMI feature to remotely obtain a root shell on the BMC.
The University of Michigan research is just the latest in a string of worrying reports on issues around IPMI. Notably, the security researcher Dan Farmer, working as part of a DARPA-funded research project, was among the first to sound the alarm on IPMI, in a paper first published in January. (The research was recently updated).
Farmer's analysis raised many of the same concerns as the University of Michigan study. In it, Farmer identified a wide range of security flaws in the firmware the runs the Baseboard Management Controller, which he described as "a bloodsucking leech" attached to the motherboard of servers that use IPMI.
In an e-mail, Farmer said the University of Michigan work confirmed what he suspected about the IPMI protocol and, more pointedly, the BMC component. BMCs were rife with exploitable vulnerabilities that had yet to be discovered or explored, Farmer said. "I talked about the appearance of really shoddy work on a visceral level in my own work - poorly written shell scripts, bad architecture, just terrible security design," he told me in an e-mail. "I suspect if they looked ...at other vendors there wouldn't be all that much difference. Each time I look at these things another piece falls off, it's amazing we've held it all together as long as we have."
Others have taken notice. HD Moore, the author of the Metasploit penetration testing tool and the Chief Research Officer at the security firm Rapid7, published a "Penetration Tester's Guide to IPMI and BMCs" in July that built on Farmer's research, highlighting some of the major vulnerabilities in IPMI and BMCs and providing tips to professional penetration testers about how to exploit them - taking advantage of default username and passwords that haven't been changed, or bypassing authentication or brute forcing usernames and passwords using known vulnerabilities.
Farmer's work and Moore's "guide" to breaking IPMI and BMCs prompted the Department of Homeland Security to issue an alert in late July about the security of systems that use IPMI. "Attackers can easily identify and access systems that run IPMI and are connected to the Internet," CERT warned. "It is important to restrict IPMI access to specific management IP addresses within an organization and preferably separated into a separate LAN segment."
In an e-mail, Moore told me that he has received numerous reports from professional penetration testers working in the field about successful exploits of systems using IPMI. "In almost all cases, they were able to use the information and code provided to gain access to an important target of their test," he wrote. That doesn't mean that IPMI and BMC hacks are being used outside of controlled tests (or "in the wild,") but Moore thinks it is likely that they will be eventually, if they haven't already. So what's a company to do? As is often the case, the level of risk from IPMI devices "depends" - in this the risk of attack due to IPMI depends on how an organization's servers are managed. "Companies using dedicated servers from public providers will be directly exposed to the most dangerous types of attacks," Moore said. Other firms, managing their own hardware, may yet leave IPMI enabled on internal servers, which can allow an intruder with internal network access to gain access to critical systems., Moore warned.
Farmer has published a list of security best practices to use with systems that support IPMI. They include "severely restricting" access to any BMC, beefing up authentication requirements and isolating systems with a BMC and supporting IPMI from being able to access the public Internet. (That would seem to be a no-brainer, but the University of Michigan researchers found more than 100,000 such servers that were reachable via public Internet searches and scans.)
Moore echoes that advice. "The best way to mitigate IPMI is to disable it or place the IPMI interface on a dedicated and physically isolated network," Moore wrote.