Hardware flaws hang some Cisco firewalls

Hardware flaws in some Cisco Systems Inc. firewalls for corporate central and branch offices have caused the systems to hang or shut themselves down and forced Cisco to replace the affected boxes.

Some Cisco Pix 515, 515-DC and 506 Firewalls have suffered system hangs when traffic on the network becomes too heavy, requiring IS staff to manually restart the firewall, Cisco reported in an Oct. 18 field notice on its Web site. Cisco expects the problem to occur most often in the 515 models, which are designed for corporate central offices, but said it may also happen in 506 units in some cases. The 506 is designed for branch offices, which tend to experience lower traffic levels.

The firewalls typically are installed between a company's internal network and the Internet to guard against intrusion. The flaws can cut off an Internet connection that runs through a firewall but will not cause a connection to become insecure, Cisco said on its Web site. Officials at the company weren't available to comment in detail about the problem.

While the failures don't pose a security issue, they could cause network availability headaches for a number of large corporations. Cisco holds about one quarter of the overall firewall market, according to Richard Stiennon, a Gartner Inc. analyst in Detroit. A serious hardware flaw in such a widely sold firewall device is probably unprecedented, Stiennon said.

Cisco has traced the source of the problem to a component that the networking giant began buying from a new supplier in May. The component's timing is slightly different from that on previous units, and the difference makes the system unstable, according to the field notice. Units made after Oct. 2 don't have the flaw.

Cisco is replacing the firewalls for registered customers, free of charge. However, because the replacement units need to come from the company's manufacturing facilities in California instead of stock in local service centers, service agreements for overnight replacement can't necessarily be met, especially outside the U.S.

The only workaround Cisco offers is to reduce the traffic load by hard-coding all the firewall's interfaces to 10M bps (bits per second), or making a change elsewhere in the network that reduces traffic to that level. The units most often hang when traffic exceeds 15M bps, though the threshold varies, according to Cisco. The devices are available with 10M-bps or 10/100M-bps interfaces.

Few enterprises are equipped to deal with a workaround that would throttle down a critical network connection so dramatically, Gartner's Stiennon said. On the bright side, only a small percentage have Internet connections of more than 10Mbps, he added.

Cisco also reported on Oct. 18 a flaw in the way power supplies are attached to motherboards in some Pix 506 Firewalls. Over time, friction and vibration can work the power connection loose, causing the firewall to freeze or reboot, according to the field notice. A cable tie-down was introduced on Oct. 2 that will keep the power supply attached.

Cisco is replacing the affected 506 units for registered customers, free of charge. As a workaround, Cisco provides instructions on its Web site for opening the firewall and reinserting the power connector in the motherboard.

The failures and possible long waits for replacements put the spotlight on one problem with integrated hardware-software "appliances" such as the Pix Firewalls, Stiennon said. If hardware problems befall a software firewall, such as one from Check Point Software Technologies Ltd., most users can solve them easily and quickly by replacing the Intel-based PC on which the software runs.

The Cisco service notes can be found at http://www.cisco.com/warp/public/770/52.html/.

Cisco, in San Jose, California, can be reached at +1-408-526-4000 or via the Web at http://www.cisco.com.