January 10, 2011, 12:29 PM — Independent security research and testing firm NSS Labs today released its most recent Network Intrusion Prevention System (IPS) Comparative Group Test Report for the fourth quarter of 2010. The previous NSS Labs network IPS report was released in September 2009. In that study, NSS Labs found that security effectiveness ranged from a dismal 17.3% to a high of 89.5%.
Many of those failures a year ago resulted from the failure of network IPS vendors to stop techniques used by attackers to simply evade the defensive properties of IPS security gear, explains Rick Moy, president og NSS Labs.
Since that time, NSS Labs has found significant improvements:
- Security effectiveness, using the default factory-shipped settings, rose to 62%. But be careful: some default settings reached a mere 31% effectiveness.
- The improvement in security came with a price: performance of these devices decreased overall. One vendor, says Moy, reached only 3% of its advertised throughput.
- A number of multi-function gateways rose to comparable effectiveness as dedicated network IPS gear.
- Tuning is required, adding an average increase of 21% more protection.
Security equipment gear from Check Point, Endace, Fortinet, IBM, Juniper, McAfee M-8000, NSFOCUS, Palo Alto Networks, Sourcefire, and Stonesoft were tested.
HP TippingPoint refused to participate in the study, Moy says.
The products were tested using nearly 1,200 live exploits under what Moy describes as real-world conditions. Each device was tested using the default settings from the vendor, then once again more finely tuned by a representative of the respective vendor.
In the test using the manufacturer's default settings, McAfee's M-8000 came out on top, with 92% effectiveness, while the IBM GX6116 faired the worst at 31% effectiveness. Security effectiveness changed dramatically once devices were tuned. In those tests the Sourcefire 3D 4500 scored best, at 98%. And, according to the report, the Endace Core-100 came at the bottom at 43%.
NSS Labs charges $1,800 per user for the report, and has requested that full results not be published.