May 07, 2012, 11:50 AM — We tested the intrusion prevention capabilities of each of the next-generation firewalls to determine how well they work and how the IPS integrates with system management.
We were especially concerned with the IPS workflow for false positives, taking the network manager from a logged IPS event to the particular IPS signature triggering the event, to the ability to disable or modify the IPS signature to reduce problems.
We started by using our Mu Dynamics Studio Security test tools to check how well each firewall's IPS would catch Mu's list of published vulnerabilities. We tested the firewalls in two different configurations, one optimized to protect end users, and a second one optimized to protect servers. For each configuration, we sent a different set of about 1,000 vulnerabilities.
For each vulnerability set (server attacking and client attacking), we created two policies for each firewall. One policy included all of the IPS signatures and the other just had the subset of signatures marked as highest priority. We were thinking that the "all" signature set would have more false positives, and most network managers would want to only block the most critical vulnerabilities.
In most products, we saw less than two percentage points of difference between the two sets, meaning that there's very little tuning of the IPS possible. Fortinet's FortiGate was the exception, showing a 10% to 25% difference in attacks blocked, offering the network manager more tools to match the IPS to their network.
When protecting clients, we found that the Check Point Security Gateway, Fortinet FortiGate, and Barracuda NG firewall all outperformed SonicWall SonicOS. However, when we tested server-protecting IPS configurations, SonicWall and Fortinet performed significantly better than Check Point and Barracuda.
We believe that most enterprises deploying next generation firewall functionality will be doing it to protect end users rather than servers, so the client-protecting IPS coverage is more important than server-protecting coverage.