Microsoft Security Essentials struggles in antivirus tests

Microsoft's popular free antivirus program Security Essentials has put in a mediocre showing in the latest quarterly tests from German test outfit AV-Test.org, finishing second bottom out of 22 products.

In Q1 2011 Security Essentials 2.0 (MSE) performed well at the least demanding test, that of spotting malware drawn from the industry-agreed Wildlist selection, scoring 100%. It also put in a good performance against a large group of recent malware samples selected by AV-Test itself, with a creditable score of 97% detection.

However, the product's performance deteriorated sharply when pitted against 107 recent zero-day malware web and email malware attacks, described by AV-Test as 'real-world' testing', spotting only half. The product's performance in 'dynamic detection testing' - noticing malware on or post-execution - was also modest at only 45%.

For context, the test average for real-world and dynamic testing was 84% and 62% respectively.

The top-scoring product in the tests was BitDefender's Internet Security Suite 2011, with a maximum weighted score of 6.0 across all tests, ahead of BullGuard Internet Security 10, F-Secure Internet Security 2011, and Kaspersky Internet Security 2011, all on 5.5. MSE scored 2.5, ahead of only one product, CA Internet Security Suite 2011.

AV-Test also looked at the impact of antivirus software on the performance of the PC. By this measure, often rated as important for many consumer users, MSE did relatively well, scoring 162 (lower being better) against the average of 171. This test showed a surprising degree of performance difference between suites, with BitDefender against doing well with a score of 111 against BullGuard's dismal 539.

Security Essentials was in the end awarded a 'pass' certification under the AV-Test assessment for making the grade in at least 11 of the 18 tests, putting it ahead of five products that failed altogether. In addition to CA's suite, these were Norman Security Suite Pro 8.0, McAfee Total Protection 2011, PC Tools Internet Security 2011, and Comodo Internet Security Premium 5.0/5.3.

Do the zero-day tests matter in everyday conditions? Arguably, yes. A common attack method is to hit users with zero-day exploits and so the ability to spot this challenging category of malware is crucial. According to AV-Test's quarterly results, MSE's performance in this test has also deteriorated quarter-on-quarter, dropping from around 75% to Q1's 50%.

"Microsoft is offering a free of charge virus scanner: MSE. The product is missing effective email and web protection and also dynamic detection/protection technologies, so the product performs worse when compared with other free or paid AV/ISS offering," said Andreas Marx of AV-Test by email to Techworld.

"That's the big problem with this tool - the majority of the other products tested includes such protection features, so they are performing better in our tests. And we expect that they are performing better in the 'real world' as well, which is the focus of our tests."

An individual user's exposure to a zero-day attack will depend on a number of factors, including the range of applications used and how assiduously a PC is patched.

As Marx noted, MSE is a free product - many of the rival suites charge upwards of £20 ($33) a year for a license. However, the dividing line isn't necessarily whether a product is free or not; several rival products offered in free versions did better than MSE. It is possible that free programs now need to include a wider range of detection features than they might have done in the past.

Version 2.0 of MSE was launched in December 2010 and anecdotal evidence suggests it has only enhanced the program's huge popularity. By September 2010, the software was said by Microsoft to have been installed on 31 million PCs globally, including 1.7 million in the UK.

The most interesting message of these tests is that a product can drop in effectiveness quite quickly, before in all likelihood rising again as a new version appears that adds new security elements.