Lab Test Results: Symantec, Kaspersky Lab, PC Tools, AVG, Detect The Most Zero-Day Attacks

Top Internet security suite products scored high when detecting zero-day attacks during a three-month period, according to new data released today from independent German lab AV-Test, with Symantec and Kaspersky Lab finding 98 and 97.5 percent, respectively.

AV-Test tested 10 zero-day threats during a three-month period on Windows XP SP3 machines running Symantec Norton Internet Security 2010, Kaspersky Internet Security 2010, PC Tools Internet Security 2010, AVG Internet Security 9.0, G Data Internet Security 2010, Panda Internet Security 2010, Avira Premium Security Suite 9.0, McAfee Internet Security 2010, CA Internet Security 2010, F-Secure Internet Security 2010, BitDefender Internet Security 2010, and Trend Micro Internet Security 2010.

PC Tools caught 95.8 percent of the threats, followed by AVG, 92.2 percent; G Data, 90 percent; Panda, 90 percent; Avira, 87.7 percent; McAfee, 87.2 percent; CA, 86.7 percent; F-Secure, 85.8 percent; BitDefender, 84.3 percent; and Trend Micro, 83.3 percent.

"The majority of the products are performing 97 to 99.9 percent in large on-demand scanner tests. The products are often tested against millions of old samples which have not been seen spreading or distributing during the past few months. However, when ... current, zero-day [samples] are used ... the products show very different results," says Andreas Marx, CEO of AV-Test. "These results reflect the product capabilities in a much better way, as they simulate what the user would see in a real-world infection scenario. The results differ a lot now, and no product scored 99.9 percent anymore."

But Rick Moy, president of NSS Labs, another independent test lab, says the recent AV-Test numbers are inflated. "There's no way AV products are catching 98 percent of attacks," he says. "This seems counter to the [results of the] real-world testing we do."

Moy says a more realistic rate of zero-day detection for an AV product would be 29 to 64 percent, which is the range his lab got in its recent tests of AV products. And vendors tell him off the record that they typically can catch about 40 to 45 percent of zero-day attacks, Moy says.

AV-Test emphasized that its testing project was independent and not sponsored by any AV company. The lab performed 600 malware and 400 clean-file tests for the 12 AV products during three months, with 14 full-time employees and up to 150 PCs and servers.

AV-Test also tested the 12 products for their ability to block the malware. PC Tools Internet Security 2010 scored the highest, blocking 94.8 percent, followed by Symantec Norton Internet Security Suite 2010, with 92.8 percent; Kaspersky Internet Security 2010, 89.8 percent; Panda Internet Security 2010, 88.7 percent; Avira Premium Security Suite 9.0, 87.2 percent; McAfee Internet Security 2010, 86.7 percent; AVG Internet Security 9.0, 84.2 percent; G Data Internet Security 2010, 83.0 percent; Trend Micro Internet Security 2010, 81.3 percent; F-Secure Internet Security 2010, 80.2 percent; BitDefender Internet Security 2010, 77.8 percent; and CA Internet Security 2010, 73.5 percent.

The lab also found the average growth rate of new malware is at around 8.8 percent, based on the malware it has collected during the past two years.

NSS Labs' Moy argues that focusing on zero-day attacks favors strong behavioral detection engines. "Focusing on just 10 zero-days does not reflect the current threat spectrum on the Internet," he says. "Many of the threats out there are older, such as SQLslammer, Koobface, and Conficker. This is why we test whatever we find, not just zero-day. And zero-day should be the hardest to detect of fall of these."

Marx, meanwhile, says AV-Test ran up-to-date versions of all of the products in its test. "The products were able to query the cloud during the testing. All samples were 100 percent working at the time we used them for reviewing the products," he says.

AV-Test's results demonstrate how even with the best tools, users are still at risk of infection, he says. "Therefore it's important that the user doesn't think that AV and ISS [Internet Security Suite] will prevent him from an infection at all times. He also needs to accept the fact that more needs to be done to protect the system in a better way. For example, security updates need to be applied, and the user shouldn't click on all attachments and visit all Websites he gets via spam mails," he says. "Precaution is always a good idea. AV and ISS tools will help him, of course."

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.