AV-Comparatives releases the latest proactive tests results

December 4, 2008 · Filed Under News, Security

AV-Comparatives, the Austrian team of experts dedicated to antivirus tests acknowledged as a reference point in the field, has published the second part of the mid-year comparative, an ideal addendum to the one already released in the past September. This time the aim is to evaluate the antimalware tools effectiveness against unknown threats, in a test scenario meant to prove the heuristic part and the generic markers of the on-demand scanning engines.

Being a continuation of the previous test focused on the detection skills for the already known malware samples, AV-Comparatives’ report n.20 takes into consideration the same antivirus software tested then at their max capabilities, with the same signatures updates dating back to August 4, 2008. The qualifying difference of the new report is made up by the unique malware samples used, collected by the Austrian labs between the 4th and the 31st of August 2008 and grouped in two separate sets on a period of one and 4 weeks for a total amount of about 45,000 viruses, worms, malicious scripts, backdoors, trojans and others.

Now as then, the group of 16 antivirus software put on test in the comparative includes the following products: avast! Professional Edition 4.8, AVG Anti-Virus 8.0, AVIRA AntiVir Premium 8.1, BitDefender Antivirus 11, eScan Anti-Virus 9.0, ESET NOD32 Anti-Virus 3.0, F-Secure Anti-Virus 2009, G DATA AntiVirusKit (AVK) 2009, Kaspersky Anti-Virus 2009, McAfee VirusScan Plus 12.1, Microsoft OneCare 2.5, Norman Antivirus & Anti-Spyware 7.1, Sophos Endpoint Protection 7.5.1, Symantec Norton Anti-Virus 2009, TrustPort Antivirus Workstation 2.8, VBA32 Scanner for Windows 3.12.8.2.

The simple summary tables highlight the fact that the new comparative brings several surprises: compared to the unknown malware, the majority of antivirus software goes badly down with the achieved detection rates, and if in the previous report there wasn’t any drop below 70% in this case the rates go from 71% of Avira and Kaspersky to the poor 8% of eScan. A special emphasis has then been given to the number of triggered false alarms (or false positives), a rate that has been used to penalize the software affecting the final classification.

As already highlighted in the previous tests, AVIRA AntiVir continues to be the leading edge of worldwide antivirus software being able to detect 71% of unknown samples after the first week and 67% after the fourth. Kaspersky regains the ground lost in the August comparative achieving the first place together with AVIRA with set A (71% in this case too) and the second (60%) with set B.

Behind AntiVir and KAV go GDATA (66% with set A and 59% with set B), NOD32 (54% and 51%), Sophos (51% & 50%) with all the rest following. It’s worth reporting that, in comparison to the August comparative when they achieved the third place, Symantec Norton Anti-Virus and McAfee VirusScan go down by 4 positions and more. Andreas Clementi highlights how those kind of proactive protections used in real-time scanning have been intentionally excluded from the tests, nonetheless their activation could increase the detection rates of unknown threats in regard of the simple heuristic on-demand scan.

As already said, the samples detection rates are only one of the two elements evaluated for the antivirus final classification, being the number of false positives the other. Rising a false alarm about a malware on a legit software can cause as much troubles like a real infection, the report states, and it is for this reason that AVIRA, Kaspersky and other products, even if they have obtained very good results in identifying samples, have been penalized with a lower classification.

So the certification level ADVANCED+ has been achieved by ESET NOD32 only, that has detected 20% less of the samples that AVIRA AntiVir has discovered but has triggered only 7 false alarms. AntiVir, on the contrary, with its 17 false positives hasn’t gone beyond the ADVANCED certification level, which also includes Kaspersky, Microsoft, Symantec, McAfee and GDATA.

The lowest rank, that is the STANDARD certification, includes TrustPort, BitDefender, AVG, Avast, Norman and VBA32. Sophos, F-Secure and e-Scan have achieved no certification, being overall the worst of the proved antivirus software. As usual, the report cares to recommend to evaluate features other than detection rates to choose an antivirus suitable for one’s case, stating that “all the tested products are already selected from a group of very good scanners and if used correctly and kept up-to-date, users can feel safe with any of them“.

This post has been featured on Slashdot on December 5, 2008, producing on the blog a maximum daily peak of 2850 unique visitors and 3790 pageviews (source: LLOOGG).

Share this post!

Comments

4 Responses to “AV-Comparatives releases the latest proactive tests results”

Bob Bowen on February 24th, 2009 2:08 pm

I was wondering whether Customer Service and PRO plays any role in your evaluation of AV products? I feel this is an important part of a good AV Company. For example, I purchased Avira Personal Premium, which installs fine, but will not update, saying an “internal error” occurred. For two months I have been trying to get either a refund, or a working AV program from Avira, and Avira simply ignores my requests. Many buyers have the same problem - please see http://www.wilderssecurity.com/showthread.php?t=225874 and http://tinyurl.com/cue9cf Thank you.

Browser Internet Explorer 7.0 on the O.S. Windows XP
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; Avant Browser; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
Sir Arthur, King of Ghouls'n Ghosts on February 24th, 2009 10:28 pm

I was wondering whether Customer Service and PRO plays any role in your evaluation of AV products?

Well, actually these aren’t my evaluations but AV-Comparatives evaluations, and they are only focused on the detection rates of the antivirus products…

Browser Firefox 2.0.0.16 on the O.S. Windows XP
Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16
Bob Bowen on February 25th, 2009 3:19 am

Thank you, but of what use are detection rates if one is unable to use the AV program because it has an updating bug. I give Avira Personal Premium 0 and many others have the same problem, But our outcries go unheeded by Avira. My Event Viewer, all three catagories, don’t show a single warning or error. With an Intel Core 2 Duo, 2.46 GHz, 140 GB free HDD space, 2 GB RAM and XP Home SP3 fully updated, there’s enough to run Avira. But the point is I purchased the AV in all good faith but cannot use it. From early January I have been on Avira’s Forum trying to get some action from them, but in vain. So much for Avira.

Browser Internet Explorer 7.0 on the O.S. Windows XP
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; Avant Browser; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
Sir Arthur, King of Ghouls'n Ghosts on February 25th, 2009 11:02 am

Well, it’s unfortunate that you and others are unable to run AntiVir, and yes, an excellent detection rate is useless if you are unable to update the AV.

So said, I have never had (or listened by myself) any kind of problems with the AntiVir update feature, even with the free edition of the AV. What I can think about the problem you talk about is some sort of incompatibility between your system(s) and the AV, between the AV and the Internet (maybe router) settings and or software/hardware firewall/other security software….

Or a bug in the AV, and in this eventuality, if Avira didn’t answered to your questions, I would insist in contacting the company or try another antivirus solution….

Browser Firefox 2.0.0.16 on the O.S. Windows XP
Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16