New proactive test by AV-Comparatives: are false positives really that important?
The AV-Comparatives Austrian labs have just released their antivirus test for November. Following the usual practice of alternating (during the year) the analysis of the known malware detection rates and that on the antivirus software proactive capabilities, report n.24 follows the previous one related to the malware test-bed collected between January and August 2009 but, contrariwise to this last one, compares the same products to more than 23,000 new samples gathered within the week following the antivirus signatures update.
The tested software are the same as report n.23 with the only, faint difference that Microsoft’s Live OneCare 2.5 is now called Security Essentials 1.0 achieving exactly the same results. To sum up, the full list of products reviewed by AV-Comparatives includes: avast! Professional Edition 4.8, AVG Anti-Virus 8.5, AVIRA AntiVir Premium 9.0, BitDefender Antivirus 13, eScan Anti-Virus 10, ESET NOD32 Anti-Virus 4.0, F-Secure Anti-Virus 10, G DATA AntiVirus 20, Kaspersky Anti-Virus 9, Kingsoft Antivirus 2009, McAfee VirusScan Plus 13, Microsoft Security Essentials 1.0, Norman Antivirus & Anti-Spyware 7.10, Sophos Anti-Virus 7.6.10, Symantec Norton Anti-Virus 17, TrustPort Antivirus 2.8.
The AV-Comparatives proactive test takes into account the security software detection capabilities during the on-demand scanning, hence passive capabilities that leave out those active features like HIPS components, on-the-fly decoding and sandboxing of malicious code, behavioural blockers, real-time update through cloud computing technologies (ie steadily connected to a remote server) and other extra protection layers.
Compared to the previous proactive test (report n.22), the new comparative introduces several differences and a couple of certainties: the antivirus software go up (F-Secure +7, Kaspersky and McAfee +1) and mostly go down (Microsoft and Sophos -3, AVG, eScan and BitDefender -2, NOD32, TrustPort, Symantec and Norman -1) in the ranking, but the first and second places are always won by the “usual suspects” Avira (74% of samples detected) and the multi-engine scanner G DATA (66%).
However, as it already happens for some time now, the simple detection rates data doesn’t meet the certification level awarded to a particular antivirus software because the number of false positives comes into play, a trait that according to the Austrian experts can do more harm than a real infection. Therefore McAfee, TrustPort, Sophos, Norman, KingSoft and above all Avira are heavily penalized by false alarms achieving an award that would have been higher otherwise.
As I have always seen it, Andreas Clementi and colleagues tend to excessively rate the aforementioned false alarms going as far as underestimating what should continue to be the most important thing in the antivirus software tests, ie the ability to defend the user against known and unknown malware. Let’s speak out, a legit software detected as a malicious program by an AV can be a nuisance, but an antivirus with few false alarms that lets banking trojans and Master Boot Record rootkits go through should in no way get a higher award.
If Clementi’s effort is meant to make his comparatives more user-friendly to unexperienced users, I think it’s a fairly unsuccessful effort: an average user would unlikely examine an antivirus test in details to choose the product suitable for him. Furthermore, after reading the full list of software wrongly detected as threats, to me it doesn’t seem that there is a huge amount of programs used on Windows machines worldwide. It would be different if the antivirus had to be deployed in corporate environments, but considering that publishers generally propose a different kind of software system for such environments one can’t help but judge the value of AV-Comparatives work from a consumer standpoint.
Don’t get me wrong, nobody here is trying to knock down the work and the analytic study efforts by AV-Comparatives, a remarkable work performed within the always difficult and complex field of computer security. But in my opinion the excessive importance granted to false positives risks to nullify the results of that work and brings falsified rankings that don’t make justice to a product real value. For truth’s sake I must make clear I am a long time Avira AnviVir Premium user, even though I think that my reasoning is effective despite this fact.
Related posts
- AV-Comparatives feels the pulse of the IT security
- AntiVir Personal is 10 years old and Avira gives its customers a gift
- AV-Comparatives tests antivirus software proactive defenses
- AV-Comparatives releases a new round of antivirus tests
- Free 6 months license for Avira AntiVir Premium
- 22,000 new malware samples per day, a network worm breakout and the sandbox-enabled antivirus
- AVIRA AntiVir is the best antivirus of the year
- AV-Comparatives releases the latest proactive tests results
- AV-Comparatives, the August tests are on-line
- EICAR test file, antimalware testing with no collateral damages
Comments
Leave a Reply
-
· Subscribe to Sir Arthur's Den ·
-
· © Some Rights Reserved ·
The exclusive contents of Sir Arthur's Den are distributed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License, and must be ascribed to Alfonso Maruccia as the sole owner of the related copyrights.
