Your Peace of Mind is our Commitment

Contact Us English Recent Articles

Imperva Study Claims Anti-Virus is Ineffective

First published: 02nd January 2012

US security company Imperva has published a report on the effectiveness of anti-virus software based on their work with students from the Technion-Israel Institute of Technology. The team collected 82 samples of malware and used the VirusTotal website to test whether they were detected by 40 anti-virus products. Based on this, they concluded that initial detection rates were as low as 5%, and recommended that compliance rules should be eased, freeing up money for "more effective" security measures.

Yui Kee's Chief Consultant Allan Dyer took a different view and harshly criticised Imperva's study, saying, "I was surprised at the small sample set Imperva used - just 82 samples, collected from honey pots, google and hacker forums. Can this really reflect on effectiveness against the millions of malware samples known to exist?"

In comparison, AV-Test uses two test sets in its Protection tests:

Dyer continued, "A second surprise is that Imperva do not do their own testing, they threw the samples at VirusTotal. VirusTotal is a useful website, but they are quite explicit that it is unsuitable for product testing. Imperva takes the short form of VirusTotal's advice, 'not designed as a tool to perform antivirus comparative analyses', and counter it in the study's 'Limitations' section saying that they are not doing a comparison. Imperva ignore the longer advice, that details why VirusTotal is unsuitable for both comparative and effectiveness testing."

Dyer concluded, "Anti-virus testing is notoriously difficult, and competent researchers put a lot of work into making sure they use methodologies that will produce relevant, reliable results. Did Imperva?"

Updated: 07th January 2012

Imperva's study has generated a lot of discussion and criticism, including from David Harley on his blog, Max Eddy at PC Magazine, and Kurt Wismer on his blog.

More Information