NSS Labs Files Antitrust Suit Against Symantec, CrowdStrike, ESET, AMTSO

UPDATED 9/20 with comments from ESET, AV-Test, and CrowdStrike

Security product testing firm NSS Labs today filed an antitrust lawsuit against cybersecurity vendors CrowdStrike, ESET, and Symantec as well as the Anti-Malware Testing Standards Organization (AMTSO) over a vendor-backed testing protocol.

The lawsuit accuses the three security vendors and the nonprofit AMTSO, of which they and other endpoint security vendors are members, of unfairly allowing their products to be tested only by organizations that comply with AMTSO's testing protocol standard. NSS Labs, which also is a member of AMTSO, earlier this year voted against adoption of the standard and says it has no plans to comply with it.

A majority of AMTSO members voted in favor of the standard in May of this year, and most plan to adopt the protocol.

Friction between security vendors and independent testing labs is nothing new. Vendors and labs traditionally have had an uneasy and sometimes contentious relationship over control of the testing process and parameters. NSS Labs' suit appears to represent an escalation of that age-old conflict, security experts say.

NSS Labs is calling foul in its lawsuit: "NSS Labs has suffered antitrust injury as a result of the acts herein alleged because it is the direct and principal target of the concerted refusal to deal/group boycott" any testing organizations that don't adopt ATMSO's testing standard, the lawsuit says.

In an interview with Dark Reading, Jason Brvenik, chief technology officer at NSS Labs, said the ATMSO standard falls short. "Our fundamental focus is that if a product is good enough to sell, it's good enough to test," and NSS Labs shouldn't be forced to comply with ATMSO's standard, he says. "It should be an independent test."

Brvenik says the AMTSO standard does not support independent testing. "It's driven by vendors to create a picture of capabilities that are not true," for example, he says. "The standard is more like guidelines to interact with than a standard, and it doesn't make things better for products" or the way they are tested, he says.

According to the NSS Labs suit, other vendors that spoke out against the adoption of AMTSO's standards included AVComparatives, AV-Test, and SKD LABS. None of those vendors are named as parties in NSS Labs' case. Efforts to reach AVComparatives, and SKD Labs were unsuccessful as of this posting.

AV-Test CEO Andreas Marx said he had just heard about the suit and is unable to comment at this time.

CrowdStrike called the suit groundless: "NSS is a for-profit, pay-to-play testing organization that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing. We believe their lawsuit is baseless," the company said in a statement.

"CrowdStrike supports independent and standards-based testing—including public testing—for our products and for the industry. We have undergone independent testing with AV-Comparatives, SE Labs, and MITRE and you can find information on that testing here," the statement said. "We applaud AMTSO’s efforts to promote clear, consistent, and transparent testing standards."

ESET said it had not been officially contacted about the suit, but that it refutes the allegations. "We are aware of the allegations stated in the blog post from NSS Labs, however, we have yet to receive official legal communication. As legal proceedings appear to have been initiated, we are unable to say more at this time, beyond the statement that we categorically deny the allegations," an ESET spokesperson said. "Our customers should be reassured that ESET’s products have been rigorously tested by many independent third-party reviewers around the world, received numerous awards for their level of protection of end users over many years, and are widely praised by industry-leading specialists." 

Symantec would not comment on the case, and efforts to reach AMTSO  were unsuccessful as of this posting.

In a blog post earlier this month, ATMSO president Dennis Batchelder wrote that the protocol is a voluntary framework for testing anti-malware software "fairly and transparently." 

For enterprises, there aren't many options for vetting security software. Most don't have the resources to perform their own in-house testing of security products, so they rely on consulting firms' recommendations, third-party testing organizations — or the claims of their vendor.

Jon Oltsik, senior principal analyst with consulting firm Enterprise Strategy Group, says he's seen enterprises struggle with the testing dilemma. "Customers don't know how to test the efficacy of next-generation endpoint security products," he says. "No one trusts vendors to test their own product."

The concept of a vetted product testing standard is a "very good idea," says Oltsik, who notes that he has not specifically studied ATMSO's protocol.

Bottom Line
NSS Labs meantime argues that the AMTSO and its standard are anti-competitive. "They claim to try to improve testing but what they're actually doing is actively preventing unbiased testing. Further, vendors are openly exerting control and collectively boycotting testing organizations that don't comply with their AMTSO standards — even going so far as to block the independent purchase and testing of their products," Vikram Phatak, CEO of NSS Labs wrote in a blog post today announcing the suit.

Meanwhile, NSS Labs claims in its lawsuit that AMTSO's efforts have hurt its bottom line. "NSS Labs has lost sales and profits from the sale and license of its public testing reports, including its AEP Group Test reports, because it cannot charge customers who purchase reports that do not include all market participants as much as it could charge for reports that included all market participants." 

Black Hat Europe returns to London Dec. 3-6, 2018, with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions, and service providers in the Business Hall. Click for information on the conference and to register.