3 Lessons From Layered Defense's Missed Attacks

Layering security measures typically protects systems better: Research (PDF) by three University of Michigan graduate students in 2008, for example, found that using multiple antivirus engines result in much better protection than using a single program.

Yet recent analysis by NSS Labs highlights that layering security devices rarely catches all attacks, and the attacks that manage to dodge defenses do so with regularity. The analysis -- a survey of the company's past tests of next-generation firewalls, intrusion prevention systems, and endpoint protection software -- found that the tested products tended to fail in similar ways. While two products always performed better together than individually, their combined performances varied tremendously.

Overall, the lesson is that companies need to carefully select technologies to derive the greatest benefits from overlapping security measures, says Stefan Frei, research director at NSS Labs and author of the analysis.

"It really depends on what you combine, so you have to know which devices have the least amount of overlap," Frei says.

While defense in depth is still a desirable strategy, NSS Labs' analysis suggests that companies need to look more deeply to get the most out of layering security measures. Most importantly, the analysis found that products tended to fail to detect attacks in ways that appeared to be correlated across products.

Frei combined data from four previous studies that NSS Labs performed during the past 18 months to study the correlation between detection failures. In 2012, the company studied how well 16 intrusion prevention systems and eight next-generation firewalls did against nearly 1,500 exploits. In 2013, the company repeated the tests against next-generation firewalls using more than 1,700 exploits. The company also ran 13 endpoint protection packages against 43 recent exploits.

Frei and other experts pointed to three main lessons from the analysis:

1. Layering defenses does work
Combining products did reduce the effectiveness of attacks, albeit not as much as expected. Over the three tests of intrusion prevention systems and next-generation firewalls, the average failure rate varied between 4 and 9 percent. But two products together typically failed only 0.8 percent of the time. And while the average failure rate for endpoint protection products was 45 percent, using two programs together reduced it to 26 percent.

Yet adding on devices typically does not drive the number of failures to zero, as models that assume random failures would predict. While four next-generation firewalls in combination should have caught all the exploits in the 2013 test, in reality eight exploit bypassed all the tested firewalls, the analysis found.

[IT security professionals working to keep their organizations' systems and data safe need to follow the adage: To protect against a cyberattacker, you have to think like a cyberattacker. See How Attackers Choose Which Vulnerabilities To Exploit.]

"This significant correlation of detection failures indicates that deploying multiple products within a security category, or even multiple products across multiple categories ... does not always provide the 'defense in depth' that we are led to believe exists from studying vendor claims for the efficacy of their products," Frei states in the report.

2. Different technologies work better together
Companies that do decide to pair products to increase security should attempt to find products that will have the least amount of overlap between their detection failures. A comparison of next-generation firewalls, for example, found that a combination of the two worst performers still did more poorly than many other single products, while a combination of the two best products did not product the best result.

Overall, companies should look for products that take different approaches to security to best lock down their systems. Pairing antivirus software with a product that uses virtual containers is one example. Malwarebytes and Immunet, now part of Sourcefire's FireAmp product, typically approach malware detection in a different way than other AV companies and so are frequently used as companion programs.

"One antivirus program is enough to cover that section of the threat landscape," says Doug Swanson, vice president of development for Malwarebytes. "You pick up something like us to go after the portion of the landscape that they do not cover."

For companies that use multiple products in the same category -- firewall, intrusion prevention system, or antivirus -- performing their own analysis of the detection failures can help them identify products that tend to have correlated failures.

3. Duplicate, but expect failure
In the end, duplicating every product to gain defense in depth is cost-prohibitive, so companies have to be smart about their strategies, says Jason Brvenik, vice president of security strategy for Sourcefire, a maker of network and endpoint protection systems. As part of their strategies, businesses should plan for when their defenses fail.

"You do see overlap when you use different vendors for different aspects of security," he says. "But we are seeing attackers still being able to get in. They are still winning."

Companies need to design their layered defense to include, not just stopping the attackers from breaking in, but to alert defenders and slow down attackers' efforts once they are in the network, says Brvenik.

"It is changing the way you solve the problems from stopping the bad guys, to stopping them where you can, and understanding that when they succeed, you need to interdict before they get away with it," he says.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.