Top 10 Reasons Security Products Don't Work

No network is really safe.

If there's one thing security users and vendors agree on, it's that there's no silver bullet for preventing an attack. No combination of security tools, and certainly no single product, can guarantee the safety of your network and data.

But customers and vendors see things very differently when it comes to why security tools can't do it all out of the box. IT managers say security tools are very reliable in protecting their sensitive data and often make more work for them with false alarms, security holes, complexity, and lousy support from the vendors. Security vendors blame user frustration on misconceptions and false expectations on just what these tools can do in the age of ever-more sophisticated hackers and increasingly profitable cybercrime, as well as how organizations deploy and maintain their products (or not).

We hear these arguments from both sides of the security equation here at Dark Reading. So we compiled the top complaints from both sides about why security products aren't cutting it and tapped some key security experts on where these complaints come from. We also offer solutions, when possible, to these problems with today's products and perceptions about them.

If you don't agree (or even if you do) with these complaints from users and vendors, please click on the link to "Discuss this story" and post a message to our message board. We want to hear from you.

(Editor's note: Please don't use email. The message boards are completely anonymous and a chance for your opinion to be heard).

The List:

— The Staff, Dark Reading

Next Page: Too many false alarms

1. Security tools generate too many false negatives and false positives

The last thing you want is to miss a real attack while manually sifting through piles of false alarms from your security tools. False positives are not only a major resource drain, but IT managers worry that these could distract IT from real threats.

IDSes have long had a bad reputation for the volume of false positives they can produce. Newer generations of these devices aren't as sensitive and labor-intensive with their alarms as previous revs, but IT managers say they still get flooded with unnecessary alarms for legitimate traffic.

"I spend a lot of time tuning them properly, or we get bombarded with false positives," says Robert Mims, vice president of security and engineering for MedAvant Healthcare Solutions, which runs ISS RealSecure IDS sensors. It's a delicate balance of sorting through the real ones and the ones where the boxes are "crying wolf." So the company has put in a rule to ignore certain unnecessary alerts, he says.

If a false positive is a headache, then a false negative is a nightmare. A false negative is when bad guys, bad traffic, or malware slip past your IPS or antivirus application undetected. These alarms are becoming more of a problem today than false negatives, security experts say.

It's the nature of signature-based security tools. IDS/IPS and AV tools go with what they know, and sometimes that means letting unknown malware slip by or sounding an alarm for something that looks suspicious based on their signature patterns. IT managers say they're frustrated with these limitations of such tools. It's either too much extraneous information or not enough critical information, they say.

The main problem, security experts say, is false expectations on the part of users for just what their security tools can do. "Security products on their best day are only 50 percent solutions out of the box," says Michael Rothman, president of Security Incite. "You have to tune them to your environment and when you screw that up, it results in false positives and false negatives."

Even IPS vendors admit there's no way to get around false positives and negatives. "You're always going to have false positives and negatives with almost any security software you use," including IPSes, IDSes, and vulnerability assessment tools, says Marc Maiffret, CTO and chief hacking officer for eEye Digital Security, which makes an IPS. "I'd love be able to say there were no false positives in our product. But that would be like saying your software is never going to have bugs."

"IPS and reactive security never solved an operational security problem for anyone," says Thomas Ptacek, a researcher with Matasano Security. "You have to be more careful about what you allow on the network and what you have exposed," he says.

But IPS vendors say their protection filters undergo through a major vetting process before they get released. "It gets a pure review. We look for false positives and performance degradation, and we can't release it if it shows a negligible percentage of performance degradation or a false positive," says David Endler, director of security research for 3Com/Tipping Point. "In some cases, coming up with the filter is easy, but getting it past the gates is hard."

Next Page: Products are riddled with holes

2. Products are riddled with holes

There's something unnerving about having to patch your security product on a regular basis. And with the recent wave of security vulnerabilities reported for security tools such as those of AV giants McAfee and Symantec, some users are wondering whether it's only a matter of time until security vendors have their own monthly patch day like Microsoft's.

It goes with the territory: Symantec, McAfee, and Cisco are big-time and, therefore, big targets, too. Cisco last week made the unusual move of reporting vulnerabilities found in its VPN and firewall product. (See Cisco Reports New Vulnerabilities.) "The hunters are becoming the hunted," says Allwyn Sequeira, vice president of engineering and operations for Blue Lane.

Security tools have always had holes. Think of the Cisco routers in the late 1980s and early versions of antivirus scanning for MS-DOS, says Nate Lawson, engineering director for Cryptography Research. The difference now is attackers in general are moving up the stack and finding more creative ways to get inside, by poking holes in security software.

Part of the problem is that security code is complicated and, therefore, prone to vulnerabilities, says Thomas Ptacek, a researcher with Matasano Security. "I'm concerned that security products are harder to build and most security companies don't get [more sophisticated] developers."

McAfee and Symantec seem to have the most holes among AV tools, says Marc Maiffret, CTO and chief hacking officer for eEye Digital Security, whose researchers have discovered holes in McAfee and Symantec products over the past few months.

Meanwhile, hackers are increasingly poking around and finding new holes in IDS/IPSes, too. (See IDS/IPS: Too Many Holes?) Black Hat researchers earlier this month showed just how easy it is to slip by IPS/IDSes, even with very old and well-known exploits. A French researcher did so with a tool he built and a slightly repackaged Blaster worm99905, for instance.

Security tools are becoming an attacker's welcome mat, and they're getting walked all over in some cases. "In the last six months, one of the easiest ways to own desktops in America is through security software," says Maiffret.

How can you stay secure when your security products (gulp) aren't?

The good news is security tools still do more good than harm. "As with any software, you're always going to have vulnerabilities," Maiffret says. "You might have a couple of bugs a year in your AV client, but that's two bugs versus the 100,000 viruses it defended you against."

Next Page: They don't protect against zero-day attacks

3. They don't protect against zero-day attacks

That, however, depends on how your vendor defines a zero-day attack. The original definition of zero-day is a vulnerability that has not yet been discovered -- no security tools are able to consistently stop those. But security vendors that claim their tools protect against zero-day attacks, usually mean they stop new exploits for a known vulnerability, security experts say.

Or they've instead come up with a generic fix for a new Patch Tuesday vulnerability, for example, and say they stopped a zero-day. "They just protected you from the exploits before they became public," says Marc Maiffret, CTO and chief hacking officer for eEye Digital Security.

None of this is very reassuring for customers who worry that if a zero-day were to hit, they wouldn't be protected by today's tools.

"No technology out there is reliable in finding new vulnerabilities as they are exploited in the wild," says Matasano's Ptacek. Ptacek says Cisco and ISS are the two most notorious for claiming they stop zero-day attacks, but these attacks are based on known vulnerabilities, so they don't fit the original definition of a zero-day.

While network-based IPS products can't stop zero-day attacks, host-based IPSes can because they detect buffer overflows, which many zero-day attacks use, says Maiffret.

Anamaly-based detection, where a security tool analyzes traffic based on behaviors, not signatures of known attacks, can help somewhat as an extra filter, but it won't necessarily catch a zero-day exploit. Change is a constant in networks: New routers get added and other new equipment in a merger or acquisition, for instance, and if these tools don't keep up, they end up reporting false positives.

"It's very hard to baseline behavioral systems," says Sequeira of Blue Lane. "It can give you clues and alerts, but you'll still spend a lot of time sifting through information, correlating it and using intuition atop that to make it work."

It all comes back to security products basically protecting you from the known -– rather than the unknown.

Next Page: Security products don't work well together

4. Security products don't work well together

Point products can be a pain –- your firewall, IPS, vulnerability assessment, antivirus, antispyware, and host-based security tools for the most part all do their own thing and don't talk to one another. Some enterprises complain that the danger of these tools not sharing is they may not have a true picture of their security landscape until it's too late and they've been hacked.

But does sharing data among all of these tools really make sense? The promise of security information management (SIM) has yet to be fulfilled, says Thomas Ptacek, a researcher with Matasano Security. And the all-in-one security tool approach so far is mostly a small- to medium-sized business phenomenon.

"People don't need a lot of correlation between tools. When they do, it's specific, and they build their own" interfaces," Ptacek says.

And integrating security data may not be useful anyway. "Security vendors haven't demonstrated that you can take these pieces of a broken mirror and get a clear picture," Ptacek adds. "These products are built separately and aren't related to one another."

Michael Rothman, president of Security Incite, says it's up to customers to press vendors for the integration if they need it -- not the other way around. "If you need shared data, you should push vendors for it," Rothman says. "The vendor that's going to prevail is the one that provides the most actionable information to make sure you can block those attacks. If that means you pull it out of a syslog, or there's a product-integration relationship, that's what they're going to have to do."

But it's not a matter of security tools interoperating, says Nate Lawson, engineering director for Cryptography Research. The missing link is a standard way for security tools to report their vulnerability data in a common format, he says. "There's room for standardization here -– if an IDS and AV scanner throw their report data into a database, that would be useful," he says.

Meanwhile, AV, anti-spyware, and host-based IPS products are gradually becoming integrated. Antivirus vendors such as Symantec and McAfee are adding more host-based IPS and spyware, for instance, and spyware and host-based IPS vendors are adding more AV features.

Still, not all security tools will, nor should, work together, says Tom Maufer, director of technical marketing for Mu Security. "AV scanners, penetration testers, and Web app scanners are designed to do a very specific job," Maufer says. "That's where that vendor's expertise lies."

Next Page: Security tools are too complex

5. Security tools are too complex

IDS/IPS products are the most frequently cited offenders here, with their log files that start spewing more volume than an "I Love Lucy" candy factory.

"It's not that the product doesn't work, but they take up so many resources to keep running," says Eric Ogren, security analyst with the Enterprise Strategy Group. "It all looks good in the demo, but the operational overhead [to keep it running] is prohibitive. This kind of complexity is a huge issue."

You could just as easily toss access control systems on that same pile, he adds, with all the rights and permissions that accumulate over time and are extremely challenging to keep current as users move around or change jobs. "I'd need to hire an army to handle the helpdesk calls and to keep permissions up to date, adding and removing ones not needed anymore," Ogren laughs. But real control is only as good as the most recent updates.

Ditto for these newly emerging content inspection suites that keep users from sending out problematic emails but require regular updating. "You keep adding words and search phrases, and then the list gets too long to maintain, and the false positives proliferate," Ogren notes.

Rather than keeping companies safe, such products threaten to bury those tasked with administering them. So what about turning to a security's customer support for help?

Russ Cooper, director of publishing for the risk intel team for security vendor Cybertrust, says that's fine, at least if you expect to get someone on the phone within an hour. That's not the way most customer support works in any industry these days, though some IT vendors might –- for a hefty premium.

"With security, you're almost always talking about something that's happening right now," Cooper says. "I can't imagine that waiting an hour works for anyone. I don't want to know tomorrow what this prompt is telling me and why I can't do what I want to do."

Too often, users may get prompts that only make the problem worse –- and may well exacerbate any problem going on. "Online help has to be the stupidest invention known to man. Never assume I have network connectivity when you’re going to go and try and get me help," Cooper fumes, noting how often this issue comes up with Microsoft Office products and Visual Studio. "If [solving the problem] means bringing up a link and opening a Web page, I may have just done something I'm not supposed to do."

Better that the documentation spell it out clearly, or that users be able to search their own desktops for the exact verbiage contained in the prompts. Again, this is a rarity in most products.

Ogren agrees that fewer prompts would help. "The granularity and visibility of help information should shift depending on the user —- an admin versus an end-user -- and the prompting should reflect that," he says. "The majority of security products that are out there don't do that. They use a one-size-fits-all mentality."

Next Page: Users don't fully understand the product's capabilities

6. Users don't fully understand the product's capabilities

While IT managers struggle with product flaws and poor customer support from vendors, the vendors themselves are also struggling with user errors and misperceptions that sometimes cause their products to fall short of their potential. One of the chief problems, they say, is that security professionals don’t always fully understand what the product can do.

Predictably, experts say, many users overestimate the capabilities of a given security product. "Companies are always looking for a silver bullet," says Slade Griffin, security engineer at Sword and Shield Enterprise Security, a consulting firm that helps corporations deploy a wide array of security technology. "The fact is there’s nothing out there that will cover all the bases, so you shouldn’t expect too much from any one product."

Interestingly, though, many vendors are also frustrated by customers that set the bar too low. "What I see often is a user that says, 'We had a breach of this type last month, and we want to make it stop,'" says Jason Anderson, vice president of engineering at Lancope, who has served as an executive at other security companies as well. "They make a quick search to find a product that solves that one problem, and they buy it. They don't pay enough attention to the product's broader capabilities."

Another veteran of several security vendors agrees. He tells a story on one of his customers: "A major service provider bought a ton of our [Network Address Translation] boxes, which came with VPN and firewall capabilities," he recalls. "Then [the customer] came back to us months later and said, 'Hey, this would be a great product if you added VPN and firewall [capabilities].' They were so focused on the one function that they didn't even see that it already had the functionality they needed."

Some customers also operate under the mistaken belief that once they've installed a new security product, they're immune to a particular exploit. "Exploits change," Anderson observes. "They evolve. That product might stop a certain type of breach for awhile, but there will be new attacks, and the product will evolve, too."

Bottom line: Find out what a particular product can do before you buy it. Don't expect it to solve all of your problems forever, but if it is capable of multiple functions, consider using as many of them as you can. Misperceptions about a product's capabilities -- expecting too much or too little -- can often lead to dissatisfaction or wasted investment.

Next Page: Users fail to install/deploy the product correctly

7. Users fail to install/deploy the product correctly

If you've ever had trouble installing a home appliance, and yes, most of us have, why should a security appliance be any different? Many IT departments pinch pennies by skipping the vendor's training sessions or eschewing its implementation consulting services, just like those of us who don't like to follow the instructions when they install a new garbage disposal. And the results can be egg (or other messy foods) on your face.

"The most common mistake I've seen with security products is that the customer installs them using the default configuration, without adding any new policies to address their specific environment," says Chris Roeckl, vice president of corporate marketing at Fortinet and another grizzled veteran of multiple security companies. "This is particularly true at small companies, where people sometimes feel that they don't have any special needs, or that they don't have the skills to do the configuration. But most security products are made to fit the customer's specific policies -- to some degree, you have to tell it what to do."

Brian Foster, senior director of product management for Symantec's end point security group, concurs. "A high percentage of threats are successful because the end points are not properly configured," he says. "If it's not set up in the right way, it's not going to work."

Foster gives the example of the Blaster worm, which infected many systems through open ports. "One of the basic best practices that companies should always follow when they install a device is to turn off ports that aren't being used," he says. "That's not even a function of the security product itself -- it's just good IT policy. But in that case, a lot of companies hadn't followed it, and they paid."

Be sure you set up your security products to record their activities as they occur, advises Sword and Shield's Griffin. "When we go in to troubleshoot a problem, one of the most frustrating things is when the client has a security product, but has failed to turn on the audit functions, such as data logging," he says. "When that happens, we can't tell how the product was used or how it behaved during an incident."

Bottom line: Be sure you've configured your new products correctly for your specific environment. If you don't know how, ask the vendor or an expert consultant to help with the implementation. Installation mistakes are often a reason why products fail to operate correctly when a threat occurs.

Next Page: Users do too much product "tuning"

8. Users do too much "tuning" of the product's functionality

Once they've got a new security product in place and properly configured, IT staffers should beware of doing so much "tuning" and customization of the product that they effectively limit its functionality, experts say. If you tinker too much with the tints and hues on your television, sometimes you can lose the whole picture.

"In larger companies with multiple administrators, what sometimes happens is that one administrator will write a 'deny all' policy that somehow ends up at the top of the policy rules," says Roeckl. "One bad policy can affect the rest of the rule set and effectively disable the whole system."

Lancope's Anderson agrees. "Sometimes, [administrators] make assumptions about what they'll need and don't need, and that will cause them to turn off important functions of the product," he says. "Some companies use a product for a long time before they become aware that some key functions have been disabled."

When an enterprise changes security administrators or staff, sometimes the new people don't understand how their predecessors have written the rules or policies in the product," Anderson notes. "The 16th rule of 200 may be the one that turns all the rest of them off," he says. "When you start changing rules, you need to understand the impact."

Some IT departments turn off key functions in their security products because their users complain that the functions make it too hard to log on or navigate the network," observes Griffin. "They may disable key security functions so they don't impact the convenience of accessing the network. It's up to the organization to assess the risk of disabling core functions in a [security] product."

Bottom line: Be sure you understand the potential impact of any change you make in the rules or settings of your security products. Experts say it's a good idea to have a third party check your security applications and appliances every 6-12 months, to ensure that you're using the full functionality of a security product and haven't turned off any core functions.

Next Page: Users fail to update the product as it evolves

9. Users fail to update the product as it evolves

Only one thing is constant about security products: Change. As exploits evolve and new technologies become available, your vendors will make changes in their products, and you must change yours as well. If you don't, you could leave yourself open to attack.

"One of our biggest frustrations in the enterprise anti-virus business is that a lot of companies don't keep their AV software up to date," says Symantec's Foster. "The threat space is constantly changing, and we're constantly changing our product to keep up. But if you don't do the updates, you're not going to get the benefit of those changes."

Roeckl echoes that frustration. "We develop a solution, and then nobody [at the customer site] does the update," he says. Fortinet is addressing this problem for some customers with a new option called FortiGuard, which distributes updates and configuration changes automatically via the company's distribution network.

"That works for a lot of smaller companies, though large companies often want to pre-stage a deployment and test it before they do an update," Roeckl says.

The vendors were careful to distinguish "updates" -- which are typically revisions designed to help the product stop a new exploit -- from "upgrades" or "new releases," in which the vendor adds new functionality to the product. Updates should be done frequently and as quickly as possible without replacing the existing software or appliance; upgrades require a wholesale replacement of older products, and often don't happen until long after a new release becomes available.

"I'm delivering a new release of our anti-virus products every 12 months, but I recognize that a lot of users can't keep up with me," says Symantec's Foster. Anderson says many enterprises can take as long as one or two years to do an upgrade, even if the new software is free under a maintenance agreement.

Whether you're doing an update or an upgrade, it's a good idea to familiarize yourself with a new piece of software before you begin to use it, Roeckl observes. "Occasionally, we'll find a small feature in a product that people really like, so we'll raise its status in the GUI," he says. "At that point, a lot of users will complain because they can't find it in the new release." In making the product more useable, a vendor may change the user interface and cause a temporary panic, he explains. "It's a little like getting Vista, and finding out that it looks a whole lot more like MacOS," he says. "It's a better interface, but it can be confusing to the user at first."

Bottom line: When a vendor updates one of your security products, you should deploy the update as swiftly as possible. Failure to do an update can leave your systems vulnerable to attack.

Next Page: The Blame Game

10. Someone must be blamed

The phase of the moon.

Vendors or resellers that are clueless beyond belief.

That recently departed CIO who never was too bright.

Al Qaeda.

Legions of stupid &*#$@! end-users surprised to find their computers are unplugged, or they're in the wrong cubicle, or gosh, the night janitor tossed their password Post-Its.

And who's your favorite scapegoat?

Sociologists might say this simply human nature. Historians might point to the way leaders have historically shifted blame to some marginalized population in order to rally the people. Psychologists might pronounce it a severe narcissistic disorder and suggest the patient take more responsibility for themselves.

While all that may be true, assessing blame doesn't move the ball forward (unless you work in the legal department). We could even say it's counter-productive to enterprise security and protecting both data and end-users. So be specific. Don't ask for generic filtering, when what you need is an AV email server gateway that will do heuristic scanning and allow you to quarantine. Talk to other similarly situated customers who use the same product -- and if you're a vendor insist that your customer undertake this sort of due diligence.

What?

That's nothing more than managing (or scaling back) expectations early in the game, you say?

Damn right it is. Better now than after countless dollars and hours have been spent trying to improve security. And who knows, this kind of process might just circumvent who gets blamed, and instead turns it into a discussion of who gets the credit.

Either way, it deserves to be shared.