Five Reasons SIEM Deployments Fail

Even as the security information and event management (SIEM) market grows, SIEM has picked up a bit of a bad rap among many IT executives for failing to live up to the hype. Many organizations, however, buy into SIEM with unrealistic expectations, believing it will offer them a quick-fix security panacea that automates security detection and protection. The truth is, SIEM is a tool like any other and requires expertise and sleeves-rolled-up work to offer true value.

This expectations gap can result in some serious deployment disappointment. Here are some of the biggest contributing factors to SIEM's spotty success rate:

1. SIEM is hard to use.
The nut of it really comes down to the fact that SIEM is not an easy technology to use. Part of that rests squarely at the feet of SIEM vendors, who still have not done enough to simplify their products -- particularly for small and midsize enterprises, says Mike Rothman, analyst and president of Securosis.

"I think that we need to see more of a set of deployment models [that] make it easier for folks that aren't necessarily experts on this stuff to use it. In order for this market to continue to grow and to continue to drive value to customers, it has to be easier to use, and it has to be much more applicable to the midmarket customer," Rothman says. "Right now the technology is still way too complicated for that."

At the same time, Rothman says organizations are not sufficiently "skilling up" their security workers to squeeze value out of their SIEM investments -- particularly when it comes to figuring out what to do once their SIEM's correlation engine alerts them to a problem.

"Ultimately someone still has to figure out on-site what the hell happened, and that's where you still have a skills gap, which is not just gathering the information, not just analyzing the information, [not just] figuring out when something has gone amiss, but then figuring out how to deal with it from a remediation standpoint," Rothman says. "That's not an indictment of the SIEM market per se; it just still shows the skills gap that we have, certainly within the midmarket type of context in terms of these folks doing whatever needs to be done once they figure out something's broken."

2. Log management lacks standardization.
In order to truly automate the collection of data from different devices and automate the parsing of all that data, organizations need standardization within their logged events, says Scott Crawford, analyst for Enterprise Management Associates.

"This is one of the biggest issues of event management," Crawford says. "A whole range of point products can produce a very wide variety of ways to characterize events."

Crawford says ArcSight has tried to help the field with its common event format, but its position as a vendor has made this effort lack the credibility needed to unify the market. He believes the best shot at the moment is with Mitre's initiative to launch the Common Event Expression -- similar to its successful Common Vulnerability Expression (CVE) for vulnerability management -- on which Mitre has been working in baby steps for the past few years and released an architectural overview earlier in the year.

3. IT can't rise above organizational power struggles.
"One of the key challenges our customers face is really getting all parts of the company to work together to actually make the connections to get the right scope of monitoring," says Joe Gottlieb, president and CEO of SenSage. "And the things you want to monitor sit in different places within the organization and are controlled by different parts of the organization."

As the old phrase goes, "garbage in, garbage out." If organizations cannot get all of the security and event data together that would help put together a picture of a specific incident, then the strongest correlation engine in the world won't do them a much good.

"We all know that has always been one of the most interesting topics in IT: the organizational factions and constituencies and sort of keeping up with all that," Gottlieb says. "That's typically something you have to address to make SIEM work well."

4. Security managers see SIEM as magic.
SIEM expectations frequently don't jibe with reality because many IT managers believe SIEM is about as powerful as Merlin's wand.

"A lot of people look at SIEM like it's this magical box -- I get a SIEM and it's going to do all my work for me," says Eric Knapp, vice president of technology marketing for NitroSecurity. "SIEM has different levels of ease of use, but they all come back to looking at information and drawing conclusions. Unless you're looking at it in the correct context for your specific environment, it's not going to help you as much as it should."

Crawford at EMA agrees.

"I think way too many customers have what we would call magical thinking about SIEM and related type of deployments, like log management," Crawford says. "They expect it to collect everything, expect it to be readily integrated with everything, and they expect correlation to be automatic out of the box without even knowing what they're looking for and what their use case is."

Crawford believes organizations need to lay out the priorities of their SIEM deployments and to figure out objectives, such as whether they are just collecting data for reporting or looking to mine historical data before they acquire their SIEM technology.

"Not nearly enough organizations define that well enough to really know what they're looking for in an acquisition," Crawford says. "And then they get into it, particularly if it is expensive or high end, and start realizing what it's really going to involve to deploy and tune it appropriately, and that can overwhelm them."

5. Scalability nightmares continue to reign.
Crawford's advice is doing that advanced homework is especially important prior to buying a SIEM because, depending on your use case, you could need a level of data analysis scalability that your first vendor may or may not be able to handle.

"Frankly, we get a lot of our projects from our competitors not doing a good enough job with scalability and data analysis," Gottlieb says, explaining that customers frequently come to him following a previously failed SIEM deployment. "Most of the deals we go into come as a result of broadened ambition and/or scope for this category and the current provider can't handle it."

According to Knapp, the more dependent organizations become on networked information and on correlated information, the more data that will flood the SIEM. While some SIEM vendors have told their customers that it is just a matter of "tuning" the tool and ignoring "irrelevant" information, others believe the industry needs to do a better job building tools that can scale so that seemingly irrelevant information that might point to a large-scale trend is not lost.

"If we're throwing away what we view as unimportant, then we'll never find the bigger patterns," Knapp says. "You have to have the ability to look at everything."