The Plural Of Data Is Not Analytics

One of the terms most recently in danger of becoming a buzzword has been "analytics." Put it together with the words "big" and "data," and it starts reaching critical mass. Everyone claims to be doing it; figuring out what's real is harder.

You can think of security analytics as information used to drive risk management or incident response decisions (that is, proactive or reactive security decisions). As such, the information is made security-relevant and useful by using data manipulation such as statistical analysis; comparisons against historical data, policies or other previously made decisions; correlation and connection-mapping with disparate data types; false-positive and false-negative identification; various methods of visualization; and other proprietary algorithms and techniques. The data that is manipulated in this fashion may range from events, states and alerts captured by security products, to the output of quantified risk modeling, social media data, directory listings, world news events, or any other searches that are deemed a part of the decision model.

Please note that this excludes the mechanisms of searches themselves, or formatting processes such as de-duping. The result of these searches is what undergoes further manipulation by the analysis process. There's a distinction between searching and/or reporting versus analytics.

Here are the kinds of decisions or statements you can infer using analytics: many of them involve a comparison against a timeline, a policy, or even a belief.

"This series of events should never have happened within this application."

"This user is providing input too quickly; we think this is automated."

"It's four in the morning in that country, not business hours. Why are we getting traffic from them?"

"It's physically impossible for this user to have logged in from two locations 500 miles away within the space of ten minutes. Something's going on."

"We're not going to put more money into this technology until we see security incidents that cost us at least 50% of our current budget." (I'm not pretending that this makes a lot of sense, but let's go with it.)

Before you can start with analytics, you need to start with a model. What questions do you want to answer, and how will you know when you've gotten an answer? What will you consider to be sufficient accuracy or precision in the answer (these are not the same thing)? From there, you can look at the data you have available, and see whether that data can address your requirements. You also need to think about how you will use that data to get to an answer, whether it's manual analysis, automated, or a combination of both. The industry is full of patent-holding mathematicians and data scientists who have come up with ways of automating analysis that had to be done by people before; this is especially important as the volume of available data goes up and the need for speed increases.

So when you're evaluating an "analytics" product, think about what questions it's assuming you have, and see how it answers them. Even more importantly, make sure it's flexible enough to be able to address new questions as they come along. When used right, analytics can help you make better security decisions.

Wendy Nather is Research Director of the Enterprise Security Practice at the independent analyst firm 451 Research. You can find her on Twitter as @451wendy.