Taming Big Bad Data For Better Security

Delivering better intelligence to security professionals is all about grabbing more and getting less.

To better detect the problems in their environments, companies need to find all the components of their networks and tap more sources for security, network, and enterprise data. Yet doing so quickly overwhelms security analysts and managers with an abundance of information, much of it having little to do with security. Instead, companies need analysis platforms that can reduce the amount of information to actionable intelligence.

Companies that fail to collect data from parts of their networks or that cannot cull the information from the noise frequently miss security issues or incidents, says Paul Stamp, director of product marketing of RSA.

"Even simple threats can hide in plain site because there is so much noise out there," he says.

In the past month, a number of companies -- RSA, IBM, and RedSeal Networks, to name a few -- have announced products or updates aimed at helping companies hone in on the information that matters. At the heart of the products is the concept of "big data," the buzz of last year's RSA Conference 2012. While it began as a description of the massive amounts of consumer transaction data that online retailers had to mine in order to gain intelligence on their businesses, security firms now talk about big data as a way to gain a better handle on the threats and problems in their networks.

"The security problem always was a big data problem," says Mike Lloyd, chief technology officer for RedSeal Network, a network and security intelligence firm. "Security has always been about finding the needle in the mountain of needles that can hurt you."

To find the right needles, security firms have to create better analysis systems and solve four problems: missing data, a poor ability to identify important information, a scarcity of knowledgeable security people, and difficulty in dealing with change in the network.

Getting a complete picture is difficult because every company has dark space in their network, RedSeal's Lloyd says. In some cases, companies have devices that are not on their mapped networks. In other cases, devices are missing from where a business believes them to be. Most often, firms suffer from both issues. In fact, about 18 percent of corporate networks are considered dark space, according to RedSeal data.

For companies, then, the first step is finding missing systems -- and missing data -- in their networks, Lloyd says. "What we found is that all security teams have dark space in terms of big data," he says. "If you don't have all your data, you can't do good big-data analytics."

[As enterprises bring threat intelligence feeds into risk management equations, they could gain a greater fluidity in risk-based decision-making. See Threat Intelligence Brings Dynamic Decisions To Risk Management.]

Yet more completely collecting event data causes a second problem: The products for combining logs and other event data overwhelm information-security workers with too many alerts. Without better filtering, correlation, and analysis, security professionals become desensitized to the influx of data and miss the real important events.

While companies once collected data from security data, now they are adding network data, endpoint data, and even business data that could indicate anomalous activity. Businesses are looking for better awareness of what transpires in their networks because online attackers have become more stealthy and devious, says Marc van Zadelhoff, vice president of strategy and product management for IBM. Attackers that infiltrate networks and compromise systems for long periods -- the advanced persistent threat, or APT -- are hard to detect unless a business is aware of what's going on in its network.

"The variety of data that you have to pay attention to has gotten a lot more complicated because of the APT problem, especially," he says.

Add to that the trend toward transient devices on the network -- such as employees' personal mobile devices -- and telecommuting employees connecting their home networks or laptop systems to the corporate network. Companies have to be better able to deal with a network constantly in flux, says Scott Crawford, managing research director of Enterprise Management Associates, an analyst and consulting firm.

"We need to analyze across these different types of data to see things in a richer context," Crawford says.

Finally, by adding expert analysis to the products, companies can mitigate the shortfall in experienced security analysts. Collecting and analyzing large amounts of network and security data is a difficult problem, even for a data scientist, says RSA's Stamp. And with the current shortfall in qualified security professionals, building as much expertise into the products as possible is important.

"The number of people that are both data scientists and threat experts are miniscule," he says. "That is where the vendors come in because we have to act as that intersection."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.