3 Inconvenient Truths About Big Data In Security Analysis

Big data analytics tools and distributed databases may offer a lot of potential to change the way security monitoring and investigation is done. But these innovative ways of concentrating security data stores and accelerating analysis bring with them some unwanted baggage.

Not only are these tools and services harder to bake into processes than vendors market them, but they're also introducing a lot of risk to security organizations that use them uncritically, warned security researcher H.D. Moore, chief research officer at Rapid7, at the UNITED Security Conference earlier this week in Boston.

According to Moore, big data holds the potential to "start changing the mathematics for both attackers and defenders." However, organizations beware because there are some pretty inconvenient truths about big data that Moore laid out in his conference session at the show.

1. Big Data Isn't Magic
According to Moore, the buzz around the big data trend is so pitched that the term is being used interchangeably with all types of security analysis tools. And it's almost used as a security open-sesame incantation that will offer the holy grail of security: instantly clear visibility.

"People say if you have all of your data in one place, you'll magically get the security benefit. That's not true," he says. "It's a ton of data -- you can dig into it, and you can find stuff. You can obviously find some good security benefit to having this data. But it doesn’t come for free."

Without someone dedicated to the process of going through and writing the right queries and generally asking the right security questions, the benefits will be minimal. "So just be careful about where you invest, and make sure that if you are investing in a data analytics tool, you at least have one body sitting in front of it and you're investing just as much in people as you are in the process," he says.

2. Putting All Our Eggs In One Rickety Basket
Even more distressing to Moore, though, is the insecure infrastructure backing many big data security analysis tools -- homegrown or otherwise.

"We see a lot of stuff in development around big data toolkits -- things like Mongo and Cassandra -- and there's not a lot of security built into these tools," he says. For example, MongoDB doesn't support SSL by default, and there isn't the same level of security offered in similar tools as more established traditional relational databases. "It's actually pretty frightening how insecure these tools are by default, yet they're becoming the back-end for most of the big data services being sold today."

Meanwhile, organizations are consolidating their risks into these systems by aggregating huge stores of security metadata, log files, and more in order to do large-scale analysis.

"Organizations are doing whatever they can to get all of their data in these central locations," he says. "You're making these really juicy targets for someone to go after. Everyone kind of cringes when we look at some of those big password breaches in the past, but that's nothing compared to a multiterabyte data leak."

Not only are organizations putting their sensitive security data in one giant basket, but it is a rickety one at that.

3. Law of Averages Says An Analytics Provider Breach Is Coming
In many cases, the basket isn't tended in-house, either. As more big data security analytics service providers come into play, organizations are increasing their risk profile if they aren't vetting their providers, Moore warns.

"The amount of data these folks are processing and the type of data they're processing is really important," Moore says. "You're seeing everything from SIP phone call logs -- who talked to who, to when users logged on -- to sensitive information being sent in the log files themselves."

Moore believes that with the greater proliferation of service providers, the insecurity in many of the products they're using and the growing list of important customer data held by these providers make a big breach inevitable, and very soon.

"One thing that's almost guaranteed to happen in the next year is we're going to see one of the large providers of analytics services -- whether security, log data, or something else -- get breached," he says. "It's just the law of averages at this point. There's enough folks offering services who don’t necessarily know what they're doing that we're going to see a big breach."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.