Unstructured Data: The Threat You Cannot See

Every day, IT security teams are inundated with data — security events, network flows, configuration information, and so on — which then must be collected and analyzed for potential vulnerabilities. Your team probably has a solid, established approach or even a documented strategy for doing this. If so, great. But is that enough?

The data collected by most security tools, such as firewalls and antivirus software, is structured — that is, organized in an easily searchable, relational database. Structured data, however, amounts to only a small portion of a larger, more complicated puzzle. It's the remaining unstructured data that security teams struggle most to collect, analyze, and act upon — and the amount of unstructured data only continues to increase.

Think of how much security data flows from sources you don't control, including the massive swaths of unstructured data living on the Deep Web — from blogs, forums, or bookmarking sites. This unorganized, often text-heavy data accounts for a majority of the Internet's data. IDG believes unstructured data is growing at the rate of 62% per year, and that by 2022, 93% of all data will be unstructured. How can IT teams keep pace? The answer could lie in cognitive security — the use of big data platforms, data mining, AI, and machine learning to analyze raw data whether structured and unstructured.

But first, let's examine the problem.

Why It Matters
Understanding the magnitude of this issue requires examining the foundation of current security measures. Traditional security focuses on mitigating external threats — perimeter defenses to ward off the bad guys. As such, we often focus our security strategies on firewalls, antivirus software, and secure passwords.

Security innovation has almost always had this perimeter philosophy at its core. However, a myopic focus on perimeter protection severely limits the overall security strategy, potentially rendering it ineffective without complementary, proactive measures in place.

Consider the average IT organization's reaction to the hundreds of thousands of daily security events. The process for today's security teams involves analyzing data from antivirus software and firewalls, and then correlating that data to create a story, which in turn helps inform a solution.

In the process, security professionals are left with mountains of events to manually analyze and execute. Meanwhile, when they're busy responding to old threats, new threats continue to arise undetected. Consequently, the entire team finds itself fighting fires instead of solving or preventing problems. That doesn't leave much bandwidth for data aggregation and analysis.

Unstructured, Untold, Unknown
Next, let's think about how we, as IT professionals, share and consume security information, particularly during a major crisis. The current norm for security professionals is to update websites and social channels to explain how they've addressed a particular security issue and simply hope it reaches all relevant and necessary parties. Take, for example, this year's WannaCry attack.

The first real solution offered to organizations affected by WannaCry was explained via Twitter, by a user known as MalwareTech. Although certainly helpful, social is by no means a perfect means of circulating widely sought, urgent information to security teams around the world. Merely posting online assumes that in the middle of a major crisis, frantically busy security professionals are manually scouring the Internet for the information you're providing — something few people have time for in calmer times, let alone when the proverbial sky is falling.

Information sharing is critical to IT security — not only within individual organizations, but in the security industry as a whole. We rely on one another to share information about new and known threats, and often benefit from each other's knowledge and experience. Unfortunately, the majority of information generated and shared by security professionals about breaches, threats, malware, etc., is unstructured, and thus much more difficult to unearth and apply in real time, particularly during critical security events that require immediate action.

How much time is lost and how much damage done, simply because we lack access to or awareness of viable solutions provided by our industry peers? Or because we lack a strategy for gathering and analyzing the flood of unstructured data at our disposal? This is where cognitive security offers vital, immediate benefits.  

Welcome to the Cognitive World
A cognitive approach uses AI, data mining, and machine learning technologies to parse through thousands of security feeds and data sources — including the low-key, often invisible world of white- (and black-) hat bloggers and discussion forums — to aggregate and analyze unstructured and structured security data. Meanwhile, a security professional works to perform predictive data analysis, ultimately training the system on best practices, organizational policies, and more.

Over time, the system begins to learn on its own, including how to prioritize events and recommend responses. While cognitive security cannot replace existing security tools — antivirus software, for instance, or intrusion prevention systems — the data generated can be plugged into traditional perimeter defenses. As a result, IT pros gain a better understanding of their data's meaning and how to convert insights into action.

Beyond the Perimeter
Unstructured data will only continue to proliferate. It's time to get ahead of it so that security teams can better locate analyze and respond to threats. That requires thinking beyond the perimeter and embracing security technologies that will bolster traditional defenses and provide a more proactive, intelligent security strategy. 

Related Content:

Join Dark Reading LIVE for two days of practical cyber defense discussions. Learn from the industry’s most knowledgeable IT security experts. Check out the INsecurity agenda here.