Next-Generation Malware: Changing The Game In Security's Operations Center

In a quiet, secluded spot, a malware author is creating a new piece of code that no antivirus tool has ever seen before. It's not a particularly creative exploit -- just a slight tweak on an existing Trojan -- but it should be enough to bypass the signature-based defenses of the company he's targeting.

Your company.

In other cases, there's no human author involved -- the malware is being created by an automated program that continually tweaks known attacks in new ways, so that they won't be recognized by antivirus or intrusion prevention systems (IPS). Researchers estimate that, across the Internet, an average of 70,000 to 100,000 new malware samples are created and distributed each day, often through automated, "polymorphic" programs that automatically alter malware into a new, previously unseen form factor each time it is delivered.

Today's malware is becoming both more prolific and more sophisticated -- and the problem is growing more acute every day.

"The bad guys are generating their attacks programmatically," says Roger Thompson, chief emerging threat researcher at ICSA Labs, who has been studying malware for more than 20 years. "For years, enterprises have been using signature scanners as their primary means of defense. But that's no longer enough anymore."

Most enterprises still rely heavily on antivirus technology as their primary means of defense against malware. AV systems work by identifying malware through a blacklist -- a database of known viruses, Trojans, and other malicious code -- and blocking and eradicating any code that's on the list. The premise of AV technology is that it is possible to identify the unique characteristics of any known malware -- its "signature" -- and use that signature to prevent it from penetrating the enterprise.

But with new "zero-day" malware being created each day -- each minute -- AV systems often cannot keep up, and their blacklists have become bloated and slow to perform. This growing problem has spurred many vendors -- and many enterprises -- to begin looking for ways to recognize malware not by how it looks -- its known signature -- but by how it behaves.

"The interesting thing about malware is that while there are millions of instances, there are really only a few types of behavior that it exhibits, and they are very different from the behavior that a legitimate program would display," says Dennis Pollutro, president and founder of Taasera, a stealth-mode startup vendor that is planning to roll out a next-generation malware defense technology later this year. "If [an unknown application] tries to access certain functions, or if it tries to install or replace an existing program, for example, then you know it's malware. You can identify it by what it does, even if it has never been seen before."

Thompson agrees. "The underlying behavior of all malware is essentially the same," he says. "The bad guys today only have to generate new code to beat the existing base of signature-based defenses. But soon, if they also have to beat 20 or 25 behavior-based products -- plus tools for whitelisting -- they're going to find the going is much harder."

While most security experts agree that signature-based tools need help -- and that behavior-based tools may be an important solution -- the industry is only just beginning to wrestle with the implications of this radical shift in technology.

In the old days, enterprises relied primarily on their AV vendors to automatically detect, analyze, and eradicate malware. But with the proliferation of new zero-day malware, many enterprises now find themselves tasked with doing their own malware analysis -- correlating information from AV and signature-based tools, next-generation behavior-based tools, threat intelligence services, and their own system logs and security information and event management (SIEM) data.

In a nutshell, the process of malware analysis and defense has evolved from a "set it and forget it" task into a skills-intensive, do-it-yourself research project. And that shift is having a profound effect on the staffing and day-to-day activities of the enterprise security department.

"Malware analysis used to be something that only AV vendors did -- they were the only companies that ever hired malware analysts," says Alex Cox, a member of RSA's FirstWatch threat analysis team and a longtime malware analyst. "Today, malware analysis is a critical skill set that every business should have. Usually, these are people who are part of your incident response team -- or even a specialized malware response team -- who helps interpret the threat and feed data back to an operations team that will do something about it."

Derek Manky, senior security strategist at Fortinet and a well-known industry researcher, concurs. "We definitely see enterprises developing their own security operations centers, doing their own malware analysis, collecting their own threat intelligence," he says.

"As vendors, we're becoming more of a partner with the enterprise team, rather than a sole source," Manky says. "We're sharing threat intelligence with them, developing a working relationship that allows them to correlate all of the data they have, whether it's from their SIEM systems, log systems, or other security vendors. We have an API in place so that they can reach out to our systems and access our threat intelligence."

Doing in-house malware research and analysis can help enterprises interpret the potential risk associated with a new threat, enabling them to develop customized priorities and defenses based on their specific business requirements, Manky observes. But it also creates demands on the enterprise SOC that most have never seen before.

"Every SOC now has to do its own security event correlation, and then correlate that data with many other data sources, such as threat intelligence services or other security information sources," Manky observes. "They don't always have the skills in-house to interpret all of that data, or to determine all of the actions they might need to take."

Indeed, skilled malware analysts can be difficult to find, and the industry is having difficulty keeping up with the demand, experts say. According to statistics from (ISC)2, one of the industry's leading security professional organizations, there currently is a shortage of some 30,000 security professionals in the U.S. alone, and it is estimated that an additional 2 million security pros will be needed across the globe by the end of 2015.

"Malware analysis is one skill that's needed broadly across the industry right now," RSA's Cox says. "It's becoming a central function of the security team -- there is a lot of hiring in that space."

But some experts say the rapid proliferation of malware -- and the increased sophistication and specificity of attacks -- will soon outstrip the human-oriented capabilities of internal malware analysis teams.

"Most companies don't have the resources they need to do this sort of analysis," says Anup Ghosh, founder and CEO of Invincea, an emerging security company that advocates "compartmentalization" of software operations, essentially relegating new applications to a safe, virtualized environment that limits the potential damage that malware might cause.

Malware analysis may help companies understand the threat they face, but creating a large, in-house malware response team may be short-sighted, Ghosh suggests. "A lot of companies have developed a process and hired people to try to find threats, do damage assessment, and reverse-engineer the malware to help determine who the threat actor may be," he notes. "This is an increasingly common process to find in enterprises, but it doesn't scale over the long term. And in the end, it doesn't defend against the threat -- it only gives you a better picture."

Many enterprises have given up on the idea of trying to prevent the threat posed by new malware, and are simply expanding their incident response efforts to help clean up the inevitable infection, Ghosh says. "They are moving from trying to stop the breach toward becoming detectives who try to isolate the problem after the fact," he observes. "But I don't think giving up on the idea of prevention is the answer, either."

What the industry needs is to automate the process of malware analysis, so that the enterprise can respond at a speed that is comparable to the speed that malware is being created, without building up a huge staff, Taasera's Pollutro suggests. "They need to find a way to get ahead of the problem -- something that's more than just a cluster of applications that interpret it," he says. "And they need to find a way to apply threat intelligence and new malware information more locally, so that it can help their specific organization, rather than just collecting more generalized information."

In the meantime, however, the best strategy for stopping next-generation malware is not to rely too heavily on any one technology, Manky advises. A combination of signature-based tools, behavior-based tools, traditional perimeter defenses, and next-generation application defenses can create such a muddle of problems for attackers that can discourage them -- and send them looking for easier pickings elsewhere, he says.

But no matter how sophisticated your defenses, having someone on your team who can do malware analysis is still a good idea, Manky suggests. "A layered defense is your body armor," he states. "But even body armor has holes and places it doesn't cover -- you need to be ready to respond if something gets through."

Have a comment on this story? Please click "Add a Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.