Hunting Botnets In The Cloud

Comparing botnet command-and-control (C&C) traffic or malware within an organization to activity seen in other parts of the Internet isn't new. It's just that some security analysts are increasingly going there to gather better intelligence that they can use to quell an infection or help take down a botnet.

"The approach of using large bodies of data to identify botnets or malware, in general, has been going on for a long time. Now it's starting to become so widespread that startups are being galvanized by it ... making attention [be] paid to it," says Al Huger, vice president of development for the cloud technology group at Sourcefire and a co-founder of Immunet.

A group of researchers from Northeastern University, Symantec Research Labs, Eurecom, and UC Santa Barbara recently built a prototype system for detecting botnets on a large scale and for finding previously unknown botnet C&C servers. The so-called Disclosure tool uses the NetFlow protocol as well as custom features to spot botnet markers and to differentiate between C&C traffic and legitimate network traffic.

The breakthrough of the tool is that it spots botnet activity over the Internet as a whole, rather than just within an organization, the researchers say. And it ultimately can provide botnet protection "of the Internet at large," says William Robertson, assistant professor at the College of Computer and Information Science at Northeastern University, one of the developers of Disclosure. It's also a big-data type of tool that can process large amounts of data quickly, and can also spot previously unknown botnet servers operating out there, he says.

Some security vendors are expanding their botnet investigation into more cloud-based models: Seculert, for example, last month rolled out Seculert Sense, a cloud-based analysis engine that analyzes on-premise logs from an organization with its cloud-based botnet intelligence data. "Using the cloud as a technology enabler helps Seculert to better detect botnets and APTs, and therefore protect our customers," says Aviv Raff, co-founder and CTO at Seculert. "Only a cloud-based solution is capable of digesting a huge amount of data over a long enough period of time at an affordable cost in order to detect such persistent attacks."

So when Seculert detects a botnet infection in one organization, it can then spot the same attack on its other customers. "This is 'crowdsourcing' in order to battle the botnet and APT problem," Raff says. Seculert first spotted the Shamoon targeted attacks against Middle Eastern oil organizations, he says, with early versions of Seculert Sense.

At the heart of this cloud-based botnet-fighting model is "big data." And Seculert uses the Hadoop-based Amazon Elastic Map Reduce service in its offering. "It basically allows us to analyze huge amount of data using statistical analysis and machine-learning methodologies that consume large amount of CPU and large amount of storage for the logs," Raff says. "Therefore, we are able to see the bigger picture of the problem."

Incident response company Mandiant, meanwhile, recently quietly acquired Unveillance, a cloud-based botnet intelligence firm, and last month rolled out a new subscription cloud-based threat detection serviced based on Unveillance. "With its acquisition of Unveillance and its cloud-based botnet threat intelligence product, Mandiant can tell the enterprise whether it has any compromised hosts talking back to a criminal C2 infrastructure," writes Wendy Nather of The 451 Group.

[As companies try to make sense of a greater amount of information on their networks, anomaly detection becomes more difficult but more important as well. See Security Intelligence Starts With Detecting The Weird.]

It's crucial to have both an inside look at how a botnet has infected a particular organization, as well as external data on the larger operations and spread of the botnet, security experts say.

"Often it becomes remarkably simple to identify botnets, but getting your hands on good data is the challenge," Sourcefire's Huger says. "If you want to identify large-scale botnets, you need to get your hands on data that identifies them across multiple ISPs or millions of endpoints. Very few organizations are in a position to get their hands on that reliably and consistently."

That requires the ability to analyze botnet data from local and cloud-based sources in real time. "We collect actual big data amounts of information from" endpoints, he says, but that information in isolation is limited in value. "Seeing that endpoint go to a website ... and correlate that [behavior] with 30 other systems going there" in real time, you can get a better picture of the activity, he says.

Part of the problem of gathering good big data is competition among vendors that are hunting the botnets, he says. "The security industry doesn't generally play well together" when it comes to botnet information, for example, he says. "There are commercial competitors vying for customers."

It's not like in the antivirus sector, where malware sample-sharing is routine practice. Getting useful, global views of botnet activity can be difficult, he says. "You have to take large sets of data with seemingly innocuous data and marry them to come to broader conclusions."

Another challenge to beating botnets and APTs via the cloud: The bad guys are plenty organized and often better at sharing intelligence than the security industry, he says.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.