DIY Botnet Detection: Techniques and Challenges

Botnets have been around for over two decades, and with the rise of the Internet of Things (IoT), they have spread further to devices no one imagined they would: routers, mobile devices, and even toasters.

Some botnets are legions of bot-soldiers waiting for a command to attack a target server, generally to overwhelm the server with a distributed denial-of-service (DDoS) attack. Other botnets target specific devices by stealing passwords or mining cryptocurrency. Cryptocurrency mining, in particular, has been a dramatically growing threat for organizations recently, with botnets such as Coinhive and CryptoLoot enabling cybercriminals to make as much as $100 million a year at the expense of victims' computing power. Smominru, among the largest cryptocurrency-mining botnets, has infected over half a million machines using the infamous EternalBlue exploit leaked from the NSA.

To prevent botnet infections, organizations must be able to detect them. But botnet detection isn't easy. Let's explore some of the top techniques and challenges in botnet detection.

Methods for Botnet Detection
So, what's a botnet? Simply put, it's a cluster of bots — compromised computers and devices — that perform commands given by the botnet owner. Usually, the botnet owner will dedicate a command and control server (C2), a compromised server for communicating with the bots, usually via Internet Relay Chat commands. The botnet owner uses the C2 server to order botnets to execute attacks, whether that's DDoS attacks, data theft, identity theft, or another type of attack. Thus, the smoking gun that points to a botnet is its C2 server.

Unfortunately, finding the C2 isn't usually a simple task. Many botnet commands emerge from multiple servers or take hidden forms, masking the malicious commands as harmless activity such as Tor network traffic, social media traffic, traffic between peer-to-peer services, or domain-generation algorithms. Further complicating matters, the commands are often very subtle, making it difficult to detect any anomalies.

One method for attempting to detect C2s is breaking down and analyzing the malware code. Organizations can try to disassemble the compiled code, from which they can sometimes identify the root source of the botnet's commands. However, since botnet creators and administrators increasingly are using integrated encryption, this technique is less and less effective.

Generally, C2 detection requires visibility into the communication between a C2 server and its bots, but only security solutions that specifically protect C2 servers will have this kind of visibility. A more common approach for detecting botnets is tracking and analyzing the attacks themselves — into which standard security solutions provide visibility — and determining which attacks originated from botnets.

When looking at exploit attempts, there are a few possible indications for a botnet. For example, if the same IP addresses attack the same sites, at the same time, using the same payloads and attack patterns, there's a good chance they're part of a botnet. This is especially true if many IPs and sites are involved. One prominent example is a DDoS attempt by a botnet on a web service.

False Positives
The likelihood of false positives makes botnet detection particularly difficult. Some payloads are widely used, raising the probability of a randomly occurring pattern triggering a false positive. Additionally, attackers can change their IP addresses by using a virtual private network or a proxy, making it look like many attackers or bots are involved when there's really only one.

Hacking tools and vulnerability scanners also behave similarly enough to botnets to often return false positives. This is because hacking tools generate the same payloads and attack patterns, and many hackers use them, regardless of the color of their hat. And, if different players happen to conduct a penetration test on the same sites at the same time, it may look like a botnet attack.

Organizations can often identify false positives by Googling the payload and referencing any documented information around it. Another technique involves simply gleaning any information readily available within the raw request in the security solution. For example, if a vulnerability scanner is to blame, most security solutions will reveal that by identifying it, especially if it's one of the more common vulnerability scanners.

False positives are an unavoidable challenge in botnet detection given the enormous amount of potential incidents; recent research shows that 27% of IT professionals receive over 1 million security alerts every day, while 55% receive more than 10,000. But with the right techniques and diligence, organizations can discern the harmless traffic from the malicious, botnet-driven traffic.

Related Content:

Join Dark Reading LIVE for two cybersecurity summits at Interop 2019. Learn from the industry's most knowledgeable IT security experts. Check out the Interop agenda here.