Sandbox-Aware Malware Foreshadows Potential Attacks

Sandboxes monitor behaviors to detect new strains of malware without prior knowledge of its existence. This has cemented sandboxing as a pillar of modern cybersecurity. It's also spurred a decade-long arms race with "sandbox–aware" malware that hides these behaviors. Should we expect a repeat of this arms race in the continuous monitoring space?

Many similarities exist between the observational techniques of sandboxes and continuous monitoring (CM). While sandboxes typically exist in the cloud or on central appliances, CM agents deliver this monitoring on endpoints. They hook into the operating system to observe things such as process creation, file and registry changes, and network communications. Patterns in these activities detect behaviors indicative of malware.

To predict hackers' future reaction to CM, I examined the history of sandbox-aware malware. I spoke with Christopher Kruegel, CEO of Lastline, about detecting these countermeasures. Lastline offers a platform to detect advanced persistent threats (APTs), zero-day exploits, and evasive malware. Our conversation guided my research for this article.

Countermeasures to Detect Sandbox Environments
If malware engages in malicious behavior inside the sandbox, the game is up. Hackers know this. Thus, crimeware may perform "environment scans" to determine if a sandbox is present.

Most sandboxes consist of virtual machines pre-installed with instrumentation to observe behaviors. Sandbox-aware malware may scan the file system, study OS services, or examine open ports. It may also look for DLLs or registry keys indicating a virtual environment. In recent years, Trojan.APT.BaneChan has even looked for signs a human user is present. Sandbox vendors have responded by threat scoring a potential sample if it performs these environmental scans. One could easily see this same cat and mouse with malware looking for CM agents.

Some companies have designed their architectures to rise above this arms race. Instead of using a virtual machine, Lastline built an emulator to intercept CPU instructions. As CEO Christopher Kruegel explains, "[emulation] looks at everything the program does. Not just when it calls the operating system, but all the parts between. Examining when it processes data, makes decisions, and when it goes through the instructions of the program."

While emulation is helpful, no architecture is foolproof. Researchers have suggested countermeasures that detect "emulation gaps" by including obscure machine instructions not supported by emulators. These calls would fail and thus signal the presence of emulation. Here begins another arms race to insert exotic CPU commands into malware, and for vendors to judge if they're countermeasures.

Efforts to Suppress Malicious Behavior
Sandboxes need to observe malware behavior over time. The first volleys in this battle were time-based attacks. Typically these attacks sleep or perform only benign activities until the examination times out. CM agents never stop observing, so here they have an advantage over sandboxes.

Behavioral observation is implemented by hooking into system calls. Cyber weapons may remove hooks or operate before the hook and not allow the sandbox's code to execute and record activity. These strategies could work against a CM agent's similar hooking strategy.

Coaxing out suppressed behaviors can be difficult. Western intelligence agencies built Stuxnet to target Iranian reactors. Unless Stuxnet sees certain control system components, it exhibits only benign behavior. Continuous monitoring, existing on the targeted endpoint, has an advantage here.

This is also where the emulated sandbox approach shines. If only 5% of the code runs inside Stuxnet, Lastline's emulator can force the CPU to execute additional code branches. After all of the behaviors manifest, the malware can finally be threat scored.

Overt Acts of Subversion
While a crashed sandbox could pique the interest of an analyst, hackers hope it forces them to give up on analyzing a sample. Target-built malware can purposely crash when run where it doesn't belong. It can call libraries designed to crash sandboxes not equipped with capabilities such as 3-D modeling. Crashing endpoints where CM agents exist could be an option but disables the endpoints a hacker is using to advance their breach.

For CM vendors to remain relevant long term, they'll need to match the vigor of sandbox vendors against targeted subversion. Security architects should prepare by employing a defense-in-depth approach. This assumes any detection technology could fail, and layers many of them. Use sandboxes and CM, but don't forget to pick up the forensic residue of malicious behaviors on endpoints. Forensic artifacts are tough to hide, and deleting them leaves alarming evidence of "anti-forensics."

Related Content:

Learn from the industry’s most knowledgeable CISOs and IT security experts in a setting that is conducive to interaction and conversation. Click for more info and to register.