Red Curtain Reveals Malware

4:08 PM -- Earlier this month, Mandiant, an incident response management services and solutions provider, released a free tool to assist incident response teams with identifying malware. It's a tool worth looking at. (See Mandiant Offers Free Software.)

Red Curtain, previously codenamed Caprica Six, examines files looking for anomalies that might indicate a malicious intent. In a world where antivirus software is confounded by exploits such as Storm -- which can repack itself every few minutes -- a tool such as Red Curtain is definitely welcome. (See Tool IDs Hidden Malware.)

One technique that malware authors use to evade antivirus products is using packers and crypters to compress and/or encrypt their malware. Since antivirus products primarily rely on signatures to detect malware, the simple act of packing or encrypting a file can prevent it from being detected.

When performing incident response, it is not uncommon to come upon unknown files that aren't detected by the latest virus signatures. At that point, how do you know if a file is good or bad? You could rule the suspicious file in a virtual machine and monitor its behavior, but some malware is designed to detect virtual environments and act differently to mask its true function. This is why Mandiant released Red Curtain.

Red Curtain scans files looking for characteristics that might indicate a packer or crypter was used, and then produces an overall score based on those characteristics. One of the more interesting things it searches for is entropy -- a measure of randomness which tends to be higher in compressed and encrypted files. While it is not a foolproof measurement -- users can compress and encrypt their own data -- but it is a very good indicator if you're dealing with an executable that is currently running with open ports on your system.

I've always been a packrat, and over the years, I've amassed a pretty good collection of suspicious files from the students (and family members) whose machines I've helped clean. Using Red Curtain, I scanned about 2,500 files to see what happened. Not every file was malicious, but most were -- they generally related to some sort of virus infection or compromise.

Almost all of the files I expected to score highly in Red Curtain did. The scoring is based on a 0.000 to 10.000 scale. Scores 0.7-0.9 are "somewhat interesting," 0.9 to 1.0 are "very interesting," and anything over 1.0 is "highly interesting." About 90 percent of the files I scanned scored over 0.8.

A tool like Red Curtain helps raise awareness that antivirus software can't detect everything. Hopefully, that's not a revelation to most security pros. But I can't tell you how many times I've heard, "Oh yeah, it's clean. I ran XYZ product." Red Curtain is a great tool to add to your incident response arsenal, and you can't beat the price.

— John H. Sawyer is a security geek on the IT Security Team at the University of Florida. He enjoys taking long war walks on the beach and riding pwnies. When he's not fighting flaming, malware-infested machines or performing autopsies on blitzed boxes, he can usually be found hanging with his family, bouncing a baby on one knee and balancing a laptop on the other. Special to Dark Reading