Running the IR Gauntlet

3:19 PM -- Ameritrade. What more can I say? They serve as such a great -- or scary -- example for so many aspects of incident response (IR) that it’s actually hard not to talk about their recent troubles as I offer a look at IR processes. (See Lawsuit Raises Questions on TD Ameritrade Breach.)

Imagine that you’re an Ameritrade IT worker trying to figure out why your employer’s customers are getting hammered with pump-and-dump spam. Let’s pretend that you don’t have any fancy IDS/IPS, network behavior anomaly detection system, data leakage, or database extrusion system that might tip you off. Where would you start?

Depending on the number of computers in your organization, it could quickly become overwhelming. First, take a look at what systems have access to the customer records that were targeted. Now, walk through each one looking for abnormalities.

Are there any rogue processes, network connections, or listening ports? Is the antivirus software, host intrusion prevention system, or firewall turned off? Do the system logs reveal anything interesting, or have they been deleted?

Windows and Linux systems include many default tools (e.g. Netstat, Tasklist, ps, and lsof) that can help answer these questions. But can you trust them? The executables from your system may have been replaced with malicious versions by the attacker, so you get out your trusty CD (maybe Helix) or USB drive with your known-good tools.

But what if the attacker has installed a rootkit that intercepts all your system calls and lies to your tools? Then you can break out your rootkit detection tools, like Rootkit Hunter and chkrootkit for Linux or Rootkit Revealer and BlackLight for Windows, but the unfortunate truth is that these vary in effectiveness and aren’t always kept up-to-date regularly by the developers.

Disheartened with the IR landscape yet? While the picture I've described here looks bleak, it isn’t always that bad. But you must be ready for the worst case at all times. I personally think that’s why IR is so exciting. I never know what I’ll be up against each time I start a new case. It all depends largely on the technical skills and motivations of the attacker.

-- John H. Sawyer is a security geek on the IT Security Team at the University of Florida. He enjoys taking long war walks on the beach and riding pwnies. When he's not fighting flaming, malware-infested machines or performing autopsies on blitzed boxes, he can usually be found hanging with his family, bouncing a baby on one knee and balancing a laptop on the other. Special to Dark Reading