First on the Scene

4:30 PM -- It isn't always easy to tell at first whether a security incident is something as simple as a virus infection, or as complex as the insider attack at UBS PaineWebber. (See Witness: Angry PaineWebber Defendant Warned “God Only Knows What I Could Do”.) IT needs to be trained to deal with incidents appropriately -- to understand how to use the appropriate tools and how to interact with the system to preserve potential evidence.

The help desk and other IT workers on the front lines need to be trained as "first responders" just like police officers, who almost always arrive on the scene of an accident before the paramedics. In the six phases of incident response --preparation, identification, containment, eradication, remediation, and lessons learned -- identifying IT personnel who are likely to be the first on the scene and training them is part of the preparation phase.

The primary issue I see with incident response in organizations of any size is the lack of first responder training. This can lead to IT workers running antivirus software and similar tools (modifying filesystem timestamps), which can deem the evidence unusable in a forensic investigation. The various statutory and industry regulations that require notification to customers and employees who have their information exposed in a data breach make these first steps even more crucial.

Phase two of incident response: First responders must be capable of determining if an incident really has occurred. Freeware tools such as those available from Microsoft Sysinternals are invaluable when conducting incident response on Windows systems, for example. With Linux-based systems, having statically precompiled binaries of system tools like ls, lsof, ps, and netstat are a must. (I’ll be discussing incident response tools in more depth in an upcoming blog post.)

If you don’t have a formal incident response plan in place, you'd better get one quick. Then identify and train your first responders on how to identify and respond to incidents. The last thing you want is to get stuck doing hundreds, thousands, or even millions of customer notifications due to a botched investigation.

— John H. Sawyer is a security geek on the IT Security Team at the University of Florida. He enjoys taking long war walks on the beach and riding pwnies. When he's not fighting flaming, malware-infested machines or performing autopsies on blitzed boxes, he can usually be found hanging with his family, bouncing a baby on one knee and balancing a laptop on the other. Special to Dark Reading