How Incident Response Fails In Industrial Control System Networks

Worries of eventual cyberattacks on utilities as well as chemical and other industrial sites have intensified in the wake of the recent attacks that led to a power blackout in western Ukraine. But the key element in mitigating the damage from a cyberattack -- an incident response plan -- is something that many industrial sites just don't have in place.

Conventional incident response procedures don't neatly map to the ICS/SCADA environment, either. Responding to an attack on an industrial control system (ICS) comes with challenges the pure IT environment typically does not face, and some of the standard IR steps just don't translate to a power plant or manufacturing plant, according to Chris Sistrunk, senior ICS security consultant for Mandiant/FireEye.

Industrial sites under NERC/CIP (North American Electric Reliability Corporation's Critical Infrastructure Protection) and Chemical Facility Anti-Terrorism Standards (CFATS) regulations have IR plans, he notes, but there are still many other ICS/SCADA organizations that do not fall under those regs and lack IR plans, including some in the water, manufacturing, and oil & gas industries.

Uptime and availability -- think electricity and other disruption-averse services -- are king in the ICS/SCADA space, as is physical (life and limb) security. That's why operators of those networks rarely apply vendor patch updates for security bugs and other issues: if a patch could potentially disturb an existing system configuration, or require any downtime or disruption, it's not likely to be installed.

"If a critical system has a virus and it hasn't actually affected the system, they may not do anything about it," for example, Sistrunk says. But if it's an engineer's laptop, it most likely will get the standard cleanup in IR, he says.

A programmable logic controller (PLC) cannot be re-imaged the same way a laptop can be, he says. An industrial system typically doesn't collect logs of events like a conventional computer does, either, and if it does, the logs may not feed to a SIEM, he points out. "Some devices don't have syslog or other types of logs," Sistrunk says.

He advocates network monitoring here, using Netflow packet capture, for example. If industrial networks aren't monitoring for the latest threats, they may not know for sure if they've been hit by a Black Energy or Havex malware attack, for example.

Network security monitoring could be set to detect known indicators of compromise for a specific attack. Sistrunk and colleague Rob Caldwell last year at the S4x15 ICS/SCADA industry conference demonstrated a set of open source network security monitoring tools from the open source Security Onion Linux suite, including Wireshark, NetworkMiner, Bro, and Snorby.

Ralph Langner, founder of Langner Communications and a renowned Stuxnet expert, says many industrial firms don't have a firm grasp on their network and system environment. "They don't have specific insight into digital dependencies and network connections," he says. And a solid IR plan requires knowing your environment and how best to contain a threat or infection there, he says.

The stakes are even higher when there's a physical consequence in an attack, he says. "You now have to coordinate with process engineers. You cannot simply reboot a control system. It's going to affect the process and might make things worse with forensics efforts.

"The plant manager will not let you mess around with all of those digital systems and reboot," Langner says. "Incident response is very difficult and very challenging in a cyber-physical environment. Nobody has procedures" for it, he says.

Langner says the key is training plant and industrial site operators on how to thwart cyberattacks. 

The good news is that there are some IR resources for the industrial space. The National Institute of Standards and Technology's SP800 Series document drills down on control systems, for example. Sistrunk says those guidelines are a good jumping-off point for unregulated organizations.

Eric Knapp, chief technical advisor for Honeywell's Industrial Cybersecurity Center, says the forensics information that investigators need from an attack on an ICS/SCADA system typically isn't available to them. "Control system software logging data is not what [they] are interested in," he says. "If you don't understand the control process system, the information has no meaning. Can you make logs that security people care about?"

In most IR engagements in industrial networks today, the victims aren't able to determine where the attacks came from nor how to protect themselves again in the future. "They don't have a well-documented way to keep track of incidents that happen," such as where, when, and how an infection came into the network, says FireEye/Mandiant's Sistrunk, who says he's heard this time and again from forensics investigators.

"We know the vulns are there and the exploits are there. Now the malware is eventually going to come," says Sistrunk, who in June blogged about IR challenges to ICS/SCADA.

If an industrial network operator suspects a breach, know who to call, he says. "The specialized IR team will need to include management, legal, engineers, technicians, safety specialists, vendors, and ICS security experts during IR activity," he wrote.

According to Sistrunk, the team's responsibilities include:

·         "Determining what to do about the threat and when

·         Restoring the ICS as safely and quickly as possible, without destroying forensic evidence

·         Reviewing any available logs and other forensics data that will be critical in piecing together what happened

·         Determining the root cause of the incident and providing recommendations for improving the ICS security to help prevent future incidents"

Related Content: