The Folly of Vulnerability & Patch Management for ICS Networks

IT security has depended on vulnerability and patch management for decades, and conventional wisdom says these programs should be replicated to make industrial networks more secure. We only partially agree.

Vulnerability and patch management programs have only modestly improved overall security for traditional IT networks. We're not throwing the baby out with the bathwater here. There is no doubt that well-planned vulnerability/patch programs — ones that prioritize what to patch based on a clear understanding of risks and that conduct proper testing — can demonstrably reduce risk. While improvements in automation across the vulnerability/patch life cycle have reduced costs and the resources required, opportunity costs remain.

Within the industrial control system (ICS) networks that underpin everything from oil/gas and water systems to manufacturing and the electric grid, many issues combine to make traditional vulnerability and patch management approaches very difficult.

The first thing IT security teams run headlong into is that patch windows for ICS networks are not, and in fact cannot, be done as frequently or regularly as IT networks. On Wednesday, you're not testing and deploying any of the patches Microsoft (or any other vendor) released on Patch Tuesday.

Check out the all-star panels at the 'Understanding Cyber Attackers & Cyber Threats' event June 21 and get an in-depth look at your cyber adversaries. Click here to register. 

Plants need to keep running, and plant executives prioritize uptime over patching something that "may cause an issue at some point." Patch windows may come about quarterly, but in some cases it can be a year or more before the plant engineering team is willing or able to patch systems — and in some instances, other priorities eclipse even these infrequent windows. We have an oil exploration customer with vessels that only come to port every three to five years for maintenance; this is when all but the most critical patching happens.

Notably, in some cases, SCADA (supervisory control and data acquisition, a control system architecture associated most commonly with critical infrastructure environments such as energy grids) and distributed control system components are shipped by vendors with outdated or end-of-life operating systems.

Another reality is that many ICS environments are outdated because of the long asset life cycles on which these plants operate — often 20 or more years between major overhauls. This is compounded by the long design/release life cycles for the ICS software that runs these plants. In many cases, these systems can't be fixed from a vulnerability perspective. We have customers that still run ICS software on Windows XP or NT — even a perfectly patched XP system is highly vulnerable since Microsoft ended support years ago. And don't forget "zero days," which are frequent for many industrial assets.

But this issue goes deeper. Most older (and even some newer) ICS environments have little or no inherent security. Security wasn't a design criterion when many of these plants were built 20 or 30 years ago. Many ICS systems have no user authentication and require no validation for code or commands downloaded to controllers that monitor and direct the operation of the physical systems — valves, actuators, pumps, robots, etc. These systems use insecure, unencrypted protocols and often operate over flat networks.

ICS vendors are making real progress, baking security into new systems, and engineering firms are beginning to prioritize cybersecurity in the design of new plants. But most plants today have what we call "M&M security" — soft on the inside and (sometimes) harder on the outside. The result? Attackers must still gain a foothold on the ICS network to inflict damage, but once they are there, they will often have nearly unfettered access to the systems that run the plant — whether patched or not.

With this in mind, a "magic wand" thought exercise is helpful. If we handed you a magic wand and you could wave it at your plant and have a perfect list of all the assets in your ICS network, and a complete list of all the known vulnerabilities, what would you do next? What steps would you take that would have the most definitive risk reduction impact on the environment?

There is definitely a need for vulnerability and patch management. Vulnerabilities, especially those as high-risk as the one that enabled the WannaCry ransomware to propagate so rapidly last month, clearly demonstrate that if patching is at all possible, then it's needed. But rather than attempting to patch everything that can be patched, we advocate a risk-adjusted approach in which you patch the most important pathways into the ICS network. Critical vulnerabilities on anything inside the enterprise network perimeter with connections back to IT networks are a high priority (for example, the VPN that provides contractors and employees remote access to the plant for maintenance purposes). Next, work on other less critical but still important vulnerabilities on the same pathways.

Instead of looping until done, we highly recommend spending time finding and fixing configuration vulnerabilities on your ICS network. In a perfect world, ICS owners/operators would implement all the recommendations of the IEC-62443 standard and cover the basics of cybersecurity hygiene. Unfortunately, the reality is that they cannot afford the downtime and all too often lack resources to do these basic steps. A practical approach is to look at overall risk posture improvements that any element can provide, and prioritize which measures and in what proportion can drive the biggest bang for the buck — considering both cost and resource constraints.

We simply don't have a decade to markedly improve the security for ICS networks. When implementing a cybersecurity program for ICS networks, you must consider opportunity costs across competing program elements. Overfocusing on an element such as vulnerability and patch management can prove to be costly in the short and longer run.

Related Content: