4 Steps For Trimming Patch Management Time

Special to Dark Reading

The time it takes hackers to exploit a vulnerability continues to shrink. While there's not a lot you can do to speed up the time it takes vendors to develop and distribute a patch, there are reasonable and prudent measures that can reduce the amount of time it takes to deploy it. Yet expediting the patch process remains one of the key factors in mitigating risk and remediating vulnerabilities.

How much time can -- or should -- you trim from the process, and what are the benefits and risks of speeding up patch deployment? Depending on the nature of systems, resources, and configurations, the answer to that question will vary from organization to organization, and often among data centers and operational groups within organizations.

There are steps you can take to whittle away at patch deployment time and without introducing the risks -- some of them potentially catastrophic, such as system or network downtime -- of deploying patches too quickly. Testing every patch, for instance, takes time, but undoing the unwanted effects of an untested patch -- a router patch, for instance, that crashes your router -- takes far more time. Make sure, as well, that the vendor has tested the patch that's being released: A few years ago, Microsoft scrambled to patch an Internet Explorer bug that fixed one vulnerability but opened up another one. Keep an eye on the news as you're keeping an eye on your test beds: Flawed patches tend to get attention quickly.

Following are four simple steps that can save time while increasing the likelihood of effective and untroubled patch deployment.

1. Level the patching field.

Not all systems or configurations are created equal, with some of the most critical bringing their own time requirements and timing factors into the patch process. Scheduling a patch deployment for the PCs in your accounting and bookkeeping departments is less disruptive than finding a convenient time to patch the Exchange Server that your entire business depends on to generate the money the bookkeepers are tracking.

Having a solid and ongoing sense of your organization's systems and configurations, most critical and less-than-critical apps, and, above all, the level of risk that vulnerabilities carry is perhaps the most important step you can take to prioritize your patching process, moving the most likely exploited vulnerabilities to the head of the patch list, which buys you time in the scheduling of less-critical patches. A critical Windows server patch, for instance, would rank as a high-priority patch over patching an application used by just one department.

"An effective patch priority strategy follows the lines of disaster recovery and business continuity planning," says Ranny Grubb, CTO at Virtual IT. "Patch prioritizing requires applying the most effort toward the applications that your business is most reliant on for continued operation."

Grubb suggests that patch scheduling and timing match the level of risk the vulnerable application poses. "Be aware of where your most significant exposure lies, which is most frequently on the perimeter, where your systems meet the Internet," he says. "Internal applications, which are not accessed, externally are important, but not as time-critical in terms of patching as those exposed to the larger world."

Time-saver: Develop a patch priority list based on business criticality: Your business continuity/disaster recovery plan is a good starting place for establishing a hierarchy of patch deployments that will see the most critical exposures patched first, with lower risk or lower exposure vulnerabilities patched on a less fast-paced (and, ironically, less time-consuming) schedule.

2. Know which systems impose their own patch schedule.

Not all systems or configurations are created equal, either, with some of the most critical bringing their own time requirements and timing factors into the patch process. Does deploying a patch require a reboot of affected systems? Does patch deployment and vulnerability remediation call for taking down a critical server or other component?

By knowing in advance that server A, for example, is down for regular maintenance for a certain amount of time every week (or other period), you also know in advance the likeliest and most efficient time to deploy a necessary patch.

This information itself can exert advantageous ripple effects throughout your overall patch process and efforts to reduce patch deployment time. In "The Laws of Vulnerabilities 2.0," Qualys CTO Wolfgang Kandek notes that, "Remediation may not be possible on all systems, such as those with criticality or uptime constraints. Consequently, IT administrators should segment their systems into fast patch and slow patch pools and review the possibility of using mitigating technologies for protecting the slower patch pools. The existence of a fast patch pool has the additional benefit of gathering additional in-house experience related to the application of the patch and its potential side effects."

In other words, while you're waiting to patch one system, use your resources to patch others.

Time-saver: Maintain a list of critical systems' regular maintenance and planned downtime schedules, and plan patch deployment accordingly, dealing with other more readily available systems in the meantime. Review and update system maintenance schedules (and their effect on other schedules) on a regular basis.

3. Know who needs to know and who signs off.

Approval paths can be time drains down which patch promptness can all too easily drown. Because even the most efficient patch deployments can cause downtime, it's important both to make technical issues clear to nontechnical staff who must sign off on the patch, and to ensure IT understands the business implications of the patch. Communicating clearly in advance that you need to add extra time to the email server's scheduled maintenance, for instance, takes less time than explaining during the downtime why it's taking so long.

"Preapproval of patch deployment authorities is not only an important aspect of an organization's emergency plan," says Andrew Bosch, product manager at Symantec, "[but] it can save vital time and anxiety in critical situations."

With the right people identified in advance, especially in relation to critical systems, "You're not going to be chasing people in the middle of the night to find out when you can patch a critical server," Bosch says.

This one should be among the simplest and most straightforward steps, but bear in mind it involves people, authority, and almost undoubtedly politics, each of which could add to the time required to get sign-off on the sign-off list.

Time-saver: Create and maintain a comprehensive patch deployment approval and sign-off path along with your systems inventory, including emergency and off-hour contact information for all personnel on the list.

4. Take time to test patches before going operational.

Even the most critical and time-sensitive patches, such as those that seal holes that are being actively and aggressively exploited in the wild, must be thoroughly tested before deployment. In fact, those critical and often rushed-to-distribution patches could require more testing before you unleash them on your systems.

So it's crucial that your test platforms and procedures related to them be established and in place constantly, not just assembled on an ad-hoc basis the day of a patch release.

Symantec's Bosch recommends building one day of test time into your planned deployment schedules.

Pay attention as well to patch-testing with new and emergent technologies your organization is using, and with an eye toward easily overlooked things. Don't forget virtualization software and virtual machines, for example.

The initial time investment required to include and establish all systems in your ongoing test platform, and especially the time required to add new systems and applications as soon as they are introduced, will be offset by the time savings achieved by your familiarity with those systems when patches for them are released.

"Patch testing shouldn't be an adventure every time," Bosch says.

Time-saver: Establish comprehensive patch test platforms, including platforms for new technologies and configurations ahead of time, and make their maintenance, readiness, and upgrades an ongoing part of your operations overhead and budget. Build a day of patch-test time into your patch deployment schedule.

Meanwhile, patches, both scheduled and emergency, require planning and time allocation.

You must address in advance the time required to prioritize, schedule, authorize, and test both the patches themselves and your organization's ability to manage patch deployment -- in both technical and business operations contexts. That's the only way to ensure a relatively painless patch process.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.