Open Source Metrics On Tap For Security Patch Management

Security consulting firm Securosis is spearheading a new effort to create metrics to quantify the cost and efficiency of an organization's security patching process.

Rich Mogull, founder of Securosis, says to date there's no real way to accurately measure the cost and productivity of an organization's security patch management process. "Those fully quantified [IT] risk models don't apply and the numbers aren't accurate," he says. "It's also bothered me to see those uber-metrics approaches that get an overview of everything in the security program. So why not start with one thing we can accurately measure and use it as a core for building security metrics?"

Securosis, with the financial backing of Microsoft for the initial phase of the project, will gather input in an open submission process for the so-called Project Quant metrics model. Version 1 is planned for release by the end of June.

Many organizations don't have actual processes for out-of-cycle security patches, and end up in "panic mode" trying to apply them, Mogull says. Some don't even have processes for the scheduled patching their Oracle software, for instance, he says.

"We know there are tremendous inefficiencies in how [organizations] approach patching," Mogull says. "We're going to solicit [organizations] out there and find out different ways people are doing this and find a way to quantify this."

Jeff Jones, a director in Microsoft's Trustworthy Computing Group who first approached Securosis about the project, says the goal is to offer metrics that are consumable for business decision-makers. "Vuln counts and data risk [data] is cool for my tech people, but we would really like to see firms doing some analysis and getting results that are more appropriate for the business level," Jones says.

The metrics model will cover everything in the patch management process, from monitoring software for updates to installing the patches. It will analyze things like the amount of time it takes to test patches and roll them out, for instance, and on how many systems, etc., Mogull says. "As compared with other things in security, you can get a reasonably accurate [accounting] of costs" here, he says. "We're going to come out with something that helps IT professionals get their jobs done better and that they can present to the business guys."

The creation of the model will be an open and transparent process, Mogull says. Any input from Microsoft's Jones, for instance, would be posted on the project Website just like any other submission, he says.

And Jones is hoping other vendors will help with the project. "I'm hoping we can drive participation from other people in the industry, like patch management tools companies and other vendors," he says.

Project Quant will be a spreadsheet-type model, where organizations can plug in numbers and rate their efficiencies and costs. Although version 1 won't include benchmarks, the final product will. Among the elements it will include: defining patch management roles and phases; measuring the actual cost of patching in an organization; and providing organizations a way to see how efficient their process is or is not, so they can improve it.

Microsoft's Jones says he hopes to gather information from some key Microsoft customers about their patch management processes and document their experiences as part of the project. "We need to have options of how the [metrics] might apply to different [size businesses] -- from the small business to the biggest enterprise," he says.

The metrics model will be released under a Creative Commons license, and spreadsheets will be available in both Excel and open formats.

"The vision is for this to [also] become an element that rolls into ... bigger models for server or systems management," Microsoft's Jones says.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.