It's Time to Stop Measuring Security in Absolutes

COMMENTARY

The context and metrics guiding risk assessments are constantly changing, and so is our understanding of what progress looks like as a security team. It's not possible to measure everything, and just because you can measure it doesn't mean it's important. This makes it easy to get lost in the details and miss the bigger picture: Are we directionally improving?

A big part of the problem is the standard security policy, which aims for perfection while losing sight of achievable goals. In our industry we have policies that say, for example, "all high-risk vulnerabilities must be addressed within 10 days," or "all user access must be reviewed quarterly." The assumption is that you will strive for 100%, with no conversation about whether this is achievable and what resources would be required to hit that goal.

Usually, a security team will hit that goal 70% of the time, which is deemed a failure. A team often spends an outsized number of resources trying to close the gap, for example, by addressing that 70% of critical vulnerabilities and the policy's goal of 100%. They may end up straining resources to aim for perfection when those resources could be better spent elsewhere.

As an industry, we need to take a step back and reevaluate the policies and metrics steering our programs, deciding whether they're realistic and whether they are even the right measurements. Here are three steps to take to achieve this.

1. Determine Your Risk Appetite

It's impossible to achieve perfection in all areas of risk. Security teams can end up playing whack-a-mole and lose focus on more subtle risks. There needs to be a business-level conversation to define where the organization's biggest security risks lie and where to devote resources, as well as areas in which its executives are comfortable with a certain level of risk. A critical vulnerability like MOVEit, for instance, could represent an acceptable risk in one area of a business, but not in another area that has tier one systems with zero to minimal allowance for an impact on the CIA triad of confidentiality, integrity, and availability. Look at where the biggest vulnerabilities lie within your industry and the types of attacks that commonly target businesses in your space to perform a risk assessment.

2. Set Flexible, Achievable Goals

The next step is to set achievable security policies, based on your risk assessment, that focus on incremental progress. You can't jump from patching 50% of vulnerabilities to 95% overnight. It's important to understand the resources it will take to hit your goal and what opportunities you'll give up by aiming for total patching versus 85%. It may not be worth the investment to close those last few points.

Instead of setting a static goal and aiming for perfection, focus on improving the program relative to where you were before. The questions you should be asking are: Are we moving in the right direction? Is the program improving? Are we reducing risk overall?

3. Reassess Regularly

Since vulnerabilities and attack methods are always changing, security leaders should hold discussions regularly with the broader business to reassess risk appetite and security policies. At a minimum, this should be done annually. Reevaluate whether goals are aligned with known risks and risk tolerance, and make conscious decisions about the trade-offs.

For example, you may determine it's achievable to address 85% of critical vulnerabilities within 10 days. To get to 90%, X amount of resources, expressed in terms like monetary investment, time, or people, will be required. You may find 85% to be an acceptable level of risk when weighed against those additional resources.

Aim for Progress, Not Perfection

Decisions about risk should not be made in a vacuum. This is why security leaders must have these conversations with other business leaders and the board. The bottom line: Perfection is rarely achievable in this industry, and aiming for that absolute can do more harm than good. Instead, focus on making progress. Set realistic goals, take small steps to get there, and keep raising the bar until you've reached the optimal level of risk mitigation.