Is Defense Winning? A Look at Decades of Playing Catch-up

Defenders are perpetually playing catch-up to attackers. For every security innovation or new technology introduced, cybercriminals develop just as many tricks to bypass them. This ongoing struggle will be the focus of an upcoming presentation at Black Hat USA 2024, this August in Las Vegas, by Jason Healey, a senior research scholar at Columbia University.

"For over 50 years, we've known that the red team always gets through," Healey says. "Despite the billions of dollars spent, thousands of patents filed, and countless hours worked, defense hasn't notably improved relative to offense."

Last year's publication of the US National Cybersecurity Strategy marked a significant milestone, setting a new goal to enhance defense at the largest scale and least cost. However, Healey argues that progress means little without measurable indicators to determine whether defense is gaining relative advantages over offense.

Healey's session at Black Hat — titled "Is Defense Winning?" — will introduce several key indicators to assess whether the balance is shifting in favor of defense.

"Many of these indicators, such as changes to mean time to detect [MTTD], are already collected by the community," he says. "Others, like measuring the mean time between catastrophes, might need to be fresh."

Drawing parallels with climate change metrics, Healey says there is a need for a similar holistic approach to security as well.

"Just as climate experts track CO2 levels and temperature changes, we need macro-level indicators to understand cyberspace as a whole," he says.

Measuring Success in Cyber Defense

Healey played a role in drafting the National Cybersecurity Strategy, which incorporates the concept of defensibility and leverage. He believes systemic changes, such as automated updates, over individual actions, like user education or isolated security measures, will be more important in affecting change for defenders.

"We need to find areas where the smallest turn of the screwdriver will have the largest impact," he says.

One of the critical challenges Healey addresses is how to measure success in cyber defense. He proposes several propositions and indicators to gauge progress, including the ability of threat actors to adapt their tactics, techniques, and procedures (TTPs).

"We would want to see them having to rapidly change their TTPs because we're thwarting them," he says.

Healey also calls for the cybersecurity community to leverage existing reports, such as Verizon's annual data breach report and Google's zero-day reports, to establish defensibility metrics.

"Companies like Veracode already report relevant metrics, but they need to be presented in time series to track trends," he says.

Achieving New Indicators for Defense

Healey's ultimate goal is to inspire the cybersecurity community to strive for measurable improvements. His presentation aims to spark a crucial conversation about the effectiveness of current strategies and the importance of setting tangible goals, challenging attendees to reflect on their collective impact.

"We need to set reasonable targets, like reducing the mean time to detect and dwell time to less than 24 hours by 2030," Healey says. "Are we actually making the difference we say we want to have in the world?"

By introducing new indicators and drawing on lessons from other fields, Healey aims to equip defenders with the tools they need to shift the balance in their favor. The date and time for Healey's presentation will be published soon.