Security Index Marks A Year Of Doing Business Dangerously

People who believe the state of cybersecurity is worse for wear have good company: For its first year, the Index of Cyber Security -- a measure of the threat to companies as viewed by their top security officers -- has risen steadily at about 2 percent per month.

The index measures the opinions of cybersecurity experts on whether various facets of security are getting better, trending worse, or staying the same. The index started at a base value of 1,000 in March 2011 and stands at 1,292 a year later. Because the questions ask the respondents to answer relative to the previous month, the index measures the relative movement of the experts' opinions on threats and is not an absolute rating of threat.

"The considered opinion of that group of people is that the risk is rising," says Dan Geer, a well-known security professional and one of the two creators of the Index of Cyber Security.

Geer and co-creator Mukul Pareek started the ICS in April 2011, using the previous month as a baseline with a score of 1,000. The pair asked a vetted group of chief security officers and security managers to rate the current trajectory of different threats and factors that contribute to cybersecurity, such as the threat from malware, the risk to Internet-facing applications, or the danger posed by third-party custodians of data. Each respondent is asked to rate each factor as falling fast, falling, static, rising, or rising fast.

The results are delivered back to the group as a monthly report with anonymized comments. The creators acknowledge that the model is simple, but that's the point. By simplifying the data in the same repeatable way, they are able to create a measure that has some meaning.

"What practitioners can take away is that what they are seeing and how it might differ from our index is informative," Geer says. "They can go to the budget side of the house, and say, 'It is not just us. Here is what hundreds of my peers say.'"

The steady rise of the index has some experts questioning whether the Index of Cyber Security is measuring the relative threat or the mindset of security practitioners.

"Things seem to be going to hell in a hand basket if you believe the ICS," says Andrew Jaquith, chief technology officer for Perimeter E-Security. Jaquith does not question the usefulness of measuring the sentiment of security professionals, but adds, "The question that I suppose you got to ask is whether security people are just pessimists, because if they are you are always inclined to think that things might be getting worse."

Yet the single threat measurement does not tell the whole story. Peering at the individual components of the index reveals some interesting data. Two factors have regularly contributed to the perceived threat: political and ideological activity and the risk due to counterparties -- hardly surprising given the rise of Anonymous over the last year and the continuing parade of hefty breaches announced.

[ Anonymous is a brand, and one that is in danger of being overwhelmed by its own chaotic nature, two security professionals argue. See Anonymous Must Evolve Or Break Down, Say Researchers. ]

"It is interesting that that is the biggest risk, not the Chinese or APT," Jaquith says.

The general trend of the underlying factors that make up the index appeared to correlate to the major stories in any given month. Significant attacks by Anonymous would make chief security officers worry more about the hacktivism threat, while media coverage of large data breaches would boost the visibility of that threat.

Geer points out that the press stories may not be the influencers in this case.

"We are not sure which way causality runs," he says. "Are the people reading the press, or is the press reading the people?"

Perhaps as interesting, the factors that least contributed to the threat rating indicate that some aspects of cybersecurity are improving, or at least not getting much worse. Operational security officers seem to believe that information-sharing between companies had improved and that their defenses were more successful against attackers. Finally, the respondents felt that they were getting more support from the government in cybersecurity matters.

The general ICS is not meant to guide chief security officers in how much they should be investing in security, but it could help them review their priorities, Perimeter's Jaquith says.

"It is the relative ranking that matters," he says. "If things change place, you may want to look at your own priorities."

The Index of Cyber Security creators plan to use the anniversary to significantly change some of the underlying components. A year after the creation of the index, Geer and Pareek have discovered a number of ways that the system could be improved. Rather than changing the index piecemeal -- and measuring the impact of the change -- the two researchers will lump all of the changes together.

"It goes back to thinking about what the index is for and what purpose does it serve," Pareek says. "The responses that don't change in any significant way month to month do not contribute to decision support."

While the changes will make data comparisons with the past year less meaningful, it could make the index a better measure of threat for the future.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.