Five Ways To Get Rational About Risk

Don't Gut It Out

It's no secret that some companies excel at information security while others run around stamping out fires and never get ahead. What's the secret of first-rate IT risk managers? To find out, we interviewed a half-dozen CISOs from various industries. We didn't talk about specific types of threats. Rather, we wanted to understand exactly how these successful security leaders prioritize money and manpower.

One thing we all have in common is data overload. Infosec leaders have their go-to sources for cyberintelligence, like vendors, newsgroups, the National Institute of Standards and Technology, and regulatory bodies. But at some point, we all find ourselves overwhelmed. Call it the "needle in a needle stack" conundrum: You know there's a ton of threats out there, many of them potentially damaging to your company, but if you pick the wrong needle at the wrong time, the stack may just fall and cause death by a thousand cuts.

And there are plenty of reasons a CISO might select the wrong needle.

We'll admit that the media doesn't always help. Early last year, my phone was ringing off the hook after the Google attacks, with information security pros asking whether APT--advanced persistent threat--was the most immediate danger to their companies. Now, data loss prevention and distributed denial of service are back in the spotlight, courtesy of WikiLeaks.

We're also all over the map with risk assessments. Every company we reviewed had some type of risk management framework, but the devil is in the details. We saw no uniform best practices. Our recent InformationWeek Analytics IT Risk Management Survey, available later this month, shows that the most popular way to measure risk, by far, is qualitative categorization of high, medium, or low. In our experience, some companies have rudimentary internal risk assessment systems, supplemented by an external vendor or third party, such as Gartner. On the other end of the spectrum are companies that deploy extensive, 50-plus-question surveys and use a stringent, quantitative approach where every response has a weight; the overall tally denotes a project's risk. Fewer than half of our survey respondents, all of whom play roles in assessing risk at their companies, use such a quantitative method.

What's interesting is that the CISOs we spoke with agree that neither a quantitative nor a qualitative approach is much help with prioritization. Quantitative risk analysis is not the be-all and end-all--just because a risk is scored at 98 out of 100 doesn't mean it will be remediated. For one thing, the business significantly influences whether to spend money. And most surprising to us, in the end, many CISOs say they ignore vendor input, media reports, pundit white papers, even all their own data and make gut decisions.

Let's be clear: Gut decisions aren't useful. Very often they're based on a confirmation bias--the tendency for people to favor information that confirms their preconceptions or hypotheses, regardless of whether the information is true. If you have a confirmation bias and think laptop theft is the largest concern, whether it is or not, you'll find a way to get encryption to be the highest-priority project.

Beyond Gut Instinct

Five Ways to Get Rational About Risk

Become an InformationWeek Analytics subscriber and get our full report on reducing security data overload. This report includes 13 pages of action-oriented analysis. What you'll find:

Get This And All Our Reports

Avoiding confirmation bias is difficult. The first step is to realize that we're all prone to it. If you have a tendency to collect a lot of information and then ignore it, or you always find yourself debating the rest of the organization on which threats are most imminent, you may be more susceptible than most. Try this exercise: Ask your peers whether they think you frequently make decisions based on instinct or intuition. And don't ignore the feedback--it's almost impossible to build trust with an information source, such as your risk assessment team, if you have this tendency.

In our experience, organizations with the best risk reduction processes don't follow any one person's gut or arguments. Their CISOs aren't so afraid to talk to peers and co-workers. They don't buy into the "always greener on the other side" argument, assuming rival organizations' risk management initiatives are more mature because those CISOs have bigger budgets, more resources, or better executive support. They take ownership.

And, most important, they aren't afraid to share decision-making power. In fact, our experience and research show that the most successful CISOs are collaborative and likely to bring various teams to the table. That's critical, because our InformationWeek Analytics 2011 Outlook Survey shows that demand for IT services is on the rise. Your ability to assess the risks associated with all these new programs better keep up.

Alter Egos

Most people see infosec pros as crusaders, fearlessly taking on evil malware distributors, fraudsters, and organized crime. But the truth is closer to this: We aren't so much superheroes as people with good intentions and way too much on our plates. As foes go, an ever-growing pile of threat notices and "expert" advice is almost as dangerous as some identity theft syndicate in Belarus. Sure, we manage through it and make decisions every day that affect our organizations' well-being. But wouldn't we do better with a set of best practices that could provide clarity in the midst of a crisis and help us budget for the right technology products?

Yeah, we thought so. Here are our five best practices to get there.

>> 1. Business leaders own the data, so make sure they take seats at the table.

Read: Even if you need to drag them there. "The business has to come first, and threats must be analyzed with business impact in mind," says the CISO of a $1.7 billion community bank who works very hard to ensure that business leaders fully comprehend the information at hand.

"Collaboration is essential to prioritization, as collaboration creates an atmosphere for questioning why we are doing things," he says. "There are no dumb questions." It's the job of every CISO to translate threats, impacts, and probabilities honestly and deliver that data clearly to executive management. However, most collaboration efforts by CISOs we work with are anything but clear. Rather, they tend to involve a lengthy PowerPoint presentation, a smattering of industry data, and a forced--if not wholly fabricated--ROI.

Stop with the insanity already. Executives don't care about industry data; their business is always different or better or stronger, no matter how similar it is to that company across town that just got breached. Don't tell the CEO that "98% of financial firms are subject to XYZ." She'll immediately assume you're in the rarefied 2% exempt from that mess. Instead, use scenarios that you have actually piloted internally. Involving business leaders in your discovery enhances their connection to the risk, the threat, and most important the results. So back away from the Twitter feed and start interacting directly with your colleagues.

>> 2. Pick your sources carefully.

Confirmation bias doesn't stop at which threat to prioritize but often extends to which vendor or threat information source to listen to. Everyone wants to be the go-to resource, but consider motivation. If a CISO has a good relationship with a sales rep, for example, that vendor may get most of the security team's attention. In fact, vendors are, by far, the No. 1 source of trusted information among the CISOs we spoke with--a situation that, frankly, amazes us given all the top-notch security communities and conferences and the fact that partnering with the wrong vendor can waste a lot of money and time. A few CISOs we spoke with told horror stories of ripping and replacing gear from vendors that provided bad information. This does nothing to increase the CFO's view of IT's business savvy.

When we probed deeper about trusted information sources, we found that security engineers often talk about successes and failures with security engineers from other companies, but CISOs and directors of security don't share well. Fortunately, this trend seems to be changing, as many of the CISOs we spoke with had dedicated time and travel budget to participate in a peer group in 2011.

Robert Allen, CISO at CNA Insurance, recently participated in a CISO summit in Chicago. While he notes that conferences are never perfect, Allen was able to gather ideas that will shape his security plans.

>> 3. Embrace transparency.

5 Fast Tips To Effectively Assess Risk

If you use an outside vendor for security risk assessments, consider switching consultants every other year for a fresh perspective. Ask senior, nontechnical executives what keeps them awake at night. Their responses to this simple query are often eye-opening. To assess the risk posed by employees, consider sending a mock phishing e-mail to see who bites. Again, you may be surprised. Be discrete, but be clear about policies. Align assessed risk to the value of data. Sounds basic, but it doesn't always happen. Get your terminology in line with how the business thinks. The language of risk management isn't native to IT; the Factor Analysis of Information Risk (FAIR) framework of interconnected models is a great resource for CISOs looking to get up to speed.

"Awareness builds trust, and trust allows us to have an open and honest conversation with the business so the threats that are urgent and important are addressed," Allen says. His advice: Focus on the present--not on what could be but on what is. Continually measure and monitor your risk profile and the current threat level to reduce the chance of making a gut decision.

Of course, the only way to reduce threat noise is to filter it. GRC (governance, risk, and compliance) tools from companies such as Archer and Relational Security can help risk assessment and mitigation based on your reality, and thus can also support transparency, but don't get caught in the weeds of managing such a system. The data is what's important, not the GRC product's bells and whistles.

>> 4. Always test new threats to figure out if you have compensating controls.

Once you identify a threat, evaluate the real-world risk to your organization. Don't just listen to the risk assessment team and accept the probability and impact they suggest. Use Metasploit, or run the virus in a VM and see what happens. You may be surprised that the effect on your network is much different from what the vendor, analyst, or article says it will be. Got an emergency, where your team is running around in a panic? That's the best time to stop and think. We've seen countless instances where a team forgot or ignored compensating controls when assessing a risk during a perceived crisis.

>> 5. Don't wait for a crisis to set goals.

The best way to prioritize is to be proactive. However, it's human nature to dither when we have no time pressures. Adding to the malaise for CISOs is the fear we'll make a bad decision. So instead, we end up with paralysis by analysis. Should we buy DLP or IPS? Focus on hardening our servers or on policies and documenting processes? Every CISO we talked with had a war story of wasting resources on a technology that didn't help with the problem at hand. Many times, business leaders forced them to choose product X because of price; other times, they admit they got sold on it by vendor Y. Risk management and resource prioritization aren't sexy. But listening to your gut just isn't effective. There are too many variables, and attackers are too wily, numerous, and adaptive.

Test. Put processes in place. Accept transparency because it will increase collaboration with business leaders, which will then enable better prioritization of risks. Only once risks are identified and ranked should you look to external resources. It's your only hope to avoid security information overload.

InformationWeek: Mon. dd, 2011 Issue

Download a free PDF of Dark Reading January Digital Issue
(registration required)