Tech Insight: Using Penetration Tests To Gauge Real Risk

Some organizations perform risk assessments based on the impact exposure of trade secrets, customer data, or other sensitive information would have. Others do it by evaluating the risks associated with not passing compliance audits. Generally speaking, there is no one right way to do it (though there are plenty of wrong ways).

Everyone has their own ideas about how best to accomplish risk assessments. Books have been published. Guidelines are available online. Countless articles have been written. But in the end, what really matters is that the organization is clear about why it is evaluating the risks, is truthful in answering the questions asked during the risk assessment, and understands what to do with the results. Without those three things, performing a risk assessment is a useless exercise no matter which approach taken.

What about penetration tests for assessing risks? They are a different beast altogether, but they can provide a great deal of value because they are intended to provide proof as to the actual impact of a particular threat. What would happen if an attacker were to exploit a particular vulnerability or series of vulnerabilities? Instead of mental "what-if" exercises, attacks are performed to simulate what a real attacker could do.

Are these types of simulations accurate? Well, that depends on a lot of variables, such as the reputation of the company, the skills of the individuals doing the test, and any limitations set forth by the customer. Is the penetration-testing company performing realistic threat analysis and attempting to attack using the same methods as those used by the identified threats? Or are they simply following a checklist or methodology set forth by some expensive training institute?

For example, some extremely bad penetration tests are being performed for which the results are purely based on the results of a vulnerability scanner. The report the client receives is simply the output from the scanner with the pen-testing company's logo on the front. Saying a vulnerability is high-risk because a vulnerability scanner said so is not a true measurement of risk.

To provide value, the scanner results would need to be validated and true risk determined based on the target environment. A scanner may identify a "high-risk" flaw in the Apache Web server, but if it's on a host that is separate from the client network and holds no sensitive data, what's the real risk? An attacker could deface the site, embed malicious code to attack visitors, or delete all content; depending on the impact of those threats, the company should determine how to mitigate the vulnerability.

It's important to be able to differentiate between what a scanner tells you and what is more likely to happen in the real world were you to be attacked by an attacker not restricted by a contract or rules of engagement. A good pen tester will be able to show that a "high-risk" vulnerability may gain an attacker nothing, while a few "low-risk" vulnerabilities, classified as such by the vulnerability scanner, can be chained together to take over the entire internal network through an Internet-facing Web server.

Chris Gates of Lares Consulting drives this last point home with his "Low to Pwned" blog series. Be sure to check out the entire series. There are approximately a dozen posts.

In addition to charlatan pen-test reports (i.e., rebranded vulnerability scanner reports) missing the business risk a vulnerability poses to an organization, they'll also completely fail at providing realistic mitigations that fit within the client's resources and environment.

Let's look at another example. Suppose a pen tester gains access to a critical Web portal through a dictionary-based password-guessing attack. A real report's recommendations may include adding a CAPTCHA or locking out accounts after repeated login failures and requiring more complex passwords to prevent dictionary-based attacks. Additionally, there may be mention of application log monitoring to detect these types of attacks.

While the above recommendations are solid, they may not fit within the resources of the client. The client may not have developers on staff who can make the changes to the Web application to address the issues, or the application could be part of a SaaS offering in which no logs are accessible. This is where a quality pen tester could have the conversation with the client to help it understand options that fit within its resources.

At the end of the day, risk assessments and pen tests are supposed to help companies identify and evaluate risks in order to better protect themselves. Performing assessments without a willingness to be honest and understand the true purpose can leave organizations with a false sense of security and in a potentially worse situation than if they'd done nothing.

Have a comment on this story? Please click "Add a Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.