How Windows 'Crash Dumps' Aid Defenders

When Microsoft embarked on its Trustworthy Computing Initiative almost 12 years ago, a primary goal was to use better development practices and tools to help secure the Windows operating system and reduce vulnerabilities in its products.

The Window Error Reporting (WER) tool, also known as Dr. Watson, has been a major force for finding and fixing the software bugs and vulnerabilities responsible for the largest proportion of crashes, with the company estimating that 1 percent of flaws are responsible for half of all crashes.

Recent news that the National Security Agency has used WER to provide feedback on its efforts to compromise targeted computers and organizations may cause information-technology managers to be wary of the tool, but Dr. Watson remains a technology that benefits defenders immeasurably, security experts say.

"The very basic use that Microsoft does with these crash dumps is defensive," says Wolfgang Kandek, chief technology officer for cloud security firm Qualys. "Just having the mechanism is already helping defense a lot."

In a 2009 paper (PDF), Microsoft studied nearly a decade of Windows error reports and found that a bug reported by WER is five times more likely to be fixed than a bug reported by a human. During the beta program leading up to Windows Vista, for example, the WER was responsible for helping developers eliminate 5,000 errors.

The WER system is a process run by Microsoft's operating system to collect information about the application and system state following a crash. In a three-stage process, WER connects to Microsoft's reporting server and sends basic information about the software involved in the crash. This information is not encrypted, according to security consultancy Leviathan Security, which has studied the benefits of mining information from crash dumps.

In the second and third stages, the server then requests more information, and WER forwards a so-called minidump file and more detailed data to Microsoft. The report-collecting server can also request the entire contents of memory from the crashed application, according to Microsoft's paper.

[Treasure trove of tools created and used by NSA hackers for planting backdoors via Cisco, Juniper, Apple products unveiled in latest document leaks. See NSA Elite Hacking Team Operations Exposed.]

Yet companies could hijack the error reporting tool for their own purposes. By changing registry keys for the werfault.exe file, a company can enable and disable Windows Error Reporting, have the crash dumps sent to Microsoft or a private corporate server, disable the user interface and dialogue warnings, and send more or less data on the crash, according to Leviathan. Over time, Microsoft has included more details in its error reports, and with Windows 8, encrypts the data with TLS. The initial barebones communication with the reporting server is not encrypted.

A large company that collects all of the data could have a better picture of potential software problems on their network. An application that continues to crash across many computers could indicate a problem, says Chris Eng, vice president of research for application-assessment firm Veracode.

Yet Microsoft's statistical approach to tackling bugs would be hard for most companies to duplicate, he says. "My challenge as an enterprise is, how do I take a tool or a technology like that and turn it into a system that can inform my defenses?" he says.

Leviathan has developed a way to determine whether a crash has been caused by an attacker's attempt to exploit the system. Under a grant from the Defense Advanced Research Projects Agency's Cyber Insider (CINDER) initiative, Leviathan built a system for using Windows Error Reports to detect the crashes caused by attackers' exploitation of application flaws. Dubbed Major Myer, the system collects Windows Error reports, analyzes them, and determines whether the crash was caused by an attacker trying to exploit the software or from a nonsecurity error.

The system has had great success, successfully detecting attacks approximately 80 percent of the time, says Frank Heidt, CEO of Leviathan.

"In [the] face of a heterogeneous environment and in the face of countermeasures, attacks fail more often than they succeed," Heidt says. "We are just taking those axioms and using them against adversaries."

The approach has already proved to be effective in using WER to eliminate security issues. In 2007, for example, Microsoft identified an attack against its customers by malicious software known as Renos, which infected systems and caused cyclical crashes, resulting in 1.2 million error reports submitted daily. When Microsoft released a Windows Defender signature for the malware, the number of reports quickly dropped.

Leviathan's system is already being tested by a number of unnamed customers as well.

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.