Google Uses Reputation To Detect Malicious Downloads

Google researchers have combined a number of reputation techniques to create a system that is 99 percent successful in detecting and blocking malicious executables downloaded by users of its Chrome browser.

The system, known as Content-Agnostic Malware Protection (CAMP), triages up to 70 percent of executable files on a user's system, sending attributes of the remaining files that are not known to be benign or malicious to an online service for analysis, according to a paper (PDF) presented at the Network and Distributed System Security Symposium (NDSS) in February.

While the system uses a blacklist and whitelist on the user's computer to initially detect known good or bad files, the CAMP service uses a number of other characteristics, including the download URL, the Internet address of the server providing the download, the referrer URL, and any certificates attached to the download.

"CAMP bridges the gap between blacklists and whitelists by augmenting both approaches with a reputation system that is applied to unknown content," the researchers wrote in the paper, adding: "One of CAMP's important properties is to minimize the impact on user privacy while still providing protection."

The approach should improve the security of Google Chrome users because it's interfering with one of the primary ways that cybercriminals attempt to infect systems, says Lance James, chief scientist of threat-intelligence firm Vigilant.

"It is sort of using the kill-chain model: We know the bad guys will do this and this and this, and you try to detect that," he says.

Google's own real-world test -- deploying the system to 200 million Chrome users over six months -- found that CAMP could detect 98.6 percent of malware flagged by a virtual-machine-based analysis platform. In addition, it detected some 5 million malicious files every month that had escaped detection by other solutions. The researchers were not available for comment on the paper by publication time.

[Nonmalicious insiders add a lot of risk when IT gives them too much access and not enough education. See Overprivileged, Well-Meaning, And Dangerous.]

In many ways, CAMP is an answer to Microsoft's SmartScreen, a technology that Microsoft built into its Internet Explorer and the latest version of its operating system, Windows 8. SmartScreen is largely responsible for Internet Explorer 8's and 9's superior performance in blocking malicious downloads in tests run by security consultancy NSS Labs in 2011. Yet SmartScreen has worried some privacy-conscious users because it sends characteristics of every file it evaluates to Microsoft's servers.

While Microsoft did not comment directly on Google's research, the company did argue that it's necessary to send data back to its service to evaluate downloaded files.

"In order to deliver file reputation, information about the files is sent to our reputation services," the company said in a statement sent to Dark Reading. "This feature has been extremely successful in helping users make better trust decisions and helping protect their privacy by helping to prevent inadvertent installation of malware."

Unlike Microsoft's solution, CAMP attempts to detect locally whether any downloaded file is malicious, before passing characteristics of the file to its server-based analysis system. First, the system checks the binary against a blacklist -- in this case, Google's Safe Browsing API. If that check doesn't returns a positive result, and if the file has the potential to be malicious, CAMP will check a whitelist to see whether the binary is a known good file.

Only after those two checks fail does the local client extract features from the downloaded file and pass that fingerprint of the file to CAMP's server infrastructure. The researchers found that the Web browser contacts the CAMP service in only about 30 percent of cases, which enhances privacy, they argue in their paper.

"User privacy is an important goal for CAMP," the researchers stated. "Verifying the content type of the file and that it neither matches blacklists nor whitelists drastically limits the number of downloads for which a remote server is contacted."

The CAMP service renders a reputation -- benign, malicious, or unknown -- for a file based on the information provided by the client and reputation data measure during certain time windows, including daily, weekly, and quarterly measurements. Information about the download URL, the Internet address of the download server, any referrer information, the size and hash value of the download, and any certificates used to sign the file are sent to Google to calculate a reputation score.

CAMP's 99-percent success rate trounced four antivirus products, which individually detected at most only 25 percent of the malicious files and collectively detected about 40 percent, the researchers stated. URL classification services -- such as McAfee's SiteAdvisor, Symantec's Safe Web, and Google's own Safe Browsing -- fared even worse, detecting at most only 11 percent of the URLs from which malicious files were downloaded.

The Google researchers who authored the paper -- including Moheeb Abu Rajab and Niels Provos -- decided to focus on executables downloaded by the user, not on malicious files that attempted to exploit a user's system. This choice will likely limit the applicability of the technology, Vigilant's James says.

"They are only dealing with certain variables," he says. "They are not discussing exploits. If there is an exploit, Google Chrome might not even know that it is downloading a binary," and so an attacker could bypass the system.

In addition, the relevance of the research may be limited to consumers and small businesses. While the results are impressive, most companies should not allow employees to download and run executables, says Anup Ghosh, CEO and founder of endpoint-protection firm Invincea.

"I would use the blacklist and the whitelist and be done with it," Ghosh says. "If it's not on either of those lists; it is in the unknown case, and as an enterprise user, I should not be running those."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message.