IBM Corrects Unpatched Vulnerability Numbers After Google Challenge

Google is no longer the software vendor with the highest percentage of unpatched critical and high-risk bugs as of the first half of this year: IBM now ranks first.

After Google and another as-yet unnamed software vendor challenged IBM's findings that showed Google with the most unpatched critical and high vulnerability disclosures, IBM revised this data set found in its IBM X-Force 2010 Mid-Year Trend and Risk Report, which it released this week.

IBM didn't name names, but said two software vendors had contacted the company about the severity and unpatched vulnerabilities data included in the X-Force report. "As a consequence of this feedback, we have manually reassessed the CVSS scoring, remedy information, and vendor information for every vulnerability that impacted the percentages that appear in this chart," blogged Tom Cross, manager of XForce Research, over the weekend.

In the original version of the report, Google was shown with 33 percent of its critical and high-risk (and disclosed) vulnerabilities unpatched, followed by IBM, with 29 percent, Oracle, with 22 percent. But with IBM's revisions, Google's numbers are now zero percent: no unpatched vulnerabilities nor any critical or high-risk ones in the first half of the year. And Sun went from 24 percent of unpatched bugs of all severity to 8 percent, and from 9 percent unpatched critical and high-risk ones to zero percent. IBM now shows the most critical and high-risk bugs without a patch.

Microsoft also fared better in the corrected data, with 7 percent unpatched critical and high-risk vulnerabilities versus 11 percent in the older version. Its percentage of bugs of all levels of severity stayed the same at 23 percent.

Google spoke out today about the changes to the IBM report. "We questioned a number of surprising findings concerning Google's vulnerability rate and response record, and after discussions with IBM, we discovered a number of errors that had important implications for the report's conclusions. IBM worked together with us and promptly issued a correction to address the inaccuracies," blogged Adam Mein, a member of the Google security team, who noted that Google's security response team prioritizes the handling of bug reports.

Mein says Google was "confused" about IBM's data showing that 33 percent of critical and high-risk bugs discovered in Google software had not been fixed. "We learned after investigating that the 33% figure referred to a single unpatched vulnerability out of a total of three -- and importantly, the one item that was considered unpatched was only mistakenly considered a security vulnerability due to a terminology mix-up. As a result, the true unpatched rate for these high-risk bugs is 0 out of 2, or 0%."

Both Google and IBM pointed to the lack of standards for reporting vulnerabilities as a major contributing factor to vulnerability database discrepancies. "Efforts are currently underway within the software industry to develop standards for reporting of vulnerability and remedy information," IBM's Cross blogged. "We believe that those standardization efforts hold the key to making sure that consumers always have the latest information from software vendors about vulnerability disclosures affecting their products."

Google's Mein said there must be more regular collaboration between organizations that compile vulnerability databases and software vendors. He pointed to inconsistencies in the way vendors disclose bugs and employ different classifications for severity. "This makes the process of measuring the number of total vulnerabilities assigned to a given vendor much more difficult," he blogged.

"As a first step, database compilers should reach out to vendors they plan to cover in order to devise a sustainable solution for both parties that will allow for a more consistent flow of information. Another big improvement would be increased transparency on the part of the compilers — for example, the inclusion of more hard data, the methodology behind the data gathering, and caveat language acknowledging the limitations of the presented data," Mein said in his post. "We hope to see these common research practices employed more broadly to increase the quality and usefulness of vulnerability trend reports."

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Dark Reading's editors directly, send us a message.