Diverse White Hat Community Leads To Diverse Vuln Disclosures

A larger, more diverse workforce of white hat web vulnerability researchers is a good thing, researchers at Penn State University have found. Growth in the community leads to more bug reports for more websites, and discovery of a wider variety of vulnerability types -- beyond just SQL injection and cross-site scripting.

Therefore, while applauding and rewarding the most prolific bug bounty-hunters continues to be a wonderful thing, the researchers recommend that the security industry should do more to attract newbies to vulnerability disclosure programs.

Assistant professor Jens Grossklags, post-doc Kai Chen, and doctoral student Mingyi Zhao from Penn State's College of Information Sciences and Technology, made these conclusions after analyzing 3.5 years of activity on Wooyun, the premier bug bounty platform in China. During this time span, 3,254 researchers disclosed a total of 16,446 vulnerabilities through Wooyun. Researchers are paid in prestige, not yuan, but Wooyun is beginning to offer monetary rewards for important discoveries and some companies give gifts to researchers for disclosing important vulnerabilities in their websites. 

"Wooyun allows for a deeper analysis [than other disclosure programs and bug bounties], since the details reported for each report are comparatively substantial," says Grossklags. "Most other forums either lack detailed information about the reported vulnerabilities, or detailed information about the track record of the submitter."

"Wooyun is the only full disclosure program. That's why we choose to study Wooyun first," says Zhao. "And based on a preliminary analysis of HackerOne's public data, we see very similar patterns in both data sets. So in general, I would say results obtained from analyzing Wooyun's data are relevant to other platforms as well."

The researchers wanted to better understand the machinations of the bug bounty market, and see how to enhance its overall productivity.

They discovered that as the number of active users on Wooyun grew, the nature of the reported vulnerabilities changed.

For example, the number of high-severity bugs reported began to exceed low-severity ones, and that gap continues to expand. Also, bug bounty-hunters -- facing bigger competition -- set their sights on a wider variety of targets. Reports of weaknesses on low-traffic websites started to outnumber those on the most popular websites.

"High-value websites benefit from a better state of security of less trafficked websites," says Grossklags. "For example, stolen information from such less popular websites, such as password datasets, can be repurposed for attacks on higher value targets."

Grossklags says they are also trying to understand "to which degree less prominent participants of the white hat community can provide similar contributions" as the "most prolific contributors." They divvy up the pool of active disclosers on Wooyun into a "head group" and a "tail group" -- each of which are responsible for roughly half of the total vulnerabilities.

The head group consists of a small number (191) of super-productive researchers who averaged 43 bug reports apiece over the time studied. Most of them specialize in hunting down SQL injection and cross-site scripting vulnerabilities. The weaknesses they unearth are on average slightly more severe than those turned up by the "tail group."

The tails are a larger group of people (3,063), who only averaged 3 vulnerability reports. This is nevertheless an important group, because they are more responsible for discovering the less common vulnerabilities -- like access privilege bypass, command execution, and logic error vulnerabilities.

Each company should be especially grateful to the person who reports the most vulnerabilities to them, sure. However, those top contributors only seem to be responsible for between 2 and 4% of the total bug reports. So, value the stars, yes, but not at the expense of the rest.

As the report states, "While rewarding top contributors may be beneficial, attracting more white hats to participate is equally helpful."

A system like Wooyun's is an interesting case to observe from the US, where seeking vulnerabilities on another person's website without explicit authorization to do so, is technically a felony under the Computer Fraud and Abuse Act. Organizations establish their own disclosure programs to specifically tell the white hat-wearing public that they're welcome to dig around, as long as they follow a few ground rules.

"There are more and more companies embracing the idea of bug bounty in the US," says Zhao. "These companies usually establish their bug bounty programs in platforms like HackerOne ... and allow white hats to test their systems. ... Definitely, Wooyun is a much more aggressive model. That is, a white hat can freely choose a target without the target's consent. I guess this can happen because relevant law in China is not mature now."

"It is certainly the case that information sharing of security information is hampered by legal uncertainty," says Grossklags. "The recent proposals of the president aim to address such sharing impediments; however, it remains to be seen whether the details of the proposal suggest that they improve the situation."