An Insider's Account of Disclosing Vulnerabilities

Vulnerability management seems vexing to organizations and tech vendors. Vulnerabilities can take months to fix. In my recent experience, it can take close to a year for a vendor to issue a patch in the first place. There is a sordid history of security researchers being threatened with lawsuits for discovering vulnerabilities, but for the most part the challenge is that vendors are noncommunicative and slow to act when vulnerabilities are discovered.

During the past two years, I've been working on Project Memoria, which discovered nearly 100 vulnerabilities in the TCP/IP stack (technology for communicating with connected devices) across multiple systems and devices

In the process of responsible disclosure, we briefed government agencies, we communicated our findings globally, and we outlined recommendations for organizations to remediate their vulnerable systems and devices. The research was hard work, but it was even more challenging to manage the disclosure process.

Typically, vulnerability disclosure involves at least three stakeholders: the researchers that discover the vulnerability, the vendor affected, and potentially an agency like Cybersecurity and Infrastructure Security Agency to help co-ordinate a response. However, supply chain vulnerabilities become even more complex as the number of stakeholders involved (the downstream vendors that have integrated vulnerable components into their own products) increases. It can become extremely challenging to assess which products are affected.

Zero Sense of Urgency
Organizations understand that time is of the essence when it comes to contracts and will optimize networks and software to shave fractions of a second off their customers' experience, but when it comes to acknowledging vulnerabilities and working to remediate them, it's anything but that.

It was just over a year ago that my team and I wrapped up a weeklong endeavor reaching out to as many as 200 potentially affected technology vendors based on the vulnerabilities we discovered. As a best practice, vendors should proactively issue an advisory as soon as possible, but it took Schneider Electric 11 months to issue an advisory.

When you consider the totality of the vendors, we engaged versus how many actually responded, it is mind boggling. Our team and collaborators reached out to 422 vendors and 341 have taken no action — that's 80%.

The Risk of Silent Patching
When it comes to patching, silence is not golden. Unfortunately, many vendors silently issue patches to fix a vulnerability without ever publishing public documentation or assigning it a CVE ID. It has always been a problem, but it's becoming bigger by the day.

We encountered an example of silent patching earlier in the spring. The specific vulnerability, CVE-2016-20009, was originally discovered by Exodus Intelligence in 2016 but was never assigned a CVE ID. We independently replicated the discovery of this vulnerability in 2020 and spent months working with CERT/CC to convince Wind River (the owners of Ipnet/VxWorks) to assign an ID to the vulnerability.

If another security research team could discover this vulnerability independently of Exodus Intelligence, then so could a malicious actor. When vendors silently patch vulnerabilities, they can leave their customers and partners vulnerable to attack because they don't know they might be affected. It also leaves us security researchers duplicating work that has already been done.

Vendor Effort Is the Exception
Security researchers are well acquainted with Newton's First Law: inertia. It can take months for a vendor to act, if ever.

In my and my colleagues' experiences, it usually took at least a week of scouting corporate websites and LinkedIn profiles to gather email addresses that were often nothing more than [email protected]. Some vendors would reach out for more information, but most vendors never reply, or they remain silent for months before acknowledging that they are affected.

Ironically, some of these companies claim to be experts in physical security because they sell surveillance systems and access badges. However, it seems they lack the fundamentals of cybersecurity. When virtually every device has an IP address, including security cameras, this should be concerning.

Transparency and Collaboration Are Key
Even when vendors do communicate vulnerabilities, some of them hide their advisories behind registration, while others make them publicly available. Some are specific and prescriptive about the vulnerability, while others remain vague. This variability in response makes it difficult for the asset owners, who ultimately must manage the risk of having vulnerable devices on their networks.

As organizations increasingly adopt Internet of Things devices they want to be confident that vulnerabilities are not putting them at risk. When it comes to security, there are no guarantees, but the manufacturers of vulnerable devices need to be more responsible for doing everything they can to harden that device's security. Their customers can and should hold them accountable.

While too many vendors stay silent or do too little, we should highlight those vendors that do respond and act quickly. These vendors have a well-established product security team that has a dedicated presence on its company website. They have readily apparent and secure communication channels, such as email and PKI. And they have established internal processes that dictate how to respond when a vulnerability is disclosed. These are the best practices that vendors should be looking to emulate.

Organizations with less mature security processes may feel anxious or afraid when they are alerted to a security vulnerability, so they need to understand that working with security researchers enables them to collaborate on solutions to mitigate vulnerable devices that cannot be patched (such as critical infrastructure). It takes time and patience to improve the security of connected devices, but it also takes a village. Manufacturers without the internal security resources to complete the due diligence of vulnerability assessment should lean into the broader cybersecurity community to collaborate with their peers and to share intelligence.