Three Years after Heartbleed, How Vulnerable Are You?

Three years ago, the Heartbleed vulnerability in the OpenSSL cryptographic library sent the software industry and companies around the world into a panic. Software developers didn't know enough about the open source components used in their own products to understand whether their software was vulnerable — and customers using that software didn't know either.

Unfortunately, the majority of today's software companies aren't any less vulnerable today than they were three years ago. To prevent future Heartbleeds, companies must put in place processes and automation to scan, track, and manage the open source components they use in their products. Doing so will let them responsibly understand where the open source components reside within their software products, which of these components are vulnerable, and which customers are exposed. 

More on Security
Live at Interop ITX

Today's Reality
Ten to 20 years ago, a VP of engineering could turn around, look at the bookshelf, and see the product boxes of the libraries the company licensed. Maybe there were a few digital downloads, or some code from a famous book. Slowly at first, then seemingly overnight, the development model changed from mostly homegrown software with a few commercial libraries to projects that are at least 50% open source, all digitally delivered, and not always in well-defined locations in the source tree. To answer the question "Are we affected by this latest vulnerability?" requires a current listing of dependencies or bill of materials (BOM).

Although most companies believe that such a list is being managed, the vast majority of software companies would be hard pressed to produce a list like that, even post-Heartbleed. Research shows that typical software companies can't create such a listing, and if they can, the average percentage of known components is only 4%! What this means is that for every known component, there are 24 other unknown and unmanaged components that are being used and delivered to customers.

A Quick Review
I recommend that you see if you can produce a current BOM and determine when it was last updated. If this list was created only using self-reporting by developers, it's almost certainly only a small percentage of reality. Perform some sampling of the codebase to check if the version numbers reported are still current, and do a quick review of library names; a quick search on copyright strings and licenses; and a review of file extensions that are likely third party (.JAR or .DLL, for instance). Even a quick review will uncover large amounts of previously unknown third-party software.

Doing a search for the string "OpenSSL" can be even more eye-opening. It's pretty much guaranteed you'll find multiple copies of previously unknown instances of OpenSSL in open source components and embedded in commercial components. Both source and binary inclusions will be found.

These self-tests can quickly show that either your BOM is incomplete or out of date. Although this isn't a happy thing to find out, at least you'll have lots of company.

Use Cases
There are a few main use cases where you will find open source components that may be affected by published vulnerabilities. The most common will have been brought in as top-level components, often in clearly named files or directories. The next case to be found will be subcomponents of these top-level components. This type of open source software use is harder to discover and is often overlooked. Versions of these components that have been compiled and linked into other larger packages are almost invisible to the typical developer when trying to create a complete list of dependences. Lastly, codebase owners will want to review the remaining source files to find cut-and-pastes, refactorings, or rearrangements of a larger open source package. This last category can be almost impossible to account for by eyeballing the codebase alone.

Once a current BOM is created, it should be compared against components with known vulnerabilities. Expect multiple vulnerable components to be found, especially on the first review cycle. Some of these may be easily exploitable, while others may not have such a clear path to exploit. It's common to triage these results. Steps to fix this problem include upgrading to the latest version, patching, blocking access, and, in some cases, removing the component and product features affected.

You may find that a vulnerability was introduced because of a subcomponent of a larger component that was delivered via your software supply chain. If you're lucky, you may find that your open source and commercial suppliers have already patched the affected component and it's ready for download. It's also common to find out that they are not aware, or not ready to remediate the issue. 

Defect Logging Process
Familiarizing yourself with the defect logging process for your open source and commercial components is an important part of participating in the vulnerability life cycle.

Once a new version of your product is delivered, the process of keeping a valid current BOM, and checking this list against known vulnerabilities, should continue as long as the product is developed and used. This should be done for all versions used by your user community. Although your development team may have moved to the latest version, you may have significant numbers of users happily using much older versions of your software.

Until the software industry puts into place processes similar to the ones detailed above, it will continue to be unprepared to quickly respond to new Heartbleed-style vulnerabilities.

Related Content: