Heartbleed & The Long Tail Of Vulnerabilities

Next week is London Technology Week in the UK, where innovative businesses will be showing off their best and brightest. Thursday afternoon, I’ll be talking about how the Heartbleed vulnerability is going to be with us for a long time as we continue to find it in the forgotten spaces of our networks.

It’s been just over two months since the disclosure of the Heartbleed vulnerability in OpenSSL -- one of the biggest security events in recent history. Webmasters, server administrators, and security professionals around the globe mobilized their forces and began a round of patching and updating that hadn’t been seen in years. According to Ivan Ristic at Qualys, we’ve been incredibly successful, and only about 1% of all web servers on the Internet are still vulnerable. So that’s the end of it, right? 

Not hardly. In fact, we are not even close to the end of our susceptibility to the Heartbleed vulnerability.

Let’s take a look at a concept called the "long tail."  Popularized by Chris Anderson in his book, The Long Tail: Why the Future of Business Is Selling Less of More, the term refers to the trailing edge of items consumers purchase that individually don’t amount to much, but as an aggregate might be equal to the top few selling items. The long tail is what makes sites like Etsy work, because even if each merchant only sells a few items, when you add up all the items sold by all the merchants, they mean a lot of money flowing across the site, making it economically viable for the company to exist.

While the long tail originally referred to the relationship between the popular and less popular items that merchants sell, it’s long since been bastardized to refer to any relationship where there is a large group of primary instances followed by a large, distributed group of instances that fade into obscurity. That pretty much describes the process of patching and updating software in the modern age of software-everywhere and the Internet of Things. The majority of high-profile systems get patched immediately, while the large number of systems that are unmanaged or receive less attention languish and are patched much later, if ever.

Think back for a moment to October 2008. If you’re like me, there’s not a lot that sticks out in your memory immediately. But if you were in security or IT at the time, you’ll remember that month’s Microsoft patches, which included MS08-067, a vulnerability in the Microsoft Server Service that allowed for remote code execution. Panic ensued, and the vulnerability was patched and all was well again. Except it wasn’t.

To this day there are still systems out there that are unpatched, there are still systems that are scanning for vulnerable systems, and there are still penetration testers using this vulnerability to break into companies every day. Attackers of all stripes still know they can find unmanaged systems at companies around the globe and can use these systems as jumping off points to get into the rest of the corporate network. Even though most of the systems that supported MS08-067 have long been decommissioned, there’s a long tail of systems still limping along in obscure corners of our networks that allow this vulnerability to be exploited on nearly a daily basis.

Heartbleed will be much worse than MS08-067 could have been, when you consider its long tail. At least the Microsoft vulnerability only affected Windows systems; Heartbleed affects OpenSSL, which is used in such a diverse range of systems and devices that no one actually knows in how many places it resides. The websites and blogs were obvious, and few were surprised to find that OpenSSL was part of the Android operating system or that it was part of many VPN software suites. But what about all the less obvious places, like home routers, CCTV systems, HVAC control systems? How many people realize that the systems that control our electricity and water, the SCADA systems, also used OpenSSL as part of their software? Probably not as many as should know.

Does this mean we should be dumping OpenSSL from systems? NO! OpenSSL serves a purpose: It’s a way for us to have a common library for the encryption of Internet traffic, and a vulnerability like Heartbleed is a problem any software could have. In fact, the amount of attention that Heartbleed has brought to OpenSSL means that in the short term we’ll be seeing many more vulnerabilities exposed, but as time goes by and the vulnerabilities are exposed, patched, and mitigated, OpenSSL will emerge from the process much more secure than it’s been in the past. 

What we should be concentrating  on at this time is understanding everywhere that OpenSSL is operating within our sphere of influences. It’s not an easy task, since every device within your network could be using code from OpenSSL if it has an administrative interface. (That’s assuming that you know about the device in the first place.) The long tail of Heartbleed will be with us for years as we find all the nooks and crannies that hold OpenSSL code and patch them or take them offline.

As Dave Lewis highlighted in his recent article, "Undocumented Vulnerability in Enterprise Security," few of us know about every system on our networks, something that’s only getting more complicated as the Internet of Things becomes less of a buzzword and more of a part of the fabric of our lives. When you think of how many undocumented nodes exist on a network, the long tail of vulnerabilities like Heartbleed only becomes more daunting and harder to deal with.