DHS Review Board Deems Log4j an 'Endemic' Cyber Threat

The US Department of Homeland Security's Cyber Safety Review Board (CSRB) has concluded that the Apache Log4j vulnerability disclosed in December 2021 will remain a significant risk to organizations for the next decade or longer.

The recently formed board, made up of private industry and government cybersecurity experts, determined that the open source community is not adequately resourced to ensure the security of its code and requires broad assistance from stakeholders across the private and public sectors. In a report published, today, the board recommended that federal agencies — as some of the largest consumers of open source code — contribute to open source security and called on the government to consider funding investments to improve security of the ecosystem.

CSRB released a set of 19 high-level recommendations for organizations to mitigate exposure to Log4j-related attacks and other similar software supply chain risks going forward. The recommendations for organizations include looking for and replacing vulnerable Log4j versions, establishing processes to prevent re-introduction of vulnerable versions into the environment, and maintaining an accurate inventory of IT assets and applications.

An Endemic Vulnerability

The CSRB's conclusions and recommendations are based on its months-long investigation into the circumstances surrounding the Log4j vulnerability disclosure and the response to it from the open source community, technology vendors, and government and private organizations.

"The Board assesses that Log4j is an 'endemic vulnerability' and that vulnerable instances of Log4j will remain in systems for many years to come," the CSRB said a report Thursday that summarized its findings.

"Though exploitation of Log4j has been at lower levels than expected and there has been no big Log4j attacks on critical infrastructure targets, the threat is not diminished," the report noted. "Significant risk remains."

"The most important aspects of the CSRB report should not surprise anyone who understands the reality of our complex interconnected world," says Katie Moussouris, founder and CEO of Luta Security and a CSRB member. "We depend on open source technology that isn't as well-supported from a security standpoint even though we need it to be, to help combat threats," she says.

The DHS established CSRB in February 2022 in response to a cybersecurity Executive Order the Biden administration issued last May. The CSRB's mandate is to get security experts from government and private organizations to review and assesses significant security events so improvements can be at a national level to prevent similar incidents. The Log4j review was the CSRB's first mission.

Apache Log4j is an open source logging tool that is present in almost every single Java application environment. In November 2021, a security engineer with China's e-commerce giant Alibaba reported a vulnerability (CVE-2021-44228) in Log4j to its maintainer, the Apache Software Foundation (ASF). The vulnerability — in a Log4j component for data storage and retrieval called Java Naming and Directory Interface (JNDI) — basically gave attackers a way to take complete remote control of vulnerable systems. Public disclosure of the vulnerability on Dec. 9, 2021, triggered widespread concern because it was easy to exploit, was ubiquitously present, and had disastrous consequences.

Another major, continuing issue — and one that the CSRB highlighted in its report — is the fact that vulnerable versions of Log4j are often not easily detected because of how deeply embedded the component can be in many environments.

A Preventable Catastrophe?

The CSRB review showed that an individual member of the open source community submitted the vulnerable JNDI component for inclusion with Log4j back in 2013. The Log4j team accepted the component, and it was later integrated into thousands of applications that used Log4j. The Board determined that the vulnerability could have been detected back in 2013 if the Log4j team had someone with security skills to review the code, or if they had training in secure coding practices.

"Unfortunately, the resources to perform such a review were not available to the volunteer developers who led this open-source project in 2013," the Board said.

Investigators found that the organizations which responded most effectively to the Log4j vulnerability disclosure were also the ones that had effective asset and risk management processes in place and had the resources to mobilize quick action on an enterprisewide scale. But few organizations were able to mount that kind of response, or had the speed required to respond to a vulnerability of this magnitude, CSRB found. As a result, there was considerable delay in both their assessment of risk from the vulnerability and in their management of it. Many had to decide whether to upgrade to the fixed version of Log4j that the ASF released — and risk business disruption from potential application breakages — or leave the vulnerability untouched and risk attack.

"The Log4j event highlighted fundamental adoption gaps in vulnerability response practices and overall cybersecurity hygiene," the report said.

Moussouris says Log4j highlighted the critical need for organizations to know their assets and what versions of software are running on their critical systems. "What might surprise the public is that so few organizations actually have a current list of their critical assets and what software is running on their networks," she says. "We're not prepared to respond to the next library incident until that changes."

One major takeaway from CSRB's report is the need for more coordinated action around open source security. Often, widely used open source components such as Log4j are maintained by volunteer teams with little consideration for security. They typically do not have coordinated vulnerability disclosure and response teams to investigate reported vulnerabilities and to address them.

"To reduce recurrence of the introduction of vulnerabilities like Log4j, it is essential that public and private sector stakeholders create centralized resourcing and security assistance structures that can support the open-source community going forward," CSRB said.

Increased Support for Open Source Ecosystem

Eric Brewer, vice president of infrastructure at Google, says the report provides a positive and nuanced view of how organizations need to approach open source use in their environments. "If you are using open source, you can't expect other people to magically fix security issues for you," he says. Implicit in the use of open source code is the fact that organizations are consuming the software "as-is." That means they need to share responsibility for mitigating risk associated with it as well, Brewer says.

He welcomes the CSRB's call for increased investments around open source security and says what's also needed are more organizations that can serve as curators for major open source projects. Big companies such as Google could fix vulnerabilities in open source code that they themselves consume and then offer the curated software so others can safely use it. He points to other organizations such as Red Hat and Databricks, which offer curated versions of major open source projects, as other examples.

"Open source software is fundamentally managed differently than commercial software, but open source software plays a key role in the success of commercial software," says Tim Mackey, principal security strategy at Synopsys Cybersecurity Research Center. Organizations that depend on a commercial vendor to alert them of a problem in an open source component are presuming the vendor is properly managing their usage of open source and that they are able to identify and alert all users of their impacted software. To mitigate the risk, "software consumers should implement a trust-but-verify model to validate whether the software they're given doesn't contain unpatched vulnerabilities," Mackey says.