Software Supply Chain Security Needs a Bigger Picture

The intricate labyrinth of open source dependencies across the global software supply chain has created an application security puzzle of mammoth proportions. Whether open source or closed, most of the world's software today is built on third-party components and libraries. Consequently, one piece of vulnerable code in even the smallest of open source projects can have a domino effect that impacts thousands of other applications, APIs, cloud infrastructure components, and more.

This issue is becoming one of the most pressing security concerns of CISOs today, and at an individual enterprise level, organizations are hard at work tackling it with projects like building out software bills of materials (SBOMs), establishing open source security management standards, and creating technical guardrails for developers to follow them.

But these efforts don't necessarily solve the problem at a more systemic level. According to many experts in the open source community, in order to make the biggest dent in the downstream supply chain, more effort needs to be put into helping open source project maintainers clean up their code.

This is the goal of the Alpha-Omega Project. About to hit its one-year anniversary next month, Alpha-Omega is a big-picture security project put together by the Open Source Security Foundation (OpenSSF) and its parent organization, the Linux Foundation, to address the fundamental issues in software supply chain security.

The "Alpha" side of the project is focused on collaborating with the maintainers of the open source projects most critical to the broader supply chain — including notables like node and jQquery — to help them level up the security posture of their code. These are projects hand-selected by the OpenSSF Securing Critical Projects working group using expert opinion and data from benchmarks like the Open SSF Criticality Score to determine the projects with the biggest downstream impact.

The "Omega" side of the project turns to the long-tail of software supply chain security, using automation and tooling to identify critical security vulnerabilities across a range of 10,000 widely deployed open source projects. It's an effort to scale up the remediation of the lowest-hanging, most obvious flaws that are pervasive across the supply chain.

Funded initially by Google and Microsoft, with additional toolchain and personnel support from financial giant Citi, Alpha-Omega wrapped up 2022 by snagging an additional $2.5 million from Amazon Web Services. More crucially, the project is preparing for 2023 with two new critical hires — Yesenia Yser, formerly a product security engineer for Red Hat, and Jonathan Leitschuh, who just finished up his one-year stint as the first Dan Kaminsky Fellow for Human Security. Yser steps in as a senior software security engineer, and Leitschuh will continue his research on automating open source security research and remediation as a senior software security researcher.

Alpha-Omega Project's First Year

This project is one of several high-profile security projects spearheaded and fundraised by OpenSSF and Linux Foundation in the past year to tackle the systemic issues in open source security. Following the organizations' successful model for rapid funding and action on security projects, Alpha-Omega has already made headway on a number of significant fronts.

According to the project's first annual report, the project has already engaged with five different open source projects: Node.js, the Eclipse Foundation, the Rust Foundation, jQuery, and the Python Software Foundation. Over the course of 2022, Alpha-Omega doled out $1.5 million in grants to different projects, including $460,000 to the Rust Foundation, $400,000 to the Eclipse Foundation, and $300,000 to Node. In the case of Node, that support helped it reactivate the Node Security Working Group and get it working on a security and threat model for Node.js. It also spurred the triaging of 20 different vulnerability reports across the project's code base.

Additionally, Alpha-Omega recently released the initial version of the Omega Analysis Toolchain, which orchestrates 27 different security analyzers for identifying critical vulnerabilities in open source packages. The project also released a number of experimental tools, including a triage portal to make security research and reporting more efficient.

For year two, the project plans to accelerate work on the Omega side of the house.

What 2023 Has in Store for the Project

The addition of Yser and Leitschuh to the Alpha-Omega Project will not only infuse more brainpower, time, and talent into existing efforts, but also plenty of enthusiasm for moving the needle on open source security.

"Open source software is in every piece of equipment that is used today, from our automotives, airplanes, phones, trackers, and even utility systems," says Yser, who has deep roots in the DevSecOps and software supply chain worlds. In her position at Red Hat she was the supply chain ops technical lead.

"The vision for the project has a global impact of improving the security posture of open source software, supply chain security, and the lives of folks around the world," she says.

Yser will be working directly on improving the Omega toolchain and the triage portal to help engineer improvements in how projects and vulnerability impacts are analyzed and prioritized for mitigation.

"For the Omega tool chain, a goal for this year will be to have an operationalized system that a maintainer or developer can leverage," she says. "For the Triage Portal, the goal will be to support a researcher's ability to triage a discovered finding via importing a SARIF report to the portal and handle their investigation within the system. The system will remain limited to the Alpha-Omega team until noted otherwise, but thanks to open source software, a researcher can run their own instance and submit pull requests to the repository and support the overall mission."

Yser will be working in close collaboration with Leitschuh, who brings significant and very fresh experience to bear in the area of scaling and automating fixes across open source projects. He spent last year's fellowship working on this exact problem. His goal is to continue the work he did there and use what he learned to further his mission of rooting out the most prevalent and impactful flaws lurking across a wide swath of open source projects.

"We may not know where those little pegs are that are holding up the entire software industry exist," Leitschuh says. "It could be one of those tiny little pieces of software that has 15 stars on GitHub that nobody knows, but it's holding up the entire Internet. So how do we secure those projects that no one knows about but [are] somehow fundamental to the entire supply chain?"

He says his work during the fellowship helped him further home in on his niche of not necessarily going very deep on any one security vulnerability, but instead looking at a certain type of vulnerability and developing automated ways of finding that same flaw in lots of different places across the open source ecosystem. This dovetails perfectly with the Omega ethos, which is what led him to his newest gig.

Leitschuh will keep supporting refinements on automated methods for running down flaws in Data Flow and Control analysis and auto pull request generation. But he's also going to be continuing the very manual work of collaboration. One of the important lessons he learned last year is that a lot of the work ahead of him and his Alpha-Omega team is not necessarily technical. It's about building relationships with maintainers to help them see how sometimes even simple fixes to their projects can have a huge impact on global software supply chain security postures.

"Technologists and software people, we don't always love the human element — it's easier for us to sit down and write a line of code that detects this thing and throw it over a wall than it is for us to engage with an actual person and try to convince them this is a thing worth fixing," he says.

He explains how one instance last year illustrates this point perfectly. In this case he worked with a maintainer of a YAML Parser that had a 6-year-old remote code execution flaw that had a lot of downstream impact. For a long time when Leitschuh approached him about it, the maintainer told him, "Don't trust untrusted YAML. This is not my vulnerability."

Finally, after sitting the maintainer down in a video call with lots of technical debate, Leitschuh was able to show him that the change he requested was extremely narrow and could have a huge impact.

"So he's now willing to fix this 6-year-old remote code execution vulnerability in this YAML Parser because someone like me sat down with him on a video call, finally, and had a conversation with him to convince him the minimal thing that he needed to do to make it more secure," he says.

While Leitschuh could have automated fixing the vulnerability downstream, the more elegant fix was having this discussion instead.

"I thought it was worth it for me to sit down and spend the time focusing on this one piece of software to try to convince this maintainer. Having those conversations is what's going to have a wider positive impact writ large on the entire industry," he says. "At that point you just need boots on the ground. You need people that know what they're talking about to sit down and spend time that is required to engage with an actual person."