It’s rare to see a community truly come together for the common good, but that’s exactly what happened yesterday within our open source community. We cherished the opportunity to participate in a conversation, led by the Open Source Security Foundation (OpenSSF), where industry, open source foundations, and government all came together to discuss how we solve the massive problem of securing open source software (OSS).
Being a part of these conversations with White House officials and nearly 50 other companies or organizations, who also believe in harnessing all the power of open source, while minimizing its risk, has been invigorating. I’m proud of the work that’s gone into developing a new plan set forth by OpenSSF titled The Open Source Software Security Mobilization Plan (more on that below). With this, we’re making incredible steps toward getting open source maintainers and the producers of open source, the help they deserve. And we are fully committed to playing whatever role we can in that endeavor.
But, I also ask that we as a community, don’t lose sight of all the things we can do right this minute to help consumers of open source better manage and secure their supply chains. As we fix the broader system, there is tooling that already exists that makes more secure and more maintainable open source possible.
Let me explain more below.
Open Source Use is Ubiquitous
When we first started Sonatype, it was an uphill battle helping companies understand how much open source they were even using. We’ve come a long way since then and there is little question that most development organizations, whether industry or government are widely using open source within their software supply chains. Just for some context on how widespread its use is, research shows:
- Approximately 85% of every application is comprised of OSS
- In 2021, developers downloaded more than 2.3 trillion open source packages from top ecosystems
- There are more than 37 million different versions of open source packages available
If you’re building software, you’re doing it on top of open source.
The Log4j Wake Up Call
While many throughout the development and security communities, including us here at Sonatype, have been focused on educating the world about the benefits, and risks, of using open source software, it wasn’t until December 2021, when many finally received the wake up call they needed.
At the end of last year, a serious zero-day Remote Code Execution exploit in Log4j (CVE-2021-44228), the most popular Java logging framework used by Java software far and wide, was discovered. This type of vulnerability is especially dangerous because it can be used to run any code via your software and requires very low skills to pull off from an attacker. That, on top of how omnipresent Log4j is in Java applications, caused widespread panic. As one reporter described it, the “internet was set on fire.”
Now, this wasn’t the first “wake-up call,” people should have heard, but most just hit the snooze button. The first alarm was in 2013 when the world witnessed the first prominent Apache Struts vulnerability. While Apache responsibly and publicly disclosed the vulnerability at the same time they offered a new version to fix the problem, industry was slow to react. Despite Apache doing their best to alert the public and prevent attacks from happening, many organizations were either not listening, did not act in a timely fashion, or had no idea they were using it, resulting in widespread exploits in the wild. Simply stated, hackers profit handsomely when companies are asleep at the wheel and fail to react in a timely fashion to public vulnerability disclosures.
Since then, we’ve witnessed Shellshock, Heartbleed, Commons Collection, the 2017 Struts2 vulnerability that caused the Equifax breach, and most recently the Spring4Shell incident. All of these followed the same pattern of widespread exploit post-disclosure.
These vulnerabilities are becoming increasingly common - and in fact 29% of popular open source components contain a known vulnerability. That said, it was Log4j, and its widespread use, that caught the attention of the President and governments around the world, to finally prompt industry to tackle this issue head on with a call for action. Sadly, even today, 35% of Log4j downloads continue to be of the vulnerable versions.
Enter The Open Source Software Security Mobilization Plan
In response to this call for change, OpenSSF organized its members, government, and the broader community to develop the The Open Source Software Security Mobilization Plan, which addresses a key part of the problem - fixing OSS software at the source, and helping open source maintainers better manage their projects. As quoted from the document, the plan is laid out into three main streams, including:
- Securing OSS Production: focus on preventing security defects and vulnerabilities in code and open source packages in the first place
- Improving Vulnerability Discovery & Remediation: improving the process for finding defects and fixing them
- Shorten Ecosystem Patching Response Time: Shorten the response time for distributing and implementing fixes.
The mobilization plan then proceeds to map out 10 key streams of focus within these 3 target areas. I won’t dive into all of them here, but wanted to highlight a few key concepts I believe will be the most impactful.
Establish a public, vendor-neutral, objective-metrics-based risk assessment dashboard for the top 10,000 (or more) OSS components.
We’ve long touted Edwards Deming’s principles of supply chain management as the best way to build software. Because software is also made from component parts (the open source we’re discussing), much of the software industry looks to the “supply chain” model Deming applied to traditional manufacturing.
And one of his key principles was - use better and fewer suppliers. With open source, that means only using the best open source projects. But, how do you know if a project is good or not? This can help change that.
While the dashboard, metrics, and badge being proposed are mainly part of the “securing OSS production,” stream to help maintainers themselves better understand how to secure their projects, this risk-based assessment dashboard, can help both maintainers and consumers.
Accelerate the adoption of digital signatures on software releases.
As the plan notes, “Digital signatures are a critical part of ensuring that all participants in the ecosystem, from software builders to end users, can verify that the components that they use are indeed the components that they intend to use.”
I couldn’t agree more with how important that is. It’s why we’ve announced plans to leverage Sigstore for modernizing component signatures in Maven Central. I’ve long written about the importance of namespacing in public repositories, which ties back to ensuring the provenance of the components in a public repository. We are working with multiple constituents across the Java ecosystem such as Apache Maven and Gradle, as well as other ecosystems (via the OpenSSF Securing Public Software Repositories WF) to bring all of this together.
Accelerate discovery of new vulnerabilities by maintainers and experts through advanced security tools and expert guidance.
One of the biggest challenges, as the OpenSSF plan states, is that open source montainers simply don’t have access to different kinds of security scanning tools.
While many organizations, including ourselves, are constantly working with maintainers to find vulnerabilities and fix them, if maintainers had the ability to scan their projects themselves, and understand the depth of dependencies they rely on better, it would certainly catch some vulnerabilities earlier.
This is exactly why we’ve made Sonatype Lift free forever for public repositories, and why we provide automatic Lift scanning for everything published to Maven Central, to help serve open source maintainers by securing the software supply chain at the source. We’re excited to collaborate with other tools within the industry to give maintainers the information they need.
SBOM everywhere — improve SBOM tooling and training to drive adoption.
SBOMs or software bills of materials are so critical to managing any software supply chain. You simply can’t manage it if you don’t know what code you’re using, and the only way to do that is with an SBOMs.
As the OpenSSF plan rightfully says, these are becoming more common practice, but are still far from ubiquitous, and more importantly, there are very few standards that people are following when creating SBOMs.
Uniting the industry on what standard SBOMs should include and making them easier to use, will go a very, very long way in making open source use safer.
Beyond Open Source Maintainers, Securing Open Source with Consumers
The OpenSSF’s Open Source Software Security Mobilization Plan does a fantastic job of laying out a program to help open source projects and maintainers become more secure, thus hopefully making open source better and safer. There are many, many things I agree with and I’m very proud to be a part of it. But we need a two-pronged approach.
Securing production is obvious and important (and may take a while), but we desperately need to secure consumption too. And maybe even more urgently.
How do we Secure Software Supply Chains Now?
Helping organizations, developers, security teams and everyone in between better manage their supply chains, is something that we can do now. And we can do it without putting too much more pressure on developers.
I already briefly introduced one of Deming’s principles of supply chain management above, but let me share the other 3 principles (for a total of 4) that I believe we need to take into consideration when building and managing software supply chains.
- Use better and fewer suppliers
- Use high quality parts from those suppliers
- Resolve defects early and never pass known defects downstream
- Create transparency and track what you use and where
By following these four principles, it’s possible to not just have more secure supply chains now, but development processes that are more maintainable, more hygienic and have more integrity.
Tooling exists today, yes Sonatype’s, but also others, that can make huge changes in how open source consumers develop more secure and sustainable software on top of better shared components. As we work to improve and secure OSS at the source, we can’t lose sight of the importance of doing something immediately and helping open source consumers protect themselves in the present.
Further, the reality of software is that it’s made by humans, and likely always will be to some extent. Humans are fallible, and thus even the best projects, leveraging all of the improvements called out in the mobilization plan, will have problems from time to time. Just focusing on the production of open source leaves the entire industry open to failure. If we don’t also help consumers figure out how to manage their software supply chains, there will always be problems.