The biggest problem facing software organizations today is an inability to track, monitor, and improve the usage of open source software. This isn’t about security alone. From DevOps to DevSecOps, there are fundamental principles that the best development teams use to guide open source software consumption. Many of these best practices come from traditional manufacturing, which is the focus of a new paper Sonatype CTO Brian Fox and I spent most of the last year researching and developing in partnership with Atlantic Council’s Open Source Policy Network and Digital Forensic Research Lab (DFRLab). The paper, “Driving Software Recalls: Manufacturing Supply Chain Best Practices for Open Source Consumption,” is available now. Feel free to jump over there, or continue reading to understand how this all came about.
Data-driven analysis of open source consumption
As with any research project, the best place to start is with data. So when it came to researching the best ways to improve open source consumption, a critical piece to software supply chains, we felt there was no better place to start than Sonatype’s State of the Software Supply Chain (SSSC) report. And it just so happens the most recent version was released two weeks ago. As has come to be expected, the SSSC report provides a wealth of insight into Open Source Software (OSS) and its role in the software supply chains. On the downside, and has already been pointed out in this blog many times and by others across the OSS, Software Security, and Cybersecurity communities, consumers (i.e., software developers and the organizations they work for) of OSS still aren’t paying attention to their consumption of open source.
The best evidence of this falls on the disconcerting percentage of downloads of vulnerable components that have a non-vulnerable version, which stands at 96 percent for the second year in a row. Even in the case of well-known OSS vulnerabilities like Log4shell, there is an unacceptable number of continued downloads of vulnerable versions of Log4j. What often follows with data like that is, “Why?” Though why can be very important in identifying the root cause, in this case, the better question is, “What can be done?”
Improving open source software security
About a year ago, the Atlantic Council published “Avoiding the success trap: Toward policy for open-source software as infrastructure.” I highly recommend reading this paper, as well as “The Tragedy of Digital Commons” by Chinmayi Sharma. Both were heavily influential in framing our own paper and providing context for Open Source Software as both a public good and as critical infrastructure. After reading the Atlantic Council’s paper, Brian and I set out to dig deeper into a few areas. Specifically, how similar Software Development had become to Software Manufacturing and identifying what best practices, principles, and mechanisms could be borrowed to improve open source software security.
One of the first outputs of our work was the Open Source Consumption Manifesto (OSCM), which was developed in collaboration with the End User Working Group at the OpenSSF. Published in August, the OSCM is intended to be a high-level organic and iterative document helping encourage and motivate software organizations to not only give more attention to their consumption of open source software but also make it a priority.
The OSCM is a great place to start, but to understand the issue related to the continued use of known vulnerable OSS more was needed. That said, our initial intention wasn’t to travel back nearly three hundred years to the start of the first industrial revolution. However, it did provide context to the rise and evolution of modern manufacturing, especially for key concepts like product liability and the case law around due care and standard of care. One concept in particular kept coming up: product recalls.
Recalls and modern manufacturing insights
On the surface, recalls are associated with direct action to remove a product. And, if that were the idea for software, it would be impossible, for example, how would one recall even just a piece of the massive infrastructure that represents “The Cloud?” We discovered that recalls aren’t exclusively focused on removing a product. In many cases, that can be just as impossible for physical products as for software. In contrast, when we look at recalls, specifically automotive recalls, it is more about communication and product improvement. That is, recalls provide a critical feedback loop that ensures a manufacturer’s ability to track, monitor, and improve parts, supplier relationships, and final goods.
For example, in 2019, Toyota encountered an issue related to a coolant leak in a particular engine used in several of their vehicles. This issue resulted in a recall of just under forty-five thousand vehicles. But it could have been much worse considering Toyota produces millions of vehicles each year, and the issue wasn’t identified until late 2020. In the paper we’ll dive into what Toyota did to minimize the impact of a defect like this one.
But our affinity for the wealth of knowledge to be gleaned from modern manufacturing isn’t new. Sonatype has spoken about the lessons, best practices, and processes that can be learned from modern manufacturing supply chains for nearly a decade, going back to our first software supply chain report. Specifically, key principles from W. Edwards Deming, a pivotal figure in shaping modern manufacturing supply chains, offer a wealth of guidance that can be adapted to help software manufacturers, as well as shape modern policy and regulation.
And this is the core premise of our paper. In a post-Log4shell world, many software manufacturers now find themselves on a precipice. Given the increased rhetoric and real policy and regulation by governments around the world, the lack of attention to open source consumption is no longer optional. But that also means policy may not be created by those closest to or with the best understanding of modern software manufacturing. Our paper, “Driving Software Recalls: Manufacturing Supply Chain Best Practices for Open Source Consumption,” aims to do that, and we encourage you to take a look.
Editor's note: Sonatype did not provide any financial or other support to the Atlantic Council in the course of producing this document.