As many of you have experienced, there’s an increasing push to deliver more, faster. And when we say “more,” we mean more features—not more non-functional requirements.
Coming from agile methodologies, we often ask, “What business problem does this solve?” or, “What value are we providing the customer?” whenever we add new features. Although these are valuable questions to ask, we need to realize that there are important and non-functional requirements that we can’t do without. And sometimes it’s up to us as developers to sound the alarm and push for features that don’t have as much visible value.
So what does this have to do with dirty rivers? Let’s take a look.
By now, most of us have heard the term “garbage in, garbage out.” It describes how, especially with data, we can’t get good output without good input. But what other garbage do we deal with?
Consider a software product as a reservoir. It provides benefits. Reservoirs provide us with water; software solves business problems or streamlines processes.
The reservoir is created by rivers and streams that flow into it. Now, what if one of those rivers is polluted? It pollutes the whole reservoir. Similarly, in software, if we add dependencies that are polluted or vulnerable, we infect the whole software product. And that vulnerability can be injected anywhere along the supply chain.
Back in the day, security in applications seemed simpler (or maybe we were more naive). We’d make sure that we weren’t allowing cross-site scripting or SQL injection and we’d call it a day. Most of the software we wrote had only a few external libraries pulled in, so as long as we used trusted libraries we were doing pretty well.
Now things have significantly changed. Most developers are not even aware of how many different libraries are included in their applications by way of dependencies.
Generally speaking, dependencies may not seem like a big deal. But they can wreak havoc when too many developers put blind faith into any random library they find. Whether it’s a security breach or simply availability threats, we need to start taking a closer look at the tools we add to our applications.
Consider the left-pad JavaScript library, the removal of which from npm disrupted thousands of applications just a few years ago. Could something like this happen again? Definitely.
And developers typically don’t think of libraries going away as security vulnerabilities. But for many companies, the loss of access to open-source libraries can be a big deal. So we must remember that vulnerabilities aren’t just breach or malware threats. They could be basic availability threats where removing a simple library that everyone depends on has huge repercussions.
In some ways, it seems like low quality actually reaps rewards. We’re in a world of MVPs and go-to-market strategies that reward being first over being right. And security concerns in the supply chain are often not considered the most critical aspect of software.
Consider the well-publicized software and security breaches of the past few years. How many of them resulted in significant fines or repercussions? Yes, many companies do pay, but when fines amount to $1 per head or less, how incentivized are these companies to change their ways?
“Today there are no real consequences for having bad security or having low-quality software of any kind. Even worse, the marketplace often rewards low quality. More precisely, it rewards additional features and timely release dates, even if they come at the expense of quality.” — Bruce Schneier
But remember: though we feel added pressure to deliver software faster, the bad guys also feel some added pressure to find vulnerabilities and exploit them. They want their businesses to improve, too. And they’re on track to make a big impact. In fact, Cybersecurity Ventures predicts that their industry will soon be more profitable than global drug cartels.
So although it seems at times that it pays to take shortcuts, in the long term we’ll have to take a harder look at our software supply chain to make sure we stay ahead.
How do we keep on top of everything that’s happening on the cybersecurity front? Security experts have spent many years relying on the National Vulnerability Database from the US government. Unfortunately, that database has not been kept as up to date as it has in the past. There just isn’t enough manpower (or automation) to keep it updated with the daily threats that arise.
Furthermore, when things are listed in the National Vulnerability Database, do you know who else is studying it? The bad guys. So when a vulnerability is listed, it’s not just our security engineers reviewing the listing. Those who want to cause mischief or profit from vulnerabilities review everything that comes out. And if they find a vulnerability they can exploit, they will. And, quickly. It will only take a few days for nefarious hackers to find ways to exploit these listings. Meanwhile, companies that are not on top of vulnerability listings or don’t have their DevOps processes set up to push upgrades quickly will be vulnerable for weeks, if not months.
We need to get faster at plugging the holes. This is why concepts like CI/CD are critical. They not only let us get features to market faster, but they also let us upgrade or remove versions of dependencies faster. The last thing we want is to have a vulnerability be announced and then have to wait for a deployment window three weeks out to be able to resolve the problem.
So, we’ve stayed pretty high level so far. But now you may be wondering how you can take steps using DevOps to protect your products. Let’s take a look at a few.
Together with the drive to deliver business value quickly, we’ll have to tune our DevOps processes to stay vigilant against threats.
As developers, we’re often tasked with additional steps to validate the security of our dependencies. Instead of fighting measures created to ensure a clean supply chain, work to make them better. Find ways to automate security into your daily processes.
And finally, know that this is an evolving process. As always, iterate on solutions, make them faster, and work towards providing the best products possible, both functionally and non-functionally.
For more insight into the role of software supply chains, new data on the OSS vulnerabilities and how to keep our reservoirs cleaner - download Sonatype’s 2018 State of the Software Supply here.
About the author Sylvia Fronczak:
Sylvia Fronczak is a software developer that has worked in various industries with various software methodologies. She’s currently focused on design practices that the whole team can own, understand, and evolve over time.