What do out-of-date third-party components, a product backlog, and a rushed software launch all have in common?
All are culprits that cause technical debt (or "tech debt") to pile up.
Technical debt is a lot like financial debt – you're acquiring an asset now and promising to pay for it later. Sonatype CEO Wayne Jackson notes that ignoring tech debt is like a "sugar high" during a software release. It allows you to ship more software faster, with all sorts of bells and whistles, making both customers and investors happy.
Until, of course, something vital breaks. This can lead to significant rework and refactoring, increasing your costs and, critically, slowing a development organization's productivity to a crawl.
Tech debt is a manageable challenge, but it can quickly snowball. Left unchecked, technical debt can become so costly that it undermines a company's short-term goals and derails the long-term ability to innovate. Some leaders know they will pay the debt eventually, but many others don't. Those who ignore tech debt are at high risk for catastrophic consequences. Just ask the management of Equifax or SolarWinds.
How technical debt affects innovation
Every company is now, at some level, a software company, and so every company has accumulated some form of technical debt. Understanding technical debt is critical to better understanding your software supply chain. In fact, tech debt is a way for you to consider your software supply chain from an angle that isn't just about cybersecurity. Managing – or eliminating – tech debt before it accumulates enhances your company's net innovation (raw innovation minus accumulated tech debt).
"Having a healthy software portfolio is about more than just cybersecurity," says Jackson. "A strategic focus on net innovation enables sustainable innovation capacity over time while also producing software with better cyber hygiene outcomes."
What are some causes of technical debt?
Evaluating technical debt can be like looking at a Rorschach test: It means different things to different people. The primary philosophical cause of technical debt is often short-sighted leadership. Some leaders simply don't want to acknowledge that rushing a product to market will eventually haunt them. Exorbitant costs and decreased development productivity are just two possible outcomes of that risky strategy.
But what specifically causes technical debt to lead to such costs? Below are three specific situations that cause tech debt: dangerous third-party components, a lack of maintainability, and software defects moving downstream.
Dangerous third-party components
Software that uses out-of-date third party components can contribute to the most common form of technical debt. For example, let's say you have a highly anticipated software launch. Your developers are working at breakneck speed to ship it out in time. In order to work at this level of speed, it's a common practice to source crucial components from an open source library or repository.
Now let's say one component sourced by the development team has 30 different versions. The developers don't necessarily choose the latest version; some may "pin" an older version of the component they want to reuse. Using a particular version of a component makes sense to developers because sometimes an updated version would break the component. However, one in ten components contains a serious security vulnerability. When the vulnerability is inevitably discovered, remediation can lead to hours of rework.
Lack of maintainability
The other problem with third-party suppliers is that there are so many of them. Enterprise software companies sometimes use thousands of open source projects. This poses two different problems – security policies and abandoned projects. It is difficult to know if these projects followed the same security protocols, policies, and procedures as your company. Any flaws, bugs, or vulnerabilities in these components adds to tech debt. Remediating the flaws cost considerable time and people-power.
Secondly, many of these projects have been abandoned by their committers (or maintainers). With 86 million active developers (according to GitHub) creating open source components, there is a high chance that your development teams are using multiple outdated projects full of security flaws and other defects.
Passing defects downstream
Open source components occur "upstream" in the software lifecycle, while testing and end users are often "downstream." One of the worst ways tech debt accumulates is allowing defects occurring upstream to pass downstream. As we discussed above, this has consequences for end users of your products, but a software defect moving from development to testing, staging, or production is another example of a defect moving downstream.
There is only one best case scenario here: the development team adds the defect to a backlog of refactoring work after the defect is identified. It will always be more costly to fix software defects in maintenance versus during its initial design.
For example, an extreme non-software example of defects moving downstream is the Takata airbag debacle, where defective airbags were placed inside millions of vehicles.
Fixing technical debt begins with your company leadership's approach to software development. One way to fix tech debt is to fully embrace lean manufacturing.
Jackson has spoken at length about how Edward Deming's lean manufacturing concepts helped reorganize auto manufacturer Toyota into an efficiency powerhouse. Sonatype has adapted some of Deming's principles to help describe tech debt optimization.
1. Choose fewer and better projects (or suppliers)
Well-managed third-party projects are better supported by the developer community. These suppliers enjoy greater stability because they have dedicated committers' support and enjoy broader assessment analysis. Fewer suppliers will provide greater long-term maintainability.
2. Pick higher quality component versions
This is the easiest way to avoid accumulating tech debt in the first place. An example of a trustworthy third-party vendor is one that uses the same security protocols as your company during component development. Using the latest version of a component across all departments of your company is also considered a software best practice.
3. Maintain an inventory for your software infrastructure
One of the easiest ways to do this is to create what Sonatype calls a "software bill of materials (SBOM)" – which allows you to triage the elimination of tech debt.
As noted in a previous blog post, an SBOM, "tells you what open source software components your development teams are using and describes which of those components have known security vulnerabilities, architectural risks, or license risks. It enables you to make those quick decisions, identify exposure, and take appropriate steps in response to new vulnerabilities."
4. Help your development teams think strategically by improving transparencyThe leadership team can help provide options for better supplier repositories so development teams aren't spending time sourcing (or sourcing suppliers or projects that are inherently bug-ridden). This helps eliminate passing defects downstream.
Self-service platforms are another way leadership teams can help support development. They're also known as "InnerSource." Self-service platforms use the same methodologies to build software as open source projects, and it allows development teams to contribute to other teams in your company, building transparency and enhancing teamwork. Self-service platforms have demonstrated success at reducing technical debt.
How Sonatype helps you eliminate technical debt
Sonatype understands technical debt's effect on a company's innovation. Many of the company's solutions specifically address ways to mitigate and remediate technical debt. Two solutions in particular – Sonatype Lifecycle and Sonatype Repository Firewall – can help companies eliminate tech debt before it occurs. Both solutions are powered by Sonatype's IQ Server, a policy engine powered by specific intelligence about open source components.
Sonatype Lifecycle: Sonatype Lifecycle provides tools that improve component usage in a company's software supply chain – through automation and accelerated delivery. Within Lifecycle, the IQ Server can automatically detect and fix open source dependency vulnerabilities that are already present. This helps to virtually eliminate the passage of defects further downstream.
Sonatype Repository Firewall: Sonatype Repository Firewall is an early warning detection system preventing bug-ridden open source components from entering a company’s repository. Firewall would virtually eliminate the tech debt scenario of dangerous third-party components entering your software supply chain. IQ Server's policy engine would automatically filter new components that don't match your company's security policies. The IQ Server within Firewall also integrates into a company's repository to do repository health checks to see if already-downloaded components meet your company’s security and compliance standards.
Control tech debt and enhance organizational longevity
Jackson says that a company’s leadership has to "truly in their soul" understand the threat of technical debt to a company's innovation.
Communication between leadership, product managers, and the teams involved in development, operations, and security is vital. Sharing the same goals and outcome will not only help secure your company, but it will also secure your company's ability to innovate.
"Innovation is essential to organizational longevity," says Sonatype CEO Wayne Jackson. "If a business loses its ability to innovate with software, then they are putting their business at risk."