This is an excerpt from Out of the Wild: A Beginner's Guide to Package and Dependency Management, a Sonatype Guide. This is the first of three installments.
What Do We Mean When We Say Package and Dependency Management?
Terms like package manager, dependency management, repository, and repository manager are thrown around a lot in software development. Most people have at least a vague understanding of their meanings, but sometimes it’s hard to know if we’re all speaking a common language, with a common understanding, when these discussions arise.
Let’s get to the heart of what we mean when we talk about these terms in the context of DevOps.
Keeping in mind the definition of DevOps we arrived at in our own What is DevOps? article, as a “discipline rooted in collaboration and communication,” with “a common goal of shortening software delivery cycles and improving the stability of deployments,” there are many different concepts, practices, and toolsets that organizations can leverage to help enable those goals.
Some of the most common DevOps concepts and related tooling include Source Control Management (SCM) solutions like GitHub, CI/CD servers like Jenkins or Bamboo for automating stages of your software development lifecycle (SDLC), automated infrastructure configuration management tooling like Anisble, Terraform, Chef or Puppet, and containerization and orchestration tools like Docker and Kubernetes.
But there is another equally important DevOps concept, practice, and related toolset that is talked about less often than those mentioned above. Olivia Glenn-Han talks about this lesser-discussed topic in her article, The Universal Package Manager - The Most Critical Link in Your DevOps Toolchain. The Universal Package Manager can be a key component in helping “further the technical and cultural goals of DevOps” in your organization.
So let’s dive deeper into the concept and practice of package and dependency management—and the toolset that helps enable them.
Not that type of package manager
First, let’s nail down what we mean by package manager for the purposes of this guide. When people say “package manager,” it’s not always clear what type is being talked about until you have some additional context. Though important, for the purposes of this guide, we’re not talking about OS or system-level package managers/installers, like Homebrew for MacOS or RPM for Linux.
What we’re talking about is package managers that operate as application-level dependency managers, their corresponding language-specific package registries, and universal package managers, and how they all work together.
It’s important to note here that many of the concepts we’ll discuss can be applied to system-level package managers as well, but the examples in the rest of this guide will focus on application-level package managers.
Application-level Package (or Dependency) Managers
So, what are application-level package/dependency managers?
In his Medium article So You Want to Write a Package Manager, Sam Boyer distinguishes an application-level package manager as“an interactive system for managing the source code dependencies of a single project in a particular language.”
Examples of application-level package managers include:
Boyer goes on to say that application-level package managers provide “collective coherency” and an output that is “precisely reproducible.”
So let’s talk about why the phrases and descriptions above about the “managing,” “coherency,” and “reproducibility” of dependencies are so important. In other words, why do we need these application-level dependency managers?
As we’ve discussed in other articles, the role of software developer has changed significantly in recent years, and the reliance on open source software to build modern applications only continues to increase.
This means that the applications we develop often depend on other people’s code. This isn’t news to anyone developing software these days.
In fact, according to Sonatype’s State of the Software Supply Chain Report, a modern application is made up of more than 80% OSS components.
With this reliance on third-party dependencies to build software comes the realization that things can get messy quickly, especially when a direct dependency pulls in another component, resulting in nested transitive dependencies.
Managing this intricate web of dependencies out in the wild, unassisted, is no small task. Here is where application-level package managers can help:
“Thus, to build our software we need to bring in all parts on which it depends, including language libraries and remote third-party modules. But it’s not trivial to ensure that we have all necessary dependencies, particularly when dependencies themselves depend on others. This is why we need a Dependency Manager, often invoked during the software build process.” (Devopedia)
Boyer goes on to explain more of the complexity that comes into play:
“There is a natural tension between the need for absolute algorithmic certainty of outputs, and the fluidity inherent in development done by humans. That tension, being intrinsic and unavoidable, demands resolution. Providing that resolution is fundamentally what [application-level package managers] do.”
Application-level package managers are often closely linked to an online repository (also called registry) that stores packages (also referred to as libraries) for that particular programming language. For example, Maven by default sources components from the Central Repository for Java (and other languages) that Sonatype manages, and npm pulls packages from the Javascript registry at npmjs.org.
This is an excerpt from Out of the Wild: A Beginner's Guide to Package and Dependency Management, a Sonatype Guide.
Find more in the Sonatype Community, a place where you can ask questions to other Nexus users and the Sonatype team. Choose from an assortment of learning paths, developed by a team of experts, that helps make using Nexus even easier. I definitely recommend it.
