Department of Defense DevSecOps Journey

March 30, 2020 By Sylvia Fronczak

3 minute read time

Editors Note: We recently discussed why the federal government should adopt DevSecOps. Here, a look at DevSecOps efforts at the Department of Defense presented at All Day DevOps. Sign up now for the upcoming All Day DevOps | Spring Break Edition happening April 17.

The U.S. Department of Defense (DoD) has a unique DevSecOps journey, and we'll discuss that today thanks to a presentation by Hasan Yasar and Nicolas Chaillan (@NicolasChaillan).

But first, here’s some background on the DoD. 

The DoD depends on software, but it doesn’t always control development. Instead, they must maintain software written elsewhere. Difficulties arise when the entire lifecycle is out of their hands.

Why is that? Well, when comparing DoD against the private sector, the DoD starts with acquisition. They purchase software that must later be integrated with all their existing systems. Surprisingly, they have more resources than the private sector, but they end up with less productivity. Because of these limitations, there’s also less agility. 

Another result of using software developed elsewhere, they must worry about latent cyber vulnerabilities. These vulnerabilities put the DoD at risk. 

Because of this ecosystem, they must work differently.

Issues the DoD Faces

So what sort of problems does DoD experience from that acquisition-based ecosystem?

First, development is a heavy waterfall process in every phase of the software development lifecycle. So when looking at the system with a DevOps perspective, things become difficult. All the timelines are extended. In fact, sometimes it takes years to identify errors in the system. 

Additionally, they experience integration difficulties. The testing is all manual, and configuration changes are extensive. To add more pain, they lack parity between their dev, integration, and prod environments.

Changing the Culture and Systems

The barriers to culture change are many. For one, the DoD cannot fail fast, nor can they fail in production, as the results could be devastating. Additionally they must integrate and manage hundreds of applications. More on the culture side, they lack the iterative and incremental mindset that many companies have developed. 

Their organizational structure creates additional barriers through excessive silos of knowledge. Systems are based on organizational structure. 

And then pushing to production creates additional issues. They’re unable to push to production as often as private organizations.

ParallelEffortsDevSecOps

Parallel efforts to DevSecOps from “DevSecOps Journey in DoD Enterprise”

How the DoD Is Implementing DevSecOps

So what’s the DoD enterprise DevSecOps initiative? First, the DevSecOps stack is open source and open to the public. Everything is infrastructure as code. And the stack can run on any environment, by leveraging Kubernetes. They also harden the K8S environment as much as possible.

The DoD brings enterprise IT capabilities with Cloud One and Platform One into the organization as well. This provides better onboarding and support for the teams.

Furthermore, they’ve standardized metrics and defined acceptable thresholds for all of the DoD. 

They're also providing training with self-learning capabilities with state of the art DevSecOps curriculum. In fact, they train over 100k people a year.

To make all this happen, they have built DevSecOps layers that all include continuous monitoring and infrastructure as code. The layers span from the application layers of the development teams all the way to the infrastructure layer.

DoD Tech Stack-1

The DoD’s tech stack from “DevSecOps Journey in DoD Enterprise”

In the DoD tech stack, they include continuous integration and continuous delivery as the core. Then, around that, they build upon stacks of tools for both building and operating their applications.

For security, the DoD uses a sidecar container security stack. This includes continuous scanning, alerting, and behavior detection with Twistlock. For container security and insider threat, they use Anchor for detecting changes in their Docker files. 

In conclusion, the DoD feels that if Kubernetes is good enough for their weapons systems, it’s definitely good enough for your business.

Tags: government, devsecops, Department of Defense, News and Views

Written by Sylvia Fronczak

Sylvia Fronczak is a software developer that has worked in various industries with various software methodologies. She’s currently focused on design practices that the whole team can own, understand, and evolve over time.