Anatomy of a Continuous Delivery Pipeline

February 04, 2020 By Peter Morlion

3 minute read time

What does a continuous delivery pipeline look like? Which pieces do we need to have in place for us to achieve true continuous delivery? Kamalika Majumder lays out the anatomy of a continuous delivery pipeline.

What Companies Need

Companies are always looking for

  • shorter release cycles
  • continuous integration
  • QA
  • security

In essence, they want to sell faster and sell better quality products. This applies to all companies, whether they’re selling services or actual products.

Most companies use many different DevOps tools, but don’t have the results to show for it because:

  • they can’t deliver when they want to
  • they encounter regression bugs in production
  • the delivery of new features is slow

Why do we still see these issues a decade after DevOps gained traction?

One-Stop Station

Everyone needs one single pipeline for the application lifecycle management and continuous delivery of quality products. This includes:

  • development
  • testing
  • monitoring
  • release

This all needs to be linked together into one single pipeline. Today, we’ll focus on the basics of a continuous delivery pipeline.

Back to Basics

Martin Fowler defined continuous delivery as

“a discipline where you build software in such a way that the software can be released to production at any time.”

It’s important to stress that your code is always deployable. This means continuous delivery is a cycle of five principles:

  • develop
  • build
  • test
  • deploy
  • and release

Development

During development, merging code can lead to merge conflicts. This is a frustrating experience. This is caused by waiting too long to merge your changes back into the master branch. You need to keep your code updated, shortening the feedback cycle.

Trunk based development solves this issue. Temporary branches are okay for features, and short-lived release branches are fine. But everyone should commit to the master branch regularly (once a day!).

If you have unfinished code, hide it with feature toggles instead of keeping a branch open for too long.

Keep your master branch in a releasable state and never add to a broken master branch.

Build

We’re still seeing “it works on my machine” issues.

We should build our artifacts once and deploy multiple times. We should version our artifacts (using semantic versioning). And this means all our artifacts: applications, configuration, database scripts, etc. Automate all of this in a pipeline.

Test

Previously, many teams built their applications first and added tests later. This leads to many manual tests and UI tests. This is a slow and error-prone process. It’s better to start from the bottom up using test driven development. Write your tests first. This leads to a shorter feedback cycle.

Then work your way “up” and automate each level: integration, API and UI tests. Don’t forget performance and security tests as well.

Deploy

When something in the infrastructure changes, this can break things and then we often have a hard time finding the cause. Version controlling your infrastructure as code helps identify changes that were made to the infrastructure.

This also applies to configuration. If everything is in source control, you can easily see what changed and why.

When you have infrastructure and configuration as code, you can run automated tests against it. Just like you write unit tests for your code, you can write unit tests for infrastructure or configuration.

Then we can version this and run it through a continuous integration pipeline. This way, you can link application versions and configuration versions to what is running in production.

Finally, look at zero downtime deployments. In this day and age, you should be able to update your system without downtime.

Release

Avoid having one deployment team. If your deployment pipeline is configured correctly, with the correct access rights, anyone from the development team can deploy to production.

Once you’re running your application, make sure you have robust monitoring, logging and alerting:

  • how long does it take to build?
  • to deploy?
  • to fix an issue?

This is useful to improve your continuous delivery cycle over time.

And always have a plan B. If something goes horribly wrong, make sure you can perform a rollback.

You can check out Kamalika Majumder’s full All Day DevOps talk “Anatomy of a Continuous Delivery Pipeline” here. 

Tags: continuous deployment, continuous development, Enterprise DevOps, Continuous Delivery, devsecops, continuous everything, pipelines, News and Views, devops frameworks

Written by Peter Morlion

Peter Morlion is a passionate programmer that helps people and companies improve the quality of their code, especially in legacy codebases. He firmly believes that industry best practices are invaluable when working towards this goal, and his specialties include TDD, DI, and SOLID principles.