Sonatype Selected by Equifax to Support OS Governance Press Release

blog-logo Sonatype Blog

Building Container Based Pipelines in AWS with BJ's Wholesale Club

June 12, 2019 By Justin Reynolds
BJ’s Wholesale Club has 217 brick-and-mortar locations spread out across 16 states. The company has more than 5.5 million members and employs more than 25,000 people.

Currently, BJ’s is in the middle of digital transformation to deliver exemplary digital experiences to their customers, which requires agile DevOps workflows routed in containers and microservices.

As a membership business, BJ’s is focused on attracting and retaining members, delivering value to get them shopping, and making it more convenient to shop at BJ’s.

Here are four areas of BJ’s digital transformation:
  1. Launching a mobile application -They deployed a mobile application that’s been downloaded more than 1.5 million times. Shoppers can clip coupons digitally, eliminating the need for paper-based coupons.
  2. Express Pay- A BJ’s mobile app user can use this feature to run in “club” mode, which enables them to scan barcodes and check out without going through any lanes or any registers. Every transaction is completed on the mobile app.
  3. Same-day delivery - Many of BJ’s fresh and grocery items can be delivered right to a member’s home.
  4. BOPIC -This service allows members to buy things online and pick them up in a nearby store a few hours later.
To enable this digital transformation while ensuring delightful user experiencing, BJ’s embraced a DevOps-first mindset and started building with containers and microservices.

How BJ’s Uses Containers to Delight Its Customers

To deliver transformative experiences, BJ’s uses a variety of tools. For example, they use the entire Atlassian suite, including Bamboo for CI/CD and Bitbucket for source control. They generate real-time dashboards that can be fed right back to the development and engineering teams. And of course, they use Nexus Firewall.

BJ’s is also heavily vested in the AWS platform, using a number of services, including:
  • Amazon ECS, for infinite scaling
  • Amazon Elastic Container Registry, to store images
  • Amazon EC2, to automatically scale containers
Finally, they use WhiteHat security for real-time feedback and container security.

pasted image 0 (12)

Leading IT teams are moving at a fast pace into the container and microservices world. When BJ’s started their journey, they decided to tackle things in phases, which can be briefly summarized as follows:
  • Phase 1—using Docker for containerization
  • Phase 2—figuring out how their applications were going to run on Docker
  • Phase 3—finding out what application is doing during runtime

How to Launch Microservices at Your Organization

When BJ’s was starting out with this transformation, there was a lot of frustrating moments.

To address these concerns, the team started by crawling before they ran.
pasted image 0 (13)

The folks at BJ’s had to figure out how to organize repositories, establish a Git flow for application and infrastructure sources, build a skeleton pipeline, and get all of it working together cohesively.

After the team got all that in place, it was time to integrate security tooling into the development process and focus on CI/CD. By leveraging a lot of the goodness that comes out of the Sonatype platform, everything got easier, according to the BJ development team.

In the beginning, there was a lot of data being thrown at them. To make it all work, the team needed to understand what every alert meant. Shortly thereafter, the team noticed that they could fix issues right out of the gate, and developers started doing precisely that immediately.

As a result of their transformation, BJ’s now has the capability to stop builds if they need to. The company is doing a lot with AWS, including blue/green deployments and A/B testing. This enables them to test new features on their website and measure how they’re received by their users. By being able to build artificial intelligence that’s based on good data in the pipeline, the team can automate a lot of change management processes. In the future, BJ’s hopes to move changes directly into production via continuous deployment.

As a result of the move to microservices and containers, BJ’s has increased its agility and, as a result, is able to ship releases with more frequency because remediation is done nearly continuously in a sandbox-like environment.

BJ’s Guiding Principles

To succeed in their transformation, the BJ’s team embraced five guiding principles:

  • Experiment. Try different approaches.
  • Use plugins wherever possible. Don’t reinvent the wheel.
  • Create pipeline patterns that other teams can leverage.
  • Don’t make security analysis a handoff. Make it part of the developer’s environment.
  • It’s all about iteration. Pipeline evolution is done incrementally.

Despite their impressive results, BJ’s isn’t done with their transformation. Some might even say they’re just getting started. Moving forward, BJ’s is focused on:

  • Integrating automation for container scanning into their pipeline
  • Integrating a Python ecosystem into Nexus
  • Getting a tighter feedback loop for Jira
  • Leveraging Nexus as private Docker repository
  • Taking advantage of other services offered within AWS (e.g., SE3 backend for artifact binaries)

Thanks to their decision to use containers and microservices, BJ’s developers can now have conversations without people falling off their chairs, so to speak.

Now that the company’s development community understands how much visibility they can have into their code, they’re committed to writing clean code and are okay with stopping a build if they have to.

It remains to be seen what the BJ’s development team will come up with next. Whatever they end up building can be shipped quickly, updated immediately, and measured to ensure users actually like it.

Hear everything that the team at BJ's Wholsale Club shared at the Nexus User Conference below: 


Tags: devsecops, aws, container, Nexus User Conference, Customer Stories

Written by Justin Reynolds

Justin Reynolds is a freelance writer who enjoys telling stories about how technology, science, and creativity can help workers be more productive. In his spare time, he likes seeing or playing live music, hiking, and traveling