*Note: Join us live and online for the 2019 Nexus User Conference on June 12. Registration is free.
“DevOps is Agile on steroids - because Agile isn’t Agile enough.”
So says Jim Bird, the CTO for BiDS Trading, a trading platform for institutional investors. Jim continued, “DevOps teams can move really fast. . . maybe too fast? This is a significant challenge for operations and security. How do you identify and contain risks when decisions are being made quickly and often by self-managing delivery teams? CABs, annual pen tests, and periodic vulnerability assessment are quickly made irrelevant. How can you prove compliance when developers are pushing their own changes to production?”
Jim was presenting at the 2018 Nexus User Conference on Continuous Delivery. Pulling on his 20+ years of experience in development, operations, and security in highly regulated environments, Jim laid how and why Continuous Delivery reduces risk and how you can get some easy wins toward making it more secure.
As Jim points out, it reduces risk because:
- Breaking work down and delivering it incrementally reduces delivery costs and risks
- Continuous Delivery takes care of the critical “last mile” of production
- Change becomes routine
- Change becomes cheap, because teams can afford to make more small bets, iterate, and learn
- Change becomes fast. Teams can push fixes out quickly to respond to failures or security threats.
To make Continuous Delivery secure without getting in the delivery team’s way, Jim outlines a few key points:
- Understand the team’s workflow and tool-chain - work the way that they work
- Address real risks and threats - problems that everyone agree are important and relevant
- Take an incremental, iterative approach
- Provide useful, quick, and continuous feedback to the team and to management
- Minimize friction and delays
- Look for controls and checks that are easy and cheap to implement
With the case made, Jim provides 5 quick wins for securing continuous delivery:
1. Automated safety checks in your pipelines
Why? Because you are making changes often, it is easier to make mistakes. You need simple, fast checks that find them before the bad guys. This makes auditors happy because they can see that policies are being enforced.
How? Add tests/assets for common mistakes to the end of the automated build/deploy pipeline in test and production. This includes: default/missing credentials; SSL and HTTPS settings; known critical vulnerabilities; file and user permission, and, open ports.
Tools and frameworks available include:
- Netflix Security Monkey, Conformity Monkey
- Gauntit
- OWASP ZAP Baseline Scan
- OSQuery
- InSpec
- Cloud services like AWS Inspector
2. Lightweight risk assessments
Why? Teams are making decisions quickly, so you want them to be able to do this safely and to look for potential new risks upfront before coding starts. You get security/compliance stories into the backlog so that they can be planned for, and so you can find the right tools, libraries, and frameworks. It also makes the risks visible to the teams and management.
How? Check out Mozilla’s Rapid Risk Assessment. It’s a good example.
3. Security Unit Tests
Why? These prove that vulnerabilities are being fixed correctly. They teach developers how to “step off of the happy path” in their testing, and it will catch/prevent regressions. Normal tests prove it works, not that it might not work, and developers need to write happy tests, sad tests, and bad tests.
How? As you find vulnerabilities, write a test first so you can prove it exists. Then, have developers fix it and then run the test again. They can use tools such as JUnit, NUnit, and Jasmine.
4. Secure the software supply chain
Why? Third-party code injects risk and security vulnerabilities and technical debt: known security vulnerabilities; abandoned and out-of-date code; and, license risks. It is relatively easy, there is some lifting at the front, but, once you do it, you have a nice control set going forward.
How? Get a tool to automatically catalog dependencies and check against vulnerability databases. Integrate it into the Continuous Delivery pipeline and fail builds on serious problems.
Tools include:
- OWASP Dependency Check
- Retire JS
- Anchore Engine
- Nexus Lifecycle/Firewall
- Licensed
- GitHub security alerts
5. Self-service code scanning
Why? Automated static analysis scanners can catch common coding mistakes and bad practices. This is the only way to cover large code bases, and it backstops code reviews because reviewers won’t waste time on superficial and subtle coding problems. It fits naturally into workflows and results can be fed directly back to developers.
How? Focus on high-risks and high-fidelity rules. Accuracy and speed is more important than completeness in Continuous Integration/Continuous Delivery. Tools include:
- Java - SpotBugs + FindSecurityBugs plugin + FBContrib plugin
- .NET - Puma Scan, Roslyn Security Guard, and Microsoft’s DevSkim
- SonarLint IDE plugin for various languages
- Ruby on Rails - Brakeman
- Docker - Hadolint
- Git Pre-commit checks to catch secrets, etc. on check in: Talisman, Yelp’s pre-commit, git-secrets
- Git tools to scan reports for secrets: TruffleHog, git-all-secrets
- A list of static analysis tools
While Jim’s quick wins reduce risks and get the ball rolling, he emphasizes that, “deep, real change to culture and how people work takes a l-o-n-g time, patience, and reinforcement.”
If you want to hear Jim’s full talk at the 2018 Nexus Users’ Conference, held in June, you can watch it here. All of the sessions are available here.
