How Are CI/CD Pipelines Evolving?

CI/CD - High Speed Low Risk

From a business perspective, CI/CD ultimately gives your organization a competitive advantage. It brings together the different aspects of what it takes to build and run software from a DevOps perspective. The benefit of doing CI/CD well is that it enables rapid software delivery but also keeps the whole software development process low-risk, giving you confidence that your end users will receive a high quality product with great functionality, which is secure and free from errors. The value of enabling a CI/CD pipeline makes the case for adoption a no-brainer.

Building a Better CI/CD Pipeline

One of the biggest mistakes that I’ve seen in the creation of CI/CD pipelines is the mindset that reaching the “finish line” and getting your new code deployed somewhere fast, is the main success factor. When your pipeline appears to be deploying your software and reaching the end goal, it is tempting for your development teams to think that you’ve reached the desired outcome. However, if you want to ensure that quality and security are embedded into your product and it meets and exceeds your customers’ expectations, how you get there is just as important as actually getting there. You need to look at the pipeline ecosystem as a whole and carefully plan all the steps that need to happen before a deployment, including for example rigorous unit testing, integration testing, Quality Assurance and a DevSecOps approach to security, to eliminate vulnerabilities. This will help your DevOps teams to build the most robust pipeline possible. CI/CD is based on a set of principles that should be shared, understood and agreed to enable a way of working that, if executed properly, will enable high quality, innovative software delivery.

The main principles are:

1. Create reliable and repeatable processes
2. Automate wherever possible
3. Create version control with a shared code base
4. Embed testing and quality assurance in every step
5. Prioritise the hardest parts and avoid procrastination
6. Every team member accepts responsibility and accountability
7. Focus on the customer
8. Define “done” as when the whole team’s work is complete and the customer has the software in their hands
9. Go beyond Continuous Integration, Continuous Delivery and Continuous Deployment to achieve Continuous Improvement.

One of the challenges I see for a lot of our clients' teams is that, over time, they become over reliant on their tools and forget the principles. It’s important to remember that the tools are only in place to enable the principles in action. To gain a competitive advantage, teams need to adjust their mindset and focus on these principles. Introducing Continuous Improvement is an excellent way to challenge ways of thinking and advise new, better, behaviours and practises, enabling a highly automated, secure pipeline that rapidly delivers the kind of quality software that genuinely delights end users.

Keeping Your CI/CD Pipeline Secure

There are many great tools out there that can be integrated into the CI/CD pipeline to conduct security testing for you. However, one of the biggest security considerations when building a CI/CD pipeline is the attitude of the developers. Unfortunately, in some instances, developers overlook the importance of security in favour of increasing the ease of working on the pipeline and providing quick access to other developers. The pipeline is a high-risk threat vector to a company, so to prevent compromising data and to mitigate potential risks, it’s helpful to treat the pipeline like any other customer-facing service and then apply the same IT Security policies to it. Implementing DevSecOps by helping your DevOps teams build a security first culture where security is a shared responsibility and is embedded into your code repository, production environment, test environments, automation strategy and every other aspect of the software development lifecycle, will help you achieve this. Companies should also frequently update their Information Security Management System (ISMS) and, as part of that, include role-based permissions around the CI/CD pipeline and infrastructure that it has access to. Ultimately, security has to be a top priority.

 The CI/CD Skills Gap Challenge

One of the biggest challenges facing CI/CD today is the skills gap. With a growing number of CI and CD Tools on the market, Continuous Integration and Continuous Deployment are more accessible than ever. CI/CD Pipelines are becoming easier to set up, without requiring much knowledge or experience. The problem then is, when developers don't find the right CD tool to do a particular job, that job can get missed from the pipeline, which can reduce the quality of controls, or require manual workarounds.

It's really important that developers understand it's their CI/CD pipeline. They are accountable for it and have to improve and develop it just as they would with any other software delivery. They need an attitude of relentless automation so they can solve problems themselves and automate steps that don't have any existing tools available. You need multi-skilled engineers.

To combat these challenges, when hiring, look for those with the aspiration to learn new aspects of the CI/CD pipeline. While you can hire individuals with specialisms, it needs to be made clear that their knowledge needs to be shared among the wider group. Not only does this help to avoid bottlenecks if certain issues start to stack up in a particular area, it also helps to advance the team’s overall understanding of the whole CI/CD process, while improving the entire CI/CD pipeline.

How are CI/CD pipelines evolving?

Previously developers owned and set up all the Continuous Integration and Continuous Deployment infrastructure themselves. The modern approach is not to build the CI/CD pipeline from scratch. The most successful teams are developing robust, sustainable and cloud native CI/CD pipelines through cloud providers like GitLab, Bitbucket or Github. As the number of tools and third-party integrations available in the market-place increase, we need to be flexible and open to changing the processes we use to create, execute and improve the CI/CD pipeline. Whereas previously software engineers would collect results and store the data of each step in the build ourselves, we are now relying on third party services to handle this aspect, enabling us to focus better on the new code, the end product and the end user. While these third-party services increase the speed of delivery, I do have concerns that it can sometimes further shift DevOps teams' and other stakeholders' attention purely onto development and the rapid shipping of code, at the expense of the Operations aspect of DevOps. To assuage this concern, when the Catapult CX team implements a CI/CD pipeline for a client, we create service dashboards with observability, real-time data, and metrics denoting the service level. This helps to ensure that Operations are an integral part of the feedback loop and improves the efficacy of DevOps and ultimately the quality of the product delivered to our clients’ customers.

Craig Cook, Principal Engineer, Catapult CX

DevOps Institute Ambassador