The biggest mistakes you can make when setting up a Continuous Delivery system
So you’re setting up a new Continuous Delivery system? It should be an exciting time for your business, but it’s also an extremely important one. For all the benefits that Continuous Delivery can bring, there are many mistakes teams often make along the way that hold them back. Let’s take a look at exactly what these mistakes entail and what you can do to avoid them as you build your own Continuous Delivery system.
1) Not using existing tools
One of the biggest mistakes you can make is to go DIY with your Continuous Delivery system. This is a recipe for disaster because tools like Docker and buildpacks as well as automation systems like Ansible, Puppet and Chef were all built based on experience. There’s a reason they exist and are so widely adopted. Not only is it easier to buy these tools versus building something net-new by yourself, but when you buy them you inherently gain all the knowledge and experience that went into them – not to mention the continued innovation that comes from being widely adopted across the industry.
If that wasn’t enough, building your own toolset can create a hiring nightmare. As you bring in new ops and development team members, they’ll be starting from square one if you give them a Continuous Delivery system built around DIY tools. On the other hand, if you introduce market-leading tools, you can actually hire around those tools (e.g. by finding specialists in Ansible management), which will make both recruiting and onboarding easier.
2) Not integrating with a Continuous Integration system
The difference between Continuous Delivery and Continuous Integration is an important one, but this distinction doesn’t mean that the two aren’t connected. Quite the contrary, using the output of your Continuous Integration system to determine what your Continuous Delivery system does is an important best practice for setting up a successful environment.
For example, your Continuous Delivery system should listen for the Continuous Integration system to give the okay on a build before deploying anything to customers. If you have a system that just does delivery and doesn’t take anything else into account, then you’re not starting off in the best place and will likely have a lot of fires to put out down the line.
3) Requiring manual intervention
What if you need someone to take action manually to make a delivery go out? Or what if your system requires constant monitoring because you don’t have full confidence in it? Neither of those situations will cut it. Quite honestly, at that point it’s no longer continuous – it’s just an old-fashioned delivery system.
If your Continuous Delivery system requires any manual work or ongoing monitoring, it defeats the whole point of this approach and you might as well just do everything by hand. Instead, you need to build a system that can run automatically without any human intervention and that instills enough confidence in your team that it doesn’t require constant babysitting. Only then will you be operating in a true Continuous Delivery fashion.
4) Building a long, slow and uninterruptible delivery process
Absolutely never, under any circumstances should you build a delivery process that takes a significant amount of time or can’t be stopped once it starts – even if it’s automated and you can walk away from it. The point of this matter isn’t the time it takes to complete the delivery, but rather the time it takes to get feedback from the delivery. The longer it takes to find out that something you just added doesn’t quite work right, the worse off you’ll be.
In general, long cycle times can be a huge productivity killer and highly demotivating for working to improve the Continuous Delivery system itself. If the process takes a while (and especially if it takes a while and can’t be stopped once it starts without having to go back to the beginning), then no one will want to touch your system with a ten foot pole. And when that happens, you’ll be stuck with a subpar system that you don’t have the motivation to improve. As a result, it’s important to make sure your system is as fast as possible and has an easy recovery process that allows users to pause and resume from that same point at any time.
5) Not checking on the health of your deployment after the fact
You know that cool guy in movies who sets off an explosion, walks away and never looks back? Don’t be him. In the world of Continuous Delivery, that’s as far from cool as you can get. What you really want to do is make sure what you said was going to happen actually happened and that you didn’t just blow up a bunch of code in your customers’ faces. Doing that will make you truly awesome.
If you don’t follow through to check that what you’re doing is actually working, you’re not building a Continuous Delivery system, you’re just building a “fire and forget” system. Importantly, you need to know if what you did was successful in order to run in a fully automated environment. Along these lines, a good Continuous Delivery system will help you out by halting itself as it deploys bad code so that the blast radius is as minimal as possible. If you can build a system that halts bad code, allows you to fix it and then resumes the process with the healthy code, that will be one hell of a Continuous Delivery system that allows you to run in automation forever. An important part of achieving this goal is building alerts around progress of the delivery system (including any errors) and monitoring for any after-effects (e.g. to ensure production doesn’t crash after a new deployment).
6) Not supporting any kind of flexible application configuration
Applications must be flexible enough to get tuned and tweaked based on the configuration you provide them. This is important because you might have a staging environment and a production environment, each of which will very likely need to be configured slightly differently (e.g. since you’re not bringing customer data from production into your staging environment).
As you build your Continuous Delivery system, you need to include enough flexibility to allow your application to be configured in different ways depending on where it’s going. If you don’t have that flexibility, you’ll need to adjust your Continuous Delivery system every time you have a configuration change – and that will no doubt drive your team crazy because those configuration needs will change all the time. Ultimately, without that flexibility, updating your Continuous Delivery system to match changing configuration needs will easily become someone’s full time job, and that doesn’t sound fun at all.
7) Building a Continuous Delivery system that doesn’t support multiple environments
While you technically don’t need multiple environments, having them definitely helps – and having the ability to support them is a must. For instance, you might want a staging checkpoint that gives your QA team an environment that’s similar enough to production that they can confidently determine what they’re testing will pass muster when it gets in front of customers. This requires the ability to support multiple environments.
In general, a system that can’t support even two environments is pretty inflexible and rigid, and that will be a problem to maintain over time. Even if you don’t need multiple environments right now, you might want to do things over time like introduce a staging environment (or two!) or run canary deployments, and if your system can’t support that, building out those environments will become a huge mess.
8) Forgetting about external systems like databases
As you build your Continuous Delivery system, it’s easy to focus on application code and forget about external systems like databases – but that’s a huge mistake. For instance, if you forget about databases and someone starts tracking a new type of data in the system and deploys it all the way to production but the appropriate database schema isn’t there, then everything will blow up pretty fast.
Of course, databases are just one example of external systems for which you need to account. The same thinking applies to anything that happens externally from the application code itself as part of the delivery process, such as pushing assets to a CDN. The best Continuous Delivery systems are flexible enough to support all of these use cases and seamlessly keep these external systems in stride with code changes to your application itself.
9) Never taking into account security updates
When it comes to security, you also need to think beyond your application and its code. That’s because your application runs on a whole set of systems that have their own, independent security measures in place. A lot of teams select an operating system or container base image and lock that foundation into place in perpetuity, setting up the server for deployments once and then never touching it again. But if you only update the software that your development team writes and never touch that foundational layer, you’ll expose your organization and your users to serious risks.
Quite simply, you need to build a Continuous Delivery system that continuously checks for security-related changes to that base image or operating system and then applies those updates. Otherwise, you’re putting yourself at greater and greater risk as time goes on, because undoubtedly some insecurity will pop up that makes the system exploitable. And when it does, it’s just a matter of time until the hacker community discovers that vulnerability and the fact that your application is built on it. As a result, it’s essential to be proactive and defensive about maintaining security, and baking it into your Continuous Delivery system makes taking that stance all the more manageable.
Building a Continuous Delivery system? We’re here to help
The road to building an effective Continuous Delivery system is ripe with common mistakes, but if you know what to look out for you can easily avoid them. Working with a team who has done it before can make all the difference. That’s where Spaceship comes in. Our platform is designed to avoid these common mistakes and is built on years of experience developing best practice programs. Interested in learning more? Contact us today to learn how we can help.