Detect Performance Issues in Production with Continuous Delivery
You can read about the benefits of continuous integration and delivery hundreds of times, but you never quite learn as well as when you experience something for yourself.
Continuous integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect different kinds of problems early.
Continuous delivery (CD) is the ability to get changes of any type (features, configuration changes, bug fixes, and experiments) into production, or into the hands of users, safely and quickly in a sustainable way. You can work toward a culture of continuous feedback loops, getting better and better from the feedback you get from different sources.
One important source of feedback—maybe the most important one—is what happens in production. In her book A Practical Guide to Testing in DevOps, Katrina Clokie talks about how testing can add value during development and in production. It’s amazing the amount of information you can get if you properly monitor and analyze the data.
After having read and experimented with performance testing CI and CD, I was able to work with a client that runs performance tests in production (A/B testing, monitoring, etc.) and see how they used it to solve a peculiar problem they faced.
Continuing with the analysis, they were able to identify the specific code merge that caused the problem. Observing the GitHub log, they could see all the files that were changed in that merge request; three specific changes appeared as potential culprits, and from there it was easy to discern which had generated the problem.
It would have been impossible to do this if it were not for the fact that the team follows a continuous delivery strategy where it releases small changes every day. If the team had released once a week, they would have had to analyze all the changes made that week, a much more complicated and time-consuming task.
Continuous delivery allows us not only to deliver value to the end-user more frequently and get feedback from them sooner, but also to obtain more precise feedback from different sources, thanks to observing the impact of every realized change while improving traceability.