The campaign for agile and DevOps teams to shift left is about moving critical testing practices earlier in the development lifecycle. 

Many testing activities occur late in the cycle, where it takes longer to figure out problems and costs more to fix them. When you wait to perform testing practices later in the development cycle, your nonfunctional business requirements in particular, such as security and performance testing, are so fundamentally ingrained in your code that all you can really do is patch them up rather than fix them properly.

Shifting left is about doing this identification and prevention of defects sooner.

Detecting and Fixing Software Defects

The shift-left testing strategy is well illustrated in the somewhat famous graph from Capers Jones, below, which shows the increasing cost of bugs and defects as they are introduced into the software at each phase of software development.

The first part of the graph shows that the vast majority of bugs come in during the coding phase, which is to be expected.

Whether they make mistakes, misunderstand the requirements, or don’t think through the ramifications of a particular piece of code, developers introduce defects as the code is produced. Defects are also introduced into the application when it's time to fit the pieces together, especially if multiple teams are involved (and as modern architectures like microservices get more complex). 

Now let’s overlay onto the same graph the line that shows when defects are found. Notice that it is basically an inverse of the first line:

This also isn’t surprising, because typically you find bugs when you start testing, and it can be difficult without a proper infrastructure to begin testing before everything is ready.

But what we also see here is that while bugs are mostly introduced during coding, they are almost never found at that phase. What does it cost to fix these bugs? 

It becomes important to understand the difference it costs to fix defects at each phase of development. This is represented with a third line:

Now it starts to get really interesting, as we see a nasty progression of cost that dramatically increases the later the defect is found. Letting a bug sneak through to system testing is forty times the cost of finding it during coding, or ten times more costly than finding that same bug during unit testing. And it gets ridiculously expensive when you look at the numbers for letting bugs slip through to the actual deployment.

There are a few reasons for this cost escalation:

• The time and effort it takes to track down the problem. The more complex the test case is, the more difficult it is to figure out which part of it is the real troublemaker
• The challenge of reproducing defects on a developer’s desktop as dependent systems like databases or third-party APIs are brought in (it’s common for organizations to experience a lag of several weeks between defect detection and defect remediation in these situations)
• The impact of the change that is needed to fix a defect. If it’s a simple bug, it doesn’t matter so much, but if you have it in many places, you’ve used the wrong framework, or you’ve built code that isn’t scalable enough for the expected load or that can’t be secured, it’s a larger problem

The Reasons behind Shifting Left

Now look at the orange line on the graph below, as it illustrates a proposed defect detection cycle that is based on earlier testing. You can see the orange detection curve growing larger on the cheap side of things and smaller on the expensive side, giving us a pretty significant cost reduction:

This shift left relies on a more mature development practice, such as one based on the software testing pyramid—developers create a set of unit tests that cover the code reasonably well, and functional testers and API testers do as much as they can and minimize reliance on late-cycle testing, so you have just enough manual and UI tests to prove that everything is working. This way, the late cycle tests are there to prove functionality, not to find bugs. “Test early, test often” is the mantra of teams shifting left.

Some organizations stop at this point. But you get even more value when you push even further left, into coding itself. After all, this is where bugs are introduced, so let’s start looking for them while development is still working.

This is where we benefit from static code analysis. You can start finding bugs during the actual coding phase, when the cost of finding bugs is as low as it can get.

Finding defects before testing begins is not only the most cost-effective, but the most time-effective as well, because it doesn’t leave developers with any issues trying to reproduce bugs or understand the failures. Being able to shrink a defect remediation cycle from days or weeks to hours or minutes is tremendously helpful. 

Applying the Shift-Left Approach

So, how do you shift left? For the sake of brevity, the shift left testing approach breaks down into two main activities: applying development and testing best practices, and leveraging service virtualization to enable continuous testing.

Doing earlier-stage development practices, such as static code analysis and unit testing, helps you identify and prevent defects earlier in the process. It’s important to remember that the goal is not to find bugs, but to reduce the number of bugs, especially those that make it into the release. Ultimately, creating fewer bugs in the first place is far more valuable than finding more bugs—and it’s a lot cheaper. See the graph below, with the lovely reduced bubble on the left.

Coding standards are the software equivalent of engineering standards, and they are key to reducing the volume of bugs (in addition to finding bugs earlier) and getting the most value out of your shift-left initiative. Coding standards help you avoid bad, dangerous, or insecure code through static code analysis. 

For software security, it is especially important to harden your software. You want to build security into your code, not test it in. Coding standards let you build a more secure application from the beginning (i.e., making it secure by design), which is both a good idea and a requirement, if you’re subject to regulations like GDPR.

Next, you must take the tests that were created at all stages of the development process, including the later stages, and execute them continuously moving forward. It is critical for teams that are adopting agile development practices to provide continuous feedback throughout the development process. Unit tests can easily be executed continuously, but shifting left the execution of later-stage functional tests is often difficult due to external system dependencies. This is where you can leverage service virtualization to enable continuous testing.

Service virtualization enables you to simulate dependent systems that might have limited availability, such as mainframes, third-party services, or perhaps systems that just aren’t ready yet. By simulating them, you can perform functional testing without having the whole system available, and you can shift test execution left all the way to the development desktop.

In terms of performance testing, service virtualization enables you to test before everything is ready, and without having a complete lab of everything in the system. You can even run all kinds of what-if scenarios, like what if the appserver is fast and the database is slow (something difficult to make happen in the real world)? Or what if my server starts throwing funny errors, like a 500 error—how will that affect system performance? You can push the system as hard as you like and beyond, and do it as early as possible. 

Similarly, you can start doing your security testing earlier. Decoupling from physical systems allows you to do something even more interesting: make the simulated systems behave in an evil fashion. Instead of just poking at your system for tainted data and distributed denial-of-service (DDoS) attacks, you can have a system flood you with packets, send malformed data, or any of the many other exploits commonly used by attackers. So not only can you test earlier, but you can also test much deeper than is possible with a test lab or production system. 

Avoiding Mistakes and Traps

One danger in shifting defect detection into the coding phase is accidentally putting too much testing burden on the software developers. The important thing to remember as you look at the graph is that while the cost of defect remediation gets drastically higher as you go right, the resources on the left have possibly the highest cost of any in the software lifecycle—not to mention that you are taking them away from focusing on developing functionality.

You don’t just want to find defects earlier; you want to decrease the number of defects you’re putting into the application in the first place.

And there’s another trap: If you were rewarding people for finding and fixing bugs, now they will find fewer—which is actually what you want, but only if you did reduce the number of bugs you’re introducing. Measuring the number of defects that make it into the field is probably a more useful metric.

Improving Your Process and Product

By leveraging modern software testing technologies, you can achieve software that is safe, reliable, and secure. By shifting testing left along the software development lifecycle, you can reduce the cost of testing by finding bugs earlier, when it’s cheaper, while also reducing the number of bugs you put into the code in the first place. Try this approach to save time, money, and headaches.