The value of regression testing in shipping stable products of desired quality is undeniable.
If enough time and resources were available, a larger regression test suite may be used to cover all grounds whenever a product change takes place—in reality it is almost always a balancing act of time and risks.
The current market dynamics of faster release times, support across multiple platforms, and drive to reduce cost of quality have made regression testing critical and challenging due to the following factors:
Faster code churn resulting in more regressions. While the agile lifecycle is iteration driven and facilitates modular development for better code quality, in the interest of time available at hand, there is potential for bug fixes to be tactical, focusing just on the superficial issue rather than the holistic root cause.
Time needed to run regression tests. Unless regression testing is a mature process where the right test suite is chosen and maintained, the right balance of manual and automation testing has been arrived at, and the tests are run in parallel with ongoing development, it can be a time-guzzler and an impediment in the ship cycle.
Difficulty in isolating regressions. With complex code bases and geographically distributed teams, it is becoming increasingly more difficult to identify regressions as soon as they are introduced and the cause that triggered them.
So, where is regression testing really heading? How can the test team take the proactive seat to carve a promising future, while mitigating the challenges outlined above?
Perform root cause analysis rather than mere defect finding—giving an added perspective of the issue at hand, helping explore all possible reasons and impacted areas.
Work with the developers collaboratively to understand their areas of concern. This will promote both better testing and defect analysis, which reduce chances of regressions.
Provide automated test suites to developers and also review their unit tests to improve the quality of testable builds to minimize regressions.
Take defect finding to the next level and plan for regression testing upfront, even before the defect has been fixed. As you analyze the cause for the defect, consider including a small automated regression suite that you would use to test it, in the defect itself, encouraging the developer to use it before the fixed code is checked in.
Establish a robust regression test process to isolate issues. Whether a tester or a developer, tackle small pieces of the system at a time before integrating them with larger pieces. Such a breakdown helps in the overall debugging process and when assigning the defect to the right person—increasing the chances of a thorough fix and thus reducing regressions.
Strive for a comprehensive automated regression test suite—and not because it is a mundane and repetitive task when done manually. You can run tests unattended as soon as code is checked in, regressions can be found as soon as they are injected, and it can help reduce the overall testing turnaround time.
What are your company's regression testing practices? Are they working? Let us know with your comments below.
