Testing in the Dark
Some test efforts are straightforward about the exact test cases that need to be developed and executed to verify proper functionality and identify any adverse effects. But requirements can only go so far in identifying areas to test. At times, due to the entangled mess of the software architecture, precise test case development and execution may be ambiguous. This is especially true of legacy software that has had numerous tweaks over the years.
In these circumstances testers may be given no information at all on the areas of impact, leaving it up to them to determine what to test.
Testing all varying combinations of inputs and state permutations with limited time and resources is impossible. More importantly, needing to perform directionless testing indicates a broken software development process.
Don’t accept the need to indiscriminately test with no clear understanding. Your testing should be targeted, and using a few practical techniques will focus your test effort where it matters.
First, focus on what you need to do now to fulfill the test directive. Then create a long-term strategy to prevent this predicament from happening in the future.
Help is not a dirty word. Sometimes just asking for assistance from the developer can reveal areas to test.
In the absence of a meaningful response, take the opposite approach: ask what not to test. You may discover irrelevant tests that can be eliminated.
If these two options don’t provide enough information, try these tactics:
- If the developer has gone through a design and code review, ask the other participants about areas to focus on
- Identify optimal, reasonable, and unlikely scenarios through the application, focusing your testing on the optimal and reasonable paths
- Identify critical features
- Identify known problem areas that tend to break due to instability
- Determine functional dependencies between the code that is currently working and the new or fixed code, and test that functionality
- Share your test strategy and cases with the team (including what you’re not planning to test) and request feedback
- Set expectations about potential risks
- Test as early in the development cycle as possible
- Run your regression test suite
- Run automation where possible
Dig the well before you get thirsty. There are steps you can take to reduce the possibility you will need to test aimlessly in the future:
- Hold architectural reviews to identify overly complicated design implementations, and campaign for their redesign
- Ensure developers who are inexperienced or unfamiliar with legacy software are mentored, guided, and held accountable
- Follow configuration management and deployment processes
- Use metrics on defect density per requirement to uncover unstable functionality, and push for its redesign
- Participate in design and code reviews to get an insider’s perspective
- Ensure developers are performing unit and integration testing
- Use root cause analyses to determine where problems originate, and be an advocate for correction
- Increase automation where possible
- Ask developers to identify areas of the software to test
Whining about the lack of test direction won’t let you achieve your objectives. Instead of cursing the darkness, light a candle. Precision test techniques will help you focus your test effort.