Are You Testing The Quality Into Your Software?

The test team shouldn’t have the onus to improve the software quality, rather the quality should already be built into the software.   A few subtle indicators can reveal if the quality isn’t being built into the software.
 
Prior to a production release, it’s not uncommon for management to have the test team “sign off” on the release, signifying the software meets requirements and quality expectations. This sign off comes at the end of the project, putting the burden on the testers to catch all the problems introduced into the software on its journey leading up to test deployment. Instead, signoffs should happen in a regular series throughout the software development lifecycle.

Short Comings of Modern Software Development Processes

Usually, problems are introduced into the software due to inadequate, broken, or non-existent processes. Not even with the use of modern-day software development processes can you be assured you are building a quality product.
 
Using Test-Driven Development (TDD), where you write and execute tests before you write code, the ability to write good unit tests is essential, a point often overlooked.
 
With DevOps, people are one of the main reasons why it fails. Collaboration is an essential trait. People are unable to bridge the gap between teams. Collaboration and communication across the teams can result in human, non-technical obstacles. Other issues include the struggle in hiring talented people along with challenges of tool selection, test coverage decisions, and integration.
 
Utilizing Continuous Integration and Continuous Delivery (CI/CD), where developers and testers work together validating new code, faulty tests and automating the wrong processes are often the cause of problems.   This can redefine CI/CD to mean Continuous Inaccuracies/Countless Defects

Determining if you are Testing Quality Into Your Product

Poor quality is most obvious by the number of defects written after the software is deployed into test. Regardless of the severity though, defects that cannot be deferred just emphasize the problem.  As a guideline, studies suggest that one defect per 1000 lines of code is generally considered as a sign of a product with good quality.

Subtle signs of poor software quality can include:

• Defects previously fixed and verified are once again an issue, usually caused by deployment overwrites
• Regression testing returns errors.  Reasons for this may be due to code complexity, inexperienced developers, or carelessness where a code update in one area causes a problem in another area
• After the software is in test, a team dialogue is required to agree upon the expected behavior.  This may be the result of non-existent, incomplete, vague, or contradictory requirements
• When defects are found while executing the optimal scenario, a.k.a. the Happy Path functionality.  Issues found in the sunny day scenario are always a warning sign of poor quality.  If the optimal path through the software has defects, you can be assured that the reasonable and unlikely paths will also be problematic
• Lastly, your customers are reporting the problems with your software. Issues are evading your last line of defense, the test organization

Solution

Several best practices are available to build quality into your product, including:

• Testers, developers, and other stakeholders thoroughly review and understand requirements, ideally written in the SMART philosophy, and then sign off on its acceptance
• Software designs and code walkthrough inspections, traceable back to the requirements, applying industry-approved guidelines, and signed off by senior-level team members
• Utilizing code complexity tools to identify code too complex, prone to errors, or difficult to maintain. Redesign until acceptable
• Test strategies and test cases are traceable back to the requirements approved and signed off by developers and senior-level testers
• Unit tests approved, executed, and signed off by developers
• Integration tests approved, executed, and signed off by the developers
• Team agreement on tests to be automated

Conclusion

No matter which software development process you use, it’s imperative to integrate quality verification steps to ensure quality is built-in into each deliverable. Move from TDD to QTDD; DevOps to QDevOps, and CI/CD to QCI/QCD.

Preventive action is always better than corrective action.  As with sweeping stairs, you always start from the top and work your way down.  The same goes for software quality, institute quality from the start.
Just as no one can whistle a symphony alone, quality is everyone’s responsibility.