The Evolution of the DevOps Quality Management Office
The DevOps quality management office is a function to drive testing efficiencies in DevOps initiatives. It is composed of a set of processes, tools, and competencies to reduce test execution time between development and operations.
The prime goal is to optimize QA efficiencies upstream, downstream, and centrally, so it is basically playing the role of an accelerated bridge.
DevOps itself is still evolving as a movement, and as teams adopt it, they will see the need for increased continuous quality along with continuous delivery. Here are some critical guidelines to build this function.
A shift-left and shift-right assessment framework: This is a DevOps-centric automation readiness assessment. The primary objectives are extending the QA scope to the developer and operations world and shortening test cycle time. The output of the assessment is a transformation roadmap and business case. Below is a sample of a feasibility assessment:
A tool competency framework for continuous delivery: This is an extension of the industry standard continuous integration methodology and is focused on automation of the development to operations lifecycle. Below are tools and test team competencies required for every component:
Test team competency
Make, GNU Make, Apache
Test lifecycle and defect management
QC, Rally, Jira
Parasoft, HP Service Test
Application, server, database monitors
Infrastructure as a code provisioning
Model-based testing workbenches: Model-based testing is a methodology to accelerate the creation of test scripts. The tools let users describe the system under test using process models, which then automate the generation of test cases. The amount of testing effort can be reduced by as much as 30 percent.
Optimization: For test cases that are executed manually, scientific techniques such as orthogonal array testing and risk-based testing are effective to minimize the amount of test execution, with minimal sacrifice on coverage.
Estimation: The estimation function focuses on having a common widget of measurement across the QA function, which will be continuously improved.
Lean technique and agile adoption: Using lean and agile techniques such as kanban, value stream mapping, and a dependency structure matrix can minimize the development-to-deployment cycle.
Close alignment of key performance indicators: DevOps aims to reduce development to deployment time, and the DevOps quality management office (QMO) should have the same goal. Below is an example of how you would track key performance indicators:
Increase in number of releases
Reduction in test cycle time
Reduced cycle time due to:
In a nutshell, speed is of the essence, and quality cannot be compromised. When you start delivering your DevOps quality management office initiative, the measurement starts instantly. Celebrate success and learn from mistakes.