Designing a quality approach for the internet of things requires morphing some of the best practices of test automation. The diversity of IoT applications, devices, and services translates into massive growth in your test matrix, as well as rethinking the structure of your test lab.
Here are three recommendations for expanding your test strategies to ensure high-quality IoT applications.
In typical IoT applications, there is the peripheral component (the wearable, insulin pump, smart fridge, etc.), a mobile application, and an entire back-end architecture designed to support the data distribution. The different components will have different innovation cycles and compliance requirements, so they require different testing approaches.
For the peripheral, hardware and firmware testing is usually done by the embedded testing team with traditional methods. Firmware testing is typically done with homegrown simulators. Note that some peripherals, specifically in the health care space, are not allowed outside the logical and physical premises of the vendor until fully approved by the FDA. The peripheral also needs to function correctly independent of the rest of the architecture—for example, an insulin injection must work when the unit is disconnected from the mobile app or back end.
The mobile application testing is split into two areas: typical mobile app testing, acting as if it isn’t an IoT-related app, and testing the app behavior in the presence of different data from the peripheral and the back end.
When testing a mobile app in the context of the IoT, the role of the app is to provide information and guidance to the user, as well as facilitate information flow from the peripheral to the back end. As such, you should test two areas: the reliability of the connection between the peripheral and the smart device, typically via Bluetooth, and the behavior of the application’s given data sets.
Your test lab needs to support interaction between the simulated peripheral and the smart device, using a large variety of data sets in an automated fashion. Service virtualization makes this process much quicker and more feasible.
Once simulated data is assembled, you should maximize coverage in the ever-shrinking mobile application release cycle. The first thing is performing test automation across repetitive function test cases. Modern technologies enable significant test automation coverage to drive the majority of executions.
Given the frequent executions, the next concern is the ability to collate and organize the test report data in order to minimize defect detection and analysis time.
Last is taking an agile approach to quality activities. Failing early lets you detect and remove bugs from production early. As such, allocating a certain set of test cases for the smoke test to run after each code commit, or every hour or so, is a good start. Then, run a nightly regression test across a larger number of devices, and so forth.
IoT apps expand test activities to more devices, test cases, and compliance requirements. Handling this expansion while maintaining high quality can be done with planning, innovation, and careful execution.
Amir Rozenberg is presenting the session Testing in the IoT Era at STARWEST 2017, October 1–6 in Anaheim, California.
