Industrializing Testing Services for SaaS Products
Software as a service (SaaS) is a key component today of every enterprise application outsourcing strategy. A driver for cloud adoption is to reduce capital expenditure and operating costs of noncore functions, such as administrative software, with the organization focusing on core business areas.
Quality, though, cannot be outsourced. The task of ensuring that the software delivers expected quality—within reasonable costs and speed—rests with the organization. Below are three scenarios and test imperatives that organizations would do well to consider while adopting an SaaS strategy.
Scenario 1: Implementation or migration to an SaaS product: This is the most complex scenario. It involves significant customization of the standard code base to meet the organizational requirement. The degree of automation possible in cloud technology is limited, making this a very manual-intensive process. Added to this is the complexity of testing third-party applications and migrated data.
This mandates comprehensive testing of configurations, integration touch points, interfaces, reports, network security, browser penetration tests, user profile tests, performance tests, data conversion tests, and mobile and cross-browser tests. Parallel testing to validate same-period processing against legacy production applications with third-party integration and country-specific compliance tests also plays a key role.
Scenario 2: Maintenance testing: This involves testing change requests of core product and third-party applications interfacing with the product. Over time 40 percent to 50 percent of regression tests can be automated.
Scenario 3: Updates testing: Most SaaS products have frequent updates, and it is critical to test a subset of the configuration, processes, interfaces, reports, and security tests during every update cycle.
The above scenarios are typical for most SaaS products. Consequently, it can be a smart decision to industrialize the testing process by creating a simple catalog of testing types, such as the one outlined below, and indicating mandatory and optional test types in all scenarios. Over time, this list can get formalized by product, eliminating dependency on the testing process.
Test type | Implement | Maintenance | Updates |
Functional | M | ||
Integration | M | M | M |
Performance | M | O | |
Data Conversion | M | ||
Configuration | M | M | |
Regression | M | M | M |
Compatibility | M | ||
User Profile | M | O | |
Parallel | M | ||
Security | M | M | |
Reports | M | O | |
Compliance | M | O | |
Mobile | O | O | O |
UAT | M | O | M |
Figure 1: Test types by scenario
Test type | Definition |
Functional | Testing of end-to-end processes between multiple product functions |
Integration | Testing of integrated environment along with other applications. This includes some level of configuration, regression, compatibility, web services, and user profile tests |
Performance | Testing to measure stress acceptance level and response time |
Data Conversion | Testing the quality of data transfer from existing database to SaaS product |
Configuration | Testing the setup of business process configurations |
Regression | Regression testing during updates and maintenance |
Compatibility | · Testing compatibility across various browsers |
Parallel | Validating same-period processing against legacy production application and third-party integrations |
User Profile | Profile-level access testing |
Security | Validating security controls at access and connectivity level |
Reports | Testing for any strategic and operational-level reports |
Compliance | Testing country-specific regulations |
Mobile | Testing functionality on standard platforms |
UAT | Validating system by end-users |
Figure 2: Catalog of test types
Smart testing with the right blend of depth and speed is the cornerstone of successfully testing SaaS applications. Reap the benefits of SaaS products by creating an effective, reusable strategy.
