xLM for TestOps - Continuous Qualification for AWS/Azure
What is TestOps? How does it help you continuously qualify AWS/Azure?
Many life-science companies are asking the question “Can I move my GxP workloads into the public cloud?” Answering “yes” to this question leads to another question: “What toolset do I need to qualify the various IaaS/PaaS services that are constantly changing?”.
“TestOps is a software validation culture and practice which aims at shorter testing and validation cycles to continuously qualify any cloud IaaS/PaaS service, in close alignment with business and regulatory objectives. ”
Our customers are leveraging our robust framework that we call TestOps. TestOps provides an end to end automation for continuously qualifying any IaaS/PaaS service running on AWS or Azure.
TestOps - How does it work?
In order to illustrate our TestOps framework, a simple use case to continuously qualify AWS DynamoDB is presented. Keep in mind that this framework can be applied to any IaaS/PaaS service or a combination of services.
Step 1 - Define the Intended Use
“We test “continuously” while you operate your cloud infrastructure!”
An IaaS or PaaS service can be configured to be used in many ways to meet varying end user needs. It is important to document the intended use to cover the various scenarios the service will be utilized in. This intended use forms the basis for designing a qualification strategy.
Figure 1: Intended Use for AWS DynamoDB
Step 2 - Design the Test Cases in the Coding Framework
Conventional wisdom directs us to develop test cases outside of coding using Microsoft Word as the authoring tool. This approach is surely outdated and does not provide “hard” traceability to your code. xLM’s TestOps supports “Given-When-Then” framework wherein the validation analyst can build test cases within the coding environment (IDE). These test cases are directly linked to the corresponding test automation code by the developers. In other words, both validation analysts and developers are working within the same environment and test case definitions are not separate from the code.
Figure 2: Test Case written using Given-When-Then Framework
Step 3 - Coding the Test Cases
Once the test case design is complete, the developers code the testing scenarios which typically involves driving the browser for UI Testing or leveraging the APIs. The code that is developed is organized into reusable snippets (features, pages, utilities, etc.) for better maintainability of the test automation model. As mentioned above, the traceability between test case design and code is built into the framework.
Once the code development, code reviews, testing and quality checks are complete, the code is checked into the Code Factory.
Figure 3: Code Build and Publish
Step 4: Build the TestOps Pipeline
Once the code is published, the TestOps Pipeline is configured:
to retrieve the published code
to deploy the qualified code into the test environment
to run the test automation code to qualify the SUT (System Under Test - in this case AWS DyanmoDB)
to generate a test automation report aka “executed protocol”
to archive all the test run artifacts
Figure 4: TestOps Pipeline Stages
Step 5: Run the TestOps Pipeline
Once the Pipeline is fully configured and released by Quality, the Pipeline can be run either manually or automatically (scheduled or event based). The Pipeline run-time steps include:
Obtaining the Pre-approval to run the “qualified” Pipeline
Running the test automation code and generating the execution report
Figure 5: TestOps Pipeline Pre-deployment Approval
Figure 6: TestOps Pipeline Run History
Step 6: Review the Results
After the Pipeline run is successful, the final step is to review the test automation results to ensure all the acceptance criteria have been met. This review is documented with the Post-deployment approval.
Figure 7: Executed Test Case (Sample)
Conclusion
xLM’s TestOps Framework is designed to continuously qualify any IaaS/PaaS service (AWS, Azure, Google Cloud) cost efficiently utilizing test automation and pipelines. Such a framework can enable you to build a Service Catalog of “qualified” IaaS/PaaS services so that your development teams can consume these services on an on-demand basis. This model completely removes the qualification hurdle each time a project is deployed in the cloud.
This framework utilizes the following key six (6) steps:
Development of the Intended Use that is traceable to all qualification models
Development of the test cases using the Given-When-Then framework that is directly linked to the test automation code
Development of the Test Automation Models.
Configuring the Pipeline
Running the Pipeline
Reviewing and approving the test results