2022: A Year of Significant QA Transformation at LoginRadius
how we transformed QA to support engineering efforts and deliver a more robust and efficient platform free of production issues for our customers
In 2022 alone, LoginRadius CIAM platform has undergone major changes as we focused on re-engineering our platform to deliver cutting-edge performance, scalability, and throughput — which we needed to deliver for our customers who had large bases of end users for whom seamless experiences supported by lightning performance was mostly non-negotiable.
The changes and upgrades in 2022 were needed due to the challenges we faced in 2019 and later; for which, by 2021, we did root cause analysis, thorough research, and strategically planned high-quality upgrades.
In this process, we needed to transform our QA systems, processes, and workflows.
I wrote earlier about this engineering transformation at LoginRadius: 2022 - A Year of Engineering Resilience at LoginRadius.
Here, I write about how we transformed QA to support engineering efforts and deliver a more robust and efficient platform free of production issues for our customers.
Reworking Test Coverage
While doing the root cause analysis in 2021, we also identified that work on test coverage was required to meet the increased demand and ultimately lead to better user experience and customer satisfaction. Additionally, we planned test coverage extension to ensure that APIs movement from .Net Core to Go Lang does not introduce regressions.
In phase one, we enhanced test coverage across all LoginRadius components, including HPA, IDX, and Admin-Console. And we then added new test cases for the identified gaps and new scenarios. This improvement provided optimal test coverage and helped us reduce the risk of issue leakage.
Advancing Automation
Automation testing has always been part of quality assurance at LoginRadius. However, increased demand and the need for faster release cycles called for the advancement of existing automation capabilities.
We made notable improvements in automating testing for customers' sites and various product components like webhooks, auth APIs, and audit logs for HPA APIs. Also, to remarkably reduce manual efforts and cumbersome maintenance, we migrated customer site automation from Katalon Studio to Cucumber BDD. This advancement enhanced the automation capabilities and reduced the product’s release time.
Executing Parallel Pipeline
Earlier, our test cases ran in parallel, but we executed jobs in CI/CD pipelines sequentially. Thus, the time involved in release verification was higher.
In the year 2022, we added more runners and separate apps for every job in test suites and reduced CI/CD pipeline execution time by more than 50%. This execution improved our testing efficiency and helped us deliver faster product releases.
Streamlining Test Case Management
While planning and executing the above strategies, we realized that existing test case management and cross-browser testing solutions required an upgrade for better performance and scalability. To fulfill these requirements, we opted for our new tools - Testmo and LambdaTest after extensive research and POCs.
These changes have brought several benefits to the product, including more effective test case management that streamlines the overall testing process and enables geo-location execution for test scripts.
The Results
All of our continuous efforts, strategic rethinking, gap-finding analysis, and deeper involvement in optimizing the quality assurance of our platform have resulted in the following:
- Significant reduction in the number of product PIs in 2022 compared to the previous years. In-depth planning, brainstorming, and gap findings have contributed to the product PI graph trending close to zero.
- Notable reduction in issue leakage risk with better test coverage ensuring through testing of all LoginRadius components.
- Remarkable reduction in product release time, regression, and overall QA verification time led to the faster and more efficient delivery of product releases.
In Conclusion
In conclusion, 2022 was a year of significant transformation in quality assurance at LoginRadius. These changes have helped to deliver a more robust and efficient platform for customers, ultimately contributing to their satisfaction and success.
Originally published at LinkedIn
Deepak Gupta