QA Automation Testing with AI
In the past decade, software development has experienced a significant transformation. Thanks to Artificial Intelligence (“AI”) and Machine Learning (“ML”) models, there is an increasing number of debates around the benefits of leveraging these technologies. The quality assurance industry is no different. There has been a substantial amount of research collected to analyze whether artificial intelligence can be beneficial for test automation. So, what are the basics of Quality Assurance (“QA”) automation and how can AI be leveraged?
QA Test Automation
Software test automation is the practice of testing software using automated test scripts using any programming language and test frameworks. These test scripts can be executed multiple times during a typical cycle of software development. When using automation, the software test engineers follow a test plan and use manual tests to generate automated test scripts. These test suites are then executed before a software version is released in the production environment.
Determining the right automation tool and framework has been crucial in the past, but developing a strategy for software testing with artificial intelligence has become a new topic of interest in the QA industry.
It is important to note that although there are opportunities to save time and improve quality, artificial intelligence should be considered as an assistant, helping testers with the existing tasks rather than a replacement.
Leveraging AI in Test Automation
Test Case Design
When dealing with the software application under test, QA engineers can leverage available AI products to facilitate the design of test cases for a release. The complexity of designing a test automation framework can become challenging, especially considering the size and scope of the application. AI tools in the market provide valuable resources, aiding QA professionals in creating comprehensive and effective test cases tailored to the unique demands of the software, ultimately enhancing the efficiency of the testing process.
Test Data for Automation
The process of generating test data often proves to be complex. It involves the ongoing challenges of maintaining and scaling test data, with a critical need to reset states after each test cycle. Addressing these complexities, AI tools can be employed to streamline the generation of mock data. By learning patterns utilized in test data and automation scripts, AI contributes to more efficient and adaptive test data generation processes.
Test Suite Optimization
Artificial intelligence and machine learning tools offer a significant advantage in swiftly analyzing data, leading to increased efficiency. Quality assurance engineers and software development engineers in test (SDET) can harness the power of AI tools to optimize their test suites effectively. This involves identifying and addressing flaky tests, and eliminating redundant code, thereby streamlining the testing process for improved overall efficiency.
Evaluate Code Changes
Given the prevalent adoption of the daily release strategy by numerous software companies, it is crucial for QA automation to align with the swift pace of code alterations. Rather than relying on conventional communication methods, AI can be harnessed to identify and report changes in the dev repository. This proactive approach helps testers promptly initiate the creation of test scripts tailored to the latest code iterations.
Update Elements and Components
In the realm of web and mobile applications, a considerable portion of the effort is dedicated to managing evolving components and test identifiers. Engineers face the task of recognizing and modifying these elements and user interface components. Leveraging AI tools proves advantageous in detecting alterations to specific components and elements, contributing to the scalability and reliability of the test automation framework.
Disadvantages of Using AI for Testing
Exploratory Testing
One of the biggest benefits of having a solid QA team is to perform exploratory testing for software applications. This requires time and human experience of using applications and predicting user actions. AI is a long way from performing such tasks, and hence human intervention is crucial when it comes to finding edge cases.
Complex to Train
Integrating AI into existing QA processes can be complex and may require specialized knowledge and skills, leading to challenges during implementation. Teams may face a learning curve while adapting to AI tools, impacting productivity initially as team members familiarize themselves with new technologies and methodologies.
Could be Prone to Bias
If not carefully trained and monitored, AI and ML learning models may exhibit bias, leading to skewed testing outcomes. This is particularly important in scenarios where fairness and impartiality are critical.
Limited Scope and Human Intervention
Not all testing scenarios may be suitable for AI automation. Some complex or highly specialized tasks may still require manual testing or other traditional approaches. Over-reliance on AI for test automation may lead to neglecting the importance of human intuition and creativity in identifying unique testing scenarios and edge cases. While these automation tools will run the test cases and provide the results, businesses still need professionals to understand how to use these tools and make sense of the output.
In 2022, there were 110 AI-related legal cases in the US, according to the AI Index Report 2023. The number of AI incidents and controversies has increased 26-fold since 2021. Moreover, only 20% of companies have risk policies in place for Gen AI use, as per McKinsey research in 2023.
Although AI is a buzzword now, businesses need to carefully assess their requirements, data, and needs and identify challenges before implementing AI solutions for test automation. As AI continues to evolve, there are significant benefits but only as an assistant to the QA engineers. Time can be saved by using AI to detect errors and improve quality, but one has to remember that human test engineers cannot yet be replaced by AI, especially for complex scenarios and edge cases.
Sheekha Singh (Author of the book, ‘The IT Girl’ )
Listed in the Agile Testing Days’ list of 125 awesome testers to keep an eye on the year 2017.
References:
- https://www.mdpi.com/2673-4591/20/1/12
- https://dl.acm.org/doi/10.1145/3616372
- https://www.forbes.com/sites/forbestechcouncil/2023/05/25/how-ai-is-changing-automation-testing-in-quality-assurance/?sh=3a7efd5742de
- https://www.tricentis.com/blog/10-ai-use-cases-in-test-automation?utm_source=google&utm_medium=paidsearch&utm_campaign=Blog_Search_DSA_High_AMS_EN&utm_term=&gad_source=1&gclid=EAIaIQobChMIlYPdjZXRhAMVOTbUAR3r1AoGEAAYAyAAEgJ0xPD_BwE
- https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2023-generative-ais-breakout-year
- https://aiindex.stanford.edu/report/
