AI testing, a shift in the paradigm?

In the constantly evolving landscape of IT, the integration of Artificial Intelligence (AI) continues to impact the industry, and one domain experiencing a significant shift is software testing. As text-based AI emerges as a powerful tool, its influence on the market and the future of software testing becomes increasingly apparent. In this blog post, we will delve into the subject of AI, the effects of Chat AI on software testing, and explore how the future of software testing may appear.

The Paradigm Shift?

The agile landscape of testing is witnessing a paradigm shift with the advent of Chat GPT. As technology advances, so does the demand for more efficient and effective testing methodologies. Many present Chat GPT itself as a solution by automating the creation of automation code and writing test cases. However, in other fields, Chat GPT is still immature for complex testing, as my colleague (Andreas Cederholm) and I agree that AI cannot see different aspects such as security, code maintenance, and adaptability.

Another significant aspect that is a growing project for many organizations is WCAG. WCAG stands for "Web Content Accessibility Guidelines." We humans understand that some people have different disabilities and could have trouble reading or seeing certain colors, for example. Therefore, we created a set of guidelines so everyone can have a pleasant experience online. AI does not have an understanding of this issue and will, therefore, not take it into consideration.
A senior tester can immediately see flaws and potential risks to Chat GPT, as it does not have a complete picture.

Educational Evolution

‍As of today, Chat GPT has become a central function in many aspects of our daily lives. The impact of Chat AI on society extends to the realm of education. As AI becomes a more integral part of organizations, there is a growing need to educate professionals and the coming generation on how to interact with AI effectively. Future education will likely emphasize the importance of asking the right questions to elicit accurate results from Chat GPT. This also connects back to software testing as a profession. Professionals will need to hone their skills in crafting queries that yield comprehensive testing scenarios. Even if AI won’t be ready to be completely self-reliant for an extensive period, we can still use it today if we ask the right questions. The AI of today is not a complete solution but a versatile tool that can let professionals work on more complex issues and work efficiently with more impact in the workplace.

Testing as a Field

‍Traditionally, software testing has been a labor-intensive process, with testers manually executing test cases and evaluating the product. However, with the innovation of test automation, many organizations created good continuous evaluation for their systems, which often runs during evenings or low network traffic. As time passed, the testing and product evaluation process became more agile, and new methodologies emerged. Methodologies are undergoing a paradigm shift. Automation-driven testing tools are designed to autonomously perform tasks that were once time-consuming and error-prone for human testers.

AI in testing fields has not taken off the way many people originally anticipated. This is because it lacks an understanding of the complete testing process:
Is it working as the ticket/story described? Does it pose a security risk? What could be impacted in the system? How will users change behavior?

However, one significant advantage of AI in testing is its potential ability to handle repetitive and mundane tasks efficiently. The AI can be trained to do a large variety of documentation where the tester's input is short sentences, and the AI will fill it out. Later, the tester just needs to review the documentation. The purpose of using AI is freeing up human testers to concentrate on more complex aspects of the application. This not only speeds up the testing process but also minimizes the likelihood of overlooking critical issues due to testers being occupied with mundane tasks. Moreover, AI testing tools excel at pattern recognition and anomalies in large datasets, allowing for the detection of defects that might go unnoticed. Machine learning algorithms can analyze historical data to predict potential areas of concern, helping testers prioritize their efforts and focus on the most likely sources of problems.

To effectively utilize AI, you need a large amount of data/data sets for it to learn. It also needs strict rules to follow and instructions for the jobs it is supposed to execute. For an organization to maximize the impact from the AI, the rules that it has and goals need to be created with expertise that knows how and what to focus on to maximize the evaluation process of the product/system.

‍