My experience using ChatGPT in the quality testing process

THE CONTENT OF THE ARTICLE:

My experience using ChatGPT in the quality testing process

Hello! My name is Diana, and I have been working as a QA manual for seven years, 5 of which are at ApexTech. In the current stage of technological development, the rapid advancement of artificial intelligence offers endless opportunities for introducing innovative methods in all areas of life. One of these areas is Quality Assurance (QA), which plays a crucial role in ensuring the functionality and reliability of software. I want to share my experience using ChatGPT during the testing process in this article.

Experiment 1: Generating test scenarios

In my first experiment, I decided to use ChatGPT to generate various test scenarios for a web application, which typically requires a lot of time and creativity.

 

After several attempts, I obtained impressive results. Even with the starting instruction "Create a test scenario for checking date and time", ChatGPT was able to generate various scenario variations, including cases with error messages and unexpected interactions with the system.
 

 

Experiment 2: Testing the reaction to incorrect data

Software should properly respond to incorrect user data. Using ChatGPT, I tried to generate various incorrect data scenarios and check how the software reacts to them.

 

For instance, I asked the model how one might break the email input form, and received a list of input options that could cause data processing errors. In this way, I enhanced the security and reliability of the form in the web application.
 

Experiment 3: Generation of Pairwise Tests

Pairwise testing is a technique for forming test data sets from the full set of input data in a system, which significantly reduces the number of test cases. In simpler terms, it's about creating data sets where each tested value of each verified parameter is combined at least once with every tested value of all other parameters being checked. This technique helps considerably reduce the number of checks while covering all the data. With the help of GPT chat, such tests can be created quickly and efficiently, saving time. For example, here's pairwise testing of a request creation form with fields such as: pipeline, amount, product, and deal owner:
 

 

As can be seen from the image, we ended up with only 5 test cases instead of 81 (3x3x3x3=81), which makes the life of a QA much easier 🙂

Experiment 4: UI Testing

In this experiment, I decided to use ChatGPT to create test scenarios and questions that would check various aspects of the interface.

 

Thanks to the model's ability to generate user interaction variants with the interface, I was able to create complex scenarios, such as interacting with various menu items, inputting data into forms, navigating between different pages, and so on. This way, I covered more test scenarios and identified unclear or unattainable issues for manual UI testing.
 

Experiment 5: Test Scenario Optimization

Time and resources for testing are limited, so it's important to maximize the efficiency of test scenarios. In this experiment, I used ChatGPT to optimize a set of test scenarios.

 

Instead of randomly selecting test scenarios, I asked the model which ones are the most critical and effective for problem detection. This made testing more focused on the key aspects of the software.

Conclusions:

My experience using ChatGPT in QA showed that this tool can be very useful, helping to save time, optimize the testing process, and reduce potential errors.

 

However, it's important to remember that ChatGPT has its limitations. It can generate plausible content, but it only sometimes understands situations related to specific business logic or specific aspects of testing.

 

Despite the limitations, ChatGPT can be a valuable addition to a QA engineer's toolkit. If you plan to use ChatGPT in the testing process, here are some tips from my experience:

• Clearly formulate tasks: Prepare clear and specific instructions for ChatGPT. This will help get more accurate results.
• Verify results in practice: It's worth verifying the answers by executing generated test scenarios or analyzing the software's reaction to the data.
• Control the process: Remember that ChatGPT can generate incorrect information, so use the model only as an additional resource.
• Creativity and experiments: ChatGPT opens up new opportunities for a creative approach to testing. Experiment with different tasks and variations to find the best ones.

Do you also use ChatGPT in testing? Share your coolest findings. Let's help each other and compile the best use cases of ChatGPT in QA.