[SOLVED] Cse 565 design of experiments assignment

Purpose
Through this project, you will demonstrate your understanding of DOE principles, your ability to utilize
appropriate tools, your proficiency in generating effective pairwise combination test cases for mobile
application, and critically analyzing the effectiveness of using an AI-powered language model in DOE
software testing.
Objectives
Learners will be able to:
● Apply design of experiments to develop tests
● Research and identify a suitable tool for DOE testing
● Develop pairwise combination test cases by using two (2) different approaches : using the
DOE tool and utilizing an AI based tool (e.g. ChatGPT)
● Analyze the effectiveness of AI-powered language models for testing in comparison with
traditional testing methods.
Technology Requirements
● ChatGPT
● DOE tool
Project Description
This assignment is focused on deriving test cases from an advanced specification-based testing
technique called Design of Experiments (DOE) which takes a more systematic approach, employing
statistical methods to design test cases and comprehensively analyze the effects of multiple input
factors on the system’s behavior.
1
Directions
You are asked to develop two (2) different sets of test cases using pairwise combination Design of
Experiments (DOE) technique by employing the use of generative AI, and a DOE tool of your choice.
Below is the specification of the application that you should use when creating the test cases.
Assume that your team has created a new mobile application which requires interaction with a
customer to collect user input. The application will require a customer to enter five (5) different types
of user inputs with two to five different options.
Below is the table showing the specification of the mobile application
Type of
Phone
Authenticatio
n
Connectivity Memory Battery Level
iPhone 14 Fingerprint Wireless 128 GB < 20 %
iPhone 13 Face
recognition
3G 256 GB 20 – 39%
Galaxy Z Text Password 4G LTE 512 GB 40 – 59%
Huawei Mate 5G Edge 1 TB 60 – 79%
Google Pixel 7 80 – 100%
Task 1
You are asked to develop a set of test cases using pairwise combination DOE technique using a DOE
tool. You are expected to research and identify a DOE tool that you will be using to create pairwise
DOE test cases for testing the application that is specified above.
Task 2
You are asked to develop a set of test cases using pairwise combination DOE technique using a
generative AI tool.
CSE 565: Software Verification and Validation 2
Design of Experiments Assignment
You are expected to research and identify a generative AI tool (e.g. ChatGPT) that you will be using
to create pairwise DOE test cases for testing the mobile application.
Task 3
You are asked to compare and contrast the two sets of test cases created in Task 1 and Task 2 and
analyze how well they perform in creating test cases to validate the mobile application.
Task 4
You are asked to assess the effectiveness of the DOE tool in creating test cases using pairwise
combination DOE technique. Your assessment should explain your experience with the tool and
discuss its performance in terms of the following topics:
● Features and functionalities of the tools
● Scope covered by the tool
● Performance of the tool
● Ease of use
Task 5
You are asked to assess the effectiveness of the generative AI tool in creating test cases using
pairwise combination DOE technique. Your are asked to:
● Describe your experience with the AI tool in terms of your experience in creating prompts and
processing the results generated by the prompts
● Discuss the significance of using generative AI in software testing using DOE technique
Submission Directions for Project Deliverables
This assignment requires submission of one (1) deliverable: You will submit a report in PDF format
including:
1. Explanation of the test cases by DOE tool – The report should describe the test cases
created by the DOE tool with evidence (i.e. screenshots) showing the test cases created by the
tool
2. Explanation of the test cases by generative AI tool – The report should describe the test
cases created by the generative AI tool, the prompt you entered when interacting with the AI
tool and the result generated by the AI tool. Include screenshots of the prompts and results to
further explain.
3. An assessment of the test cases – The report should include an assessment of the two sets
of test cases developed by the DOE tool and the AI tool in terms of their validity and discuss
how well the test cases align with the guidelines of DOE technique.
CSE 565: Software Verification and Validation 3
Design of Experiments Assignment
4. An assessment of the DOE tool – In your own words, assess the effectiveness of the DOE
tool in creating test cases using pairwise combination DOE technique. You are expected to
explain your experience with the tool and discuss its performance in terms of:
○ Features and functionalities
○ Scope covered by the tool
○ Performance of the tool
○ Ease of use
5. An assessment of the generative AI tool – In your own words, assess the effectiveness of
the AI tool that you used in creating test cases using pairwise combination DOE technique. On
the basis of your findings, you are expected to explain your experience with the tool and
discuss the significance of using generative AI in software testing using DOE technique.
Different DOE tools and generative AI can give different results. You need to evaluate them and see
which one is better for the given task, and use that tool.
When ready, title your file as yourlastname_firstname_CSE 565_DOEAssignment and submit at
the assignment submission page. You are expected to submit
● a file in PDF format
Submission Guidelines
You may submit your deliverables as many times as needed. However, only the most recent
submission will be graded.
You must submit your assignment file in the designated submission space. Learners may not email or
use other means to submit any project for review, including feedback, and grading.
Evaluation
Your assignment will be evaluated based on the criteria (worth 100 points total):
1. Explanation of the test cases by DOE tool (10 points)
2. Explanation of the test cases by generative AI tool (10 points)
3. An assessment of the test cases (20 + 20 points)
4. An assessment of the DOE tool – (20 points)
5. An assessment of the generative AI tool – (20 points)
CSE 565: Software Verification and Validation 4
Design of Experiments Assignment