AE3SQM Coursework 03
TDD to create RT Harness (20%)
Learning Aims:
This coursework is designed to give you experience doing high-quality object oriented programming using Test-Driven Development (TDD), and give you an introduction to unit testing and system testing.
Background:
You have been working at the prestigious software development company BestSoftware Inc. Since your arrival there, you have been promoted to “technical lead,” and have been receiving increasing amounts of responsibility.
BestSoftware’s Chief Technology Officer (CTO), Ms Hightech, has spoken with you a number of times about the importance of software quality and appropriate documentation. As part of this, she has decided to give you a new project:
“I want you, using TDD, to build me a test harness. The test harness will work from the command line. It will take a string of arguments, using flags if necessary, and then perform random testing on a specified program. For now, all the programs the harness will be used on will have reasonably simple, numeric, parameters.”
Your friend, Mr Helpful, was very excited to hear about this new project. Because he is a QA expert, and an advocate of not only Random Testing, but also Adaptive Random Testing, he had a lot of very helpful advice for you:
“This is a great idea. I think you should research a little about pseudo-random number generators (PRNGs), especially how to use them in Java, and the importance of the seed. It is probably a good idea to research how to scale the output of Java’s PRNG to different ranges, without causing a negative impact on the quality of the pseudo-random number sequence. Think carefully about what you’ll record in any data file your harness will generate.”
Basic/Outline Harness Interaction:
After some careful consideration, you have come up with an outline plan for how the test harness should work. It will work from the command line, and take a sequence of arguments introduced by flags. You have decided that the flags (and their arguments) needed are as follows:
-p <program under test>
-o <oracle>
-s <seed>
-n <number of test cases to generate>
-a <number of arguments/parameters the program under test takes>
-r <series of lower and upper bounds for the arguments/parameters, in a [lwr, upr) range, meaning greater than or equal to lwr, but strictly less than upr>
Assuming the harness is called TestHarness.exe, and that both the program under test and the oracle are in the same folder (and called PUT.exe and Oracle.exe, respectively), then a typical call to the harness might be:
%TestHarness.exe –p PUT.exe –o Oracle.exe –s 7 –n 1000000000 –a 2 – r 0.0 1.0 -1.0 0.0
Which would mean that TestHarness.exe would generate 1000000000 test cases using the PRNG with the seed 7; each test case would be a 2-tuple with the first number in the range [0.0, 1.0), and the second number in the range [-1.0, 0.0); these test cases would be applied to the programs PUT.exe and Oracle.exe, with the outputs from both compared to see if they are the same or not … if the output is not the same, that would mean that the harness had found a failure/fault/bug.
You have also decided that when the test harness is run, the outputs will be put into a newly created file called “TestResults.dat”. You need to think very carefully about what should go into this file, and what should happen if the file already exists.
Finally, you should be careful to build the harness in such a way that it reacts appropriately to incorrect input. Error messages or exceptions may be a good idea.
Deliverables/Submissions: Java Code:
You should use Java to produce both the Unit tests (for TDD) and the production (main) code. All the Java code (including any additional libraries) should be put into a zip file (not a package).
(Brief) Code Report
You should write a very brief report about the code used in the test harness. This is a document that would go to a maintenance team, so it should include details like algorithms used, external dependencies, known issues, etc. This should not be more than about 250 words.
(Brief) QA Report detailing testing
You should also write a brief report about the QA process followed to produce the final test harness. This should not be more than about 250 words.
(Brief) User Manual
You should prepare a very brief User Manual, no more than about 250 words. This should explain clearly to any users how to set up, prepare, and use the test harness.
Submission:
All four deliverables should be put into a .zip file and submitted through Moodle, before the deadline.
Assessment:
Submissions will be manually marked and graded, with the weighting of the parts as follows.
- – 70% for the Java code
- – 10% for User Manual
- – 10% for Code Report
- – 10% for QA Report
Deadline:
The deadline will be Monday, 4 pm, November 16th, 2015.