3/10/2021 Fortnightly Task 1 – Testing 1 Fortnightly Task 1 – Testing 1
Due Mar 19 by 23:59 Points 2
Relevant Tutorials: 1, 2
Relevant Major Assessment: Testing Assignment A1
Your first Task is a basic JUnit testing task. This is intended to achieve a few things:
Get you coding in Java quickly
Revise some of the material from SOFT2201
Give you a practice run at what you will be doing for Testing Assignment A1 (both the testing content, and the github and script submission process)
You need to develop a testing suite to test the Enrolment module. This module has a few classes in a single package: EnrolmentSystem, ReportGenerator, Student, Unit, and Faculty. You should write this test suite based on the API documentation – EnrolmentAPI.zip. For this intro Task, you will also be provided with the raw code for this module – EnrolmentModule.zip. Your goal is to test this module so that you can catch bugs in its implementation. There will be 20 bugs to catch in total, that will be announced in batches of 5 at semi-regular times between now and the due date (the last batch will be released no later than Tuesday Week 3 – 16MAR).
A warning: In Testing Assignment A1 you will NOT be given the raw code. You will also not be told what the bugs are until after submissions close. If you want this Task to do its job for you and give you practice for A1, you should try and complete this assignment first without referring to the raw code or the list of bugs. If you find yourself needing these resources, then you should try and work out what you need to improve on (boundary identification, API reading, coverage, etc) so you can do well in A1.
Task 1 Bug List
Reading the API Documentation
The API is in the format you are hopefully familiar with from looking at Java native library documentation – javadocs. However, it is written with different details than the user documentation that javadocs is usually used for, and includes the technical specification of the API so you can use it to develop your test suite. This in effect provides you with the pre- and post-conditions that valid implementations of those methods/classes will adhere to. Things under the ‘pre-condition’ label and the ‘param’ labels should be considered the pre-conditions. Things under the ‘post-condition’ label and the ‘returns’ label should be considered the post-conditions, and things under the ‘throws’ labels indicate required behaviour when pre-conditions are broken. Note that the API uses strictly defensive programming – all listed
https://canvas.sydney.edu.au/courses/31635/assignments/284429 1/7
3/10/2021 Fortnightly Task 1 – Testing 1
preconditions should be tested to ensure that the defensive programming behaviour is correct (reverting to a default, returning null, throwing an exception, etc as the documentation indicates). This is different to testing code where defensive programming is not used, where preconditions can be things you should not test for.
There will not be many pre-conditions in this Task except for parameter conditions – but the pre/post format is included so you are ready for A1 (and A2) which will use this label extensively.
Ambiguous exception behaviour – if multiple preconditions are violated such that the conditions for throwing multiple different types of exception are met, then there are 2 different ways the API handles this. In certain casesthe ambiguity may be dealt with explicitly, indicating which will be thrown. In all other cases the precedence is top-down – that is the first mentioned matching exception will be thrown.
Recommended development process
(this process is not marked, but it will likely lead to the best results for you, and will be enforced in your Exam work – it’s Test Driven Development)
Stub the classes you will be testing – a stub is a class that returns the default value for every method, has no stored members, effectively does nothing and has no code inside. This is a stubbed class:
package com.example.stub;
public class ExampleStub {
public ExampleStub(String someParam, int someOtherParam){}
public String SomeStubbedMethodWithAReferenceReturnValue() {
return null;
}
public int SomeStubbedMethodWithAPrimitiveReturnValue() {
return 0;
} }
Write your test suite, targeting your stubbed implementation. The reason you write stubbed classes is because with those stubs in place you can compile your test suite – which is a fantastic way of checking that you don’t have any syntax errors.
Run your test suite – most or all tests should fail, because your stubbed classes won’t be correct based on the API requirements
Implement your stubbed classes completely. Do this step by step, based on the tests that fail. You should be continuously running your test suite and seeing the tests move from Red (fail) to Green (pass). Once ALL tests are green, you should have finished implementing the system. You will probably discover errors in your tests while you are doing this (tests are as likely to have bugs as main code!)
https://canvas.sydney.edu.au/courses/31635/assignments/284429 2/7
3/10/2021 Fortnightly Task 1 – Testing 1
Review your implementation and test suite. If you have all tests green, but you don’t think you’ve
finished implementing the classes, then obviously your test suite wasn’t finished yet. Write more tests, then implement what is missing so those new tests pass too.
Test the provided implementation to make sure your test suite doesn’t incorrectly reject it as bugged. Because of the way the marking script works, you MUST pass the working example, otherwise the marking script won’t know if you’ve caught a bug, or are just rejecting everything. Your marks (both for this Task and for Assignment 1) depend on you first and foremost passing the working ‘no bugs’ implementation.
Wait for the marking script to tell you which bugs you caught or missed. Remember these bugs will release in batches, so you might catch them all when you first commit your code, but miss some in the next batch. If you have completed the task properly you will catch all the bugs in every batch without having to update anything – missing bugs in later batches is a warning that you might miss bugs in Assignment 1 if you approach things the same way.
Now you could of course do this another way – as each batch of 5 bugs come out you can write single tests that use single specific test cases to seek out and catch those bugs. The 2/2 is exactly that easy to get, nobody is going to stop you. But your feedback will likely be the following sentence “these tests will be useless in the wild, do testing properly” and you won’t be able to do things the same way in A1, so this would be a bad idea.
Submission Process
You will not be submitting this assignment in Canvas. Instead, you will be using https://github.sydney.edu.au (https://github.sydney.edu.au) to submit your work. This is how you will be submitting all assessable code work in SOFT3202.
You should first create your account – this will be your unikey, and should be automatically created when you log in to this platform the first time.
Create a PRIVATE repository with the following EXACT name. It must be exact so the marking script can find it.
SCD2_2021_T1
In this repository you should commit your testing suite as a Gradle project, so that the root directory of the repository has the src folder, the build.gradle file, and a readme.md file (the readme file is optional but you should be in the habit of always including one). An example (but incomplete) submission is viewable at https://github.sydney.edu.au/JBUR2821/SCD2_2021_T1 (https://github.sydney.edu.au/JBUR2821/SCD2_2021_T1) .
Note that this repository is public (so you can see it). Your Task repository however must be private, so other students cannot view/copy it. In order for the teaching team and the marking script to have access to your repository you must add us as ‘collaborators’. You must add the following usernames:
https://canvas.sydney.edu.au/courses/31635/assignments/284429 3/7
3/10/2021 Fortnightly Task 1 – Testing 1 jbur2821
mmcg5982 bsch0132 ttho6664
You can include an implementation if you want, but this will not be marked – the marking script will pluck out your test classes (under src/test/java/) and ignore everything else. This includes your build.gradle file! You should use the build.gradle version from the example submission, because if you add something different (like external libraries) then your test suite might work on your computer but fail when the script tries to mark it, because the example build.gradle file is the one that will be used.
Marking Script
The marking script will begin running early in the Task. Each day at a reasonably (but not guaranteed to be) regular time at night the script will look for your repository, and attempt to mark it. You will receive feedback in this Assignment based on what the script has uncovered. This feedback will look a bit different depending on whether all of the bugs have been announced, and what step in the process your submission makes it to – this is a simple sequence:
Script cannot find your username in github.sydney.edu.au – script ends and tells you this.
Script cannot find your repository – script ends and tells you this. This either means that you have not created the repository with the right name, or you have not added the collaboratory the script is currently authenticated as.
Script checks your repository settings – it must be private, and it must have the above users added as collaborators. If these are incorrect the script ends and tells you this.
Script now clones your repository to the marking sandbox – will checkout the head of your master/main branch (the default ‘up to date’ version).
Script looks for an src/test/java folder in the root directory of your repository. If it cannot find one script ends and tells you this.
Script pulls all *.java files from this directory and any subdirectories (subdirectory/package structure will be maintained). If it finds none script ends and tells you this.
Script places your tests into a gradle project containing the working implementation provided above. Script executes ‘gradle test’ on this merged gradle project. From here 4 things can happen:
Compile error. Script ends and tells you this. The script will include the error message.
Tests fail. Script ends and tells you this (remember you MUST pass the working version, otherwise the rest of the script can’t work). The script will include the JUnit test report so you can see what tests are failing.
Very very weird error. This shouldn’t occur unless you are doing something you really shouldn’t be. The script is set up to fail fast rather than try to correct for anything like this, so if you’re doing something like calling System.exit, trying to run network code, etc the script will just terminate.
https://canvas.sydney.edu.au/courses/31635/assignments/284429 4/7
3/10/2021 Fortnightly Task 1 – Testing 1
Don’t do this. Doing this in a way that looks like you’re trying to subvert the script or gain access to systems you shouldn’t may be grounds for an academic integrity investigation or worse. Tests pass. Script continues.
The script will now do the same thing for the 20 bugs (or less if not all have been released).
For each bugged version the script will record whether your test suite passes or fails.
If your submission has made it this far you will receive a list of which bugs you caught and missed. If all 20 bugs have been released then you will also receive a grade.
This feedback will be written on this Canvas assignment.
In addition to the marking script, you will also receive some human feedback from your tutor after the due date.
Marking Guide
The marking guide for this assessment is probably the simplest you will see:
Catch all 20 bugs: 2/2
Miss any bugs, or your code fails to compile, or you reject the working solution as bugged, or you do not give correct access to the script: 0/2
This looks much harsher than it actually is – by telling you what the bugs are this Task is made very easy. We are far more interested in giving you the chance to practice for Assignment 1, and we want to spend our human time giving feedback to improve your testing rather than trying to work out what is 0.5% vs 1%, etc. You will know in advance if there are any problems here because the script will tell you if there are problems (so long as you don’t leave doing it until the last day, which would be a terrible idea!).
Final Notes
Because Task 1 feeds right into Assignment 1, you should do your best to complete this on time. Feedback for Task 1 (the human feedback, not the script feedback) is only guaranteed to be ready in time to use it for A1 for students who submit on time. For this reason if you require special consideration for Task 1 you should also apply for the same special consideration for Assignment 1 so both dates can be extended.
From time to time adjustments or clarifications must be made to assessment tasks – if any adjustments are required they will be sent as Ed forum announcements, which should also be sent to your student email address. These announced adjustments or clarifications form a part of the assignment specification and may impact your grade, so you should make sure you review these whenever they are sent.
https://canvas.sydney.edu.au/courses/31635/assignments/284429
5/7