FIT9131
Week 7
Programming Foundations FIT9131
1
Testing & Debugging Week 7
FIT9131
Week 7
Lecture outline
•Software development and maintenance
•Program errors – syntax, execution, logic
•Testing
•Unit testing (within BlueJ)
•Designing and implementing a Test Strategy
• needed for Assignment 2 •Regression testing
•Debugging 2
FIT9131
Week 7
Stages in software development
(1) Design stage:
• designing the solution (2) Implementation stage:
• translating the design into code (3) Maintenance stage:
• adding new features, fixing bugs, etc 3
FIT9131
Week 7
Design Stage
Design stage:
1. Understand the problem
2. Design a solution
3. Design test data for that solution
4
FIT9131
Week 7
Implementation Stage
Implementation stage:
1. Translate these designs into code
2. Compile code
3. Run tests
4. Modify code as necessary until tests are satisfactory 5. File all documentation
5
FIT9131
Week 7
Maintenance stage
Maintenance stage:
1. Read all documentation. Understand how the current system works (or why it doesn’t work).
2. Figure out the required modification. Assess the impact of the modification upon the rest of the system.
3. Do all design and implementation steps for the change to be made.
4. Test the modified system, re-running all relevant previous tests. This is known as regression testing.
5. File all documentation.
6
FIT9131
Week 7
What errors can occur in programs?
7
There are three kinds of errors:
• Syntax errors
• Execution (or run-time) errors • Logic errors
• An error that occurs during the execution of the program is also known as a bug.
FIT9131 Week 7
The first computer “bug”
8
Testing of the Mark II Aiken Relay Calculator, September 1947
FIT9131
Week 7
Syntax (compilation) errors
The compiler cannot translate the program, because the code does not follow the rules of the language used.
Some examples:
• Misspelling, eg:
Retrun for return
• Incorrect phrasing of instruction, eg:
if mark < 50
• Structural error, eg:
• no semicolon at end of a statement, method
defined inside another one, etc
• Incorrect number of arguments passed to a method.
9
These errors are generally the easiest to fix.
FIT9131
Week 7
Execution (run
The compiler has successfully translated the program, but the program is run, it fails due to some unexpected errors.
Some examples:
• Divide-by-zero error.
• Attempting to access an array outside its boundaries.
• Request to write to a file when there is insufficient space on the disk.
-
time) errors
These errors can generally be avoided by careful programming .
10
FIT9131
Week 7
Logic errors
The program has been translated and run without any apparent problem, but its output/behaviour is not what was expected, due to some error in its coding logic.
private boolean validMark(int examMark) {
if (examMark < 0 && examMark > 100) return false;
return true;
}
Is there something strange here?
(Hint : try & read the boolean condition in English…)
11
These errors are generally the hardest to fix.
FIT9131 Week 7
Importance of testing
Some logical errors have no immediately obvious manifestation.
Commercial software is rarely (never?) error free, due to the size of the code.
12
We have to deal with errors …
FIT9131 Week 7
Testing Failure Example : Mariner 1 disaster
13
FIT9131
Week 7
Mariner 1
14
July 28, 1962 — Mariner I space probe. A bug in the flight software for the Mariner 1 caused the rocket to divert from its intended path on launch. Mission control destroyed the rocket over the Atlantic Ocean.
The investigation into the accident discovered that a formula written on paper in pencil was improperly transcribed into computer code, causing the computer to miscalculate the rocket’s trajectory.
FIT9131
Week 7
Some other well disasters …
–
known software
•Therac-25 (1985-1987) •AT&T Phone System (1990) •Y2K bug (2000)
Some interesting readings : http://en.wikipedia.org/wiki/List_of_software_bugs 15
FIT9131
Week 7
Prevention vs. Detection (Developer vs. Maintainer)
It is almost impossible to completely eliminate software bugs, especially for large-scale projects. However, we can :
• lessen the likelihood of errors.
• Use software engineering techniques, like encapsulation to reduce the scope of variables – we will discuss this in a later lecture.
• improve the chances of detection.
• Use software engineering practices, like modularisation and documentation.
16
FIT9131
Week 7
Testing and debugging
These are crucial skills in programming.
• Testing searches for the presence of errors.
• this allows programmers to remove/fix the errors.
• Debugging searches for the source of errors.
• Note : the manifestation of an error may well occur some ‘distance’ from its source, making it harder to locate.
17
FIT9131
Week 7
Unit testing
Each unit of an application may be tested. A “unit“ could be a:
• method, class, module (package in Java). Unit testing can (and should) be done during
development. The benefits of early testing are:
• Finding and fixing errors early lowers development costs (e.g. programmer time).
• Enables a test suite to be built up. This can be used to test the unit repeatedly throughout development.
• Eg. How would you test your Player class for the assignment?
18
FIT9131
Week 7
Testing fundamentals
Some points to consider:
• Understand what the unit should do – its contract. • Look for violations.
• Use positive tests and negative tests.
• Consider test boundaries, e.g.
• reading from an empty file.
• adding to a data collection which is full.
There are typically many tests that need to be conducted to thoroughly test a program.
19
FIT9131 Week 7
Testing strategically
We will first consider how to test a simple class. This technique can then be extended to test programs with multiple interacting classes.
When testing a class, every constructor and method in the class must be systematically invoked.
20
FIT9131
Week 7
Testing strategically
BlueJ makes it easy to test a class:
• Objects of individual classes can be created • Individual methods can be invoked
• The state of objects can be inspected
… but you need to document how you do the testing in a more formal manner, eg :
• which values did you use to perform the tests? • what were the expected results?
• what were the actual results?
21
FIT9131 Week 7
Test Strategy
A Test Strategy is a documentation which lists how you tested (and in some cases, how you will test) your class or program. This document should be developed before the program’s code is written, and then updated throughout the coding process.
22
• you will be expected to do this in Assignment 2
FIT9131
Week 7
Test Strategy
A Test Strategy typically contains 2 components :
1) 2)
A Test Plan
• this is a summary of ALL the tests which will be
eventually performed for the program. The Actual Tests
• These are the specifications/details of all the tests listed in the Test Plan.
• Each test will contain the following 3 components :
• TestData
23
• Expected Results
• Actual Results
using actual values (eg. 5), not just
general descriptions (eg. “a number”)
FIT9131
Week 7
Example : Testing an class
Employee
We will use a small class, Employee, to demonstrate how to develop and implement a test strategy.
The Employee class has three attributes: Staff ID, Pay Sccale and Employee Status.
• the Staff ID is a string of 3 numeric characters.
• the Pay Scale is a character : ‘A’, ‘B’, ‘C’ or ‘X’.
• the Employee Status is either true to indicate employed, or false to indicate not currently employed
There are two constructors, plus the usual accessors & mutators.
24
FIT9131
Week 7
Class diagram for class
Employee
Employee
staffId: String
payScale: char
isEmployee: boolean
25
Employee() Employee(String,char,boolean) display()
getStaffId(): String getPayScale(): char getIsEmployee(): boolean setStaffId(string) setPayScale(char) setIsEmployee(boolean)
FIT9131
Week 7
(1) Test Plan Test Plan
1. Create an Employee object with the default constructor.
2. Create Employee objects with all non-default constructors.
for
Employee
class
26
3. Test all the get methods. 4. Test all the set methods. 5. Test the display method.
a summary of what are to be tested
FIT9131
Week 7
(2) Actual Tests Test 1
Create an Employee object with the default constructor. description of the test to be
: Test 1
Test Data:
• No input
Expected Results:
• staffId:
• payScale:
• isEmployee:
carried out
description of the actual results/outputs
NOTE : using actual values, not just general descriptions
“000” ‘X’ false
description of the test data to be used
27
FIT9131 Week 7
Test 1 continued : Actual Results Actual Results:
28
Screen capture of Test 1’s results
FIT9131
Week 7
class default constructor
Example :
Employee
29
public Employee()
{
staffId = “000″;
payScale = ‘X’;
isEmployee = false;
}
if these were wrong, the errors will show up in Test 1 (the Expected Results will not match the Actual Results)
FIT9131 Week 7
Test 2
Test 2
description of the test to be carried out
Create an Employee object with the non-default constructor.
Test Data:
• staffId: “517” • payScale: ‘C’
• isEmployee: true
Expected Results:
• staffId: “517” • payScale: ‘C’
• isEmployee: true
30
NOTE : using actual values, not just general descriptions
NOTE : using actual values, not just general descriptions
FIT9131 Week 7
Test 2 continued : Actual Results Actual Results:
Screen capture of Test 2’s results
31
FIT9131
Week 7
32
Positive and Negative tests Goal of testing is to identify, isolate and correct as
A positive test demonstrates that the program works correctly when the input data is as expected.
(all the tests shown so far have been positive ones. It means that we feed in the kind of data that is supposed to work.)
A negative test demonstrates that the program does not do anything incorrect if the input data is not as expected.
eg. what would you expect to happen if you entered a string of length 2 characters for the staffId?
many errors as possible.
FIT9131
Week 7
An improved Test Plan
Test Plan
1. Create an Employee object with the default constructor
2. Create an Employee object with the non-default constructor
(a) with valid field values
(b) with invalid field values 3. Test all the get methods
4. Test all the set methods
(a) with valid arguments
(b) with invalid arguments 5. Test the display method
33
Positive Tests
Negative Tests
FIT9131
Week 7
Example : Test 2(b)
Test 2(b)
Create an Employee object with invalid attributes (where possible)
Test Data:
• staffId: “99” • payScale: ‘a’
• isEmployee: true
Expected Results:
• staffId: “000” • payScale: ‘X’
• isEmployee: false
34
invalid values are given for the attributes
we are expecting some sensible default values here, since the given data is “wrong”
FIT9131 Week 7
Actual results of Test 2(b) Actual Results
35
Screen capture of Test 2(b).
these results would indicate errors in the code!!!
FIT9131
Week 7
Elaborating on a test plan
Each test listed in the test plan has to be shown in much more detail, separately.
Some tests may need to be broken down more, Eg :
Test 3 (test all the get methods) needs to be broken down into:
Test 3(a) Test the getStaffId method
Test 3(b) Test the getPayScale method
Test 3(c) Test the getEmployeeStatus method
The set methods will be expanded in the same way. 36
FIT9131
Week 7
Useful Testing Heuristic #1 Example:
if each page of a report is to contain no more than 60 lines, then need to set up:
1: Check any boundaries in the specification.
boundaries
• a set of test data that will print exactly 60 lines on the report (or as near as possible on the low side), and
• another set of test data that will print 61 lines on the report (or as near as possible on the high side).
37
FIT9131 Week 7
Useful Testing Heuristic #2
Eg :
public String calculateGrade()
{
if (mark < 0)
return "Invalid";
if (mark < 50)
return "N";
if (mark < 60)
return "P";
if (mark < 70)
return "C";
if (mark < 80)
return "D"; if (mark <= 100)
return "HD";
return "Invalid";
2: Test every path that the flow of control can take through the logic of the program.
}
38
How many paths are there in this code?
FIT9131
Week 7
Precision Issues
39
Sometimes we may need to consider possible precision issues, for instance :
• when double or floating point (real) numbers are involved, the user should be asked to specify the precision of the input data.
• if converting marks to grades, you should find out to what precision a mark can be recorded. Eg. If it is to one decimal place, your testing should include 49.9 and 50.0 If it is to 2 decimal places, the testing should include 49.99 and 50.00
FIT9131
Week 7
Regression testing
Whenever you make a change to a program, you need to make sure that you haven’t introduced any new errors.
• do not assume that errors will only occur in the area of the program where the changes were made.
Regression Testing is the process of re-running all previous tests, after some changes are made to the code
- sometimes new tests may need to be introduced, after errors are fixed, and the code then re-tested
40
FIT9131 Week 7
Summary
Each class should be tested separately. Each method in each class should be tested separately.
In order to test a method, create an instance of the class, then send it a message invoking the method to be tested, giving it various values as actual arguments.
The testing of the class should invoke all the methods in the public interface. If possible, all the private methods should also be invoked, by public methods in the interface.
For initial testing, make all methods public so that you can test them separately. When you are satisfied that everything works, you can make some methods private (if appropriate).
41
-
testing a class
FIT9131
Week 7
Debugging
It is important to develop code-reading skills.
• Debugging are often performed on code written by other people.
It is important to write code that can be easily read by others. Need to consider:
• Comments/documentations in code
• Layout of code – Java coding standards
There are techniques and tools designed to support the debugging process.
42
FIT9131
Week 7
Debugging : Manual Walkthroughs
• This is a low-tech method where a program is “run by hand”. The tester pretends to be the computer, and check through the code manually.
• Conducted away from the computer using printouts of class diagrams or code.
• Involves tracing the flow of control between classes and objects, while observing changes of state and other behaviour.
• May use test data from test strategy
• This is a relatively under-used, but powerful, debugging approach.
43
FIT9131
Week 7
Debugging :Verbal Walkthroughs
• Explain to someone else what the code is doing.
• They might spot the errors that you missed.
• The process of explaining might help you to spot the errors for yourself.
• Group-based processes exist for conducting formal walkthroughs or inspections.
44
FIT9131
Week 7
Debugging : Using Print Statements within the code
• If an error is not obvious, try printing out the values of relevant variables at critical points in the code, to see what is happening.
• This can be done by adding print statements temporarily to code.
• Very common debugging technique.
• No special tools are required.
• Supported by all programming languages.
• Output may be voluminous!
• Turning printing off and on requires forethought.
Note that with BlueJ you may be able to see things more easily using the Debugger and the Inspector.
45
FIT9131
Week 7
Dedicated “Debuggers”
• Debuggers are programs designed to debug code • They are typically both language- and
environment-specific.
• BlueJ has an integrated debugger :
• Support breakpoints.
• Step and Step-into controlled execution. • Call sequence (stack).
• Inspection of object state.
46
FIT9131
Week 7
Review
• Errors are a fact of life in programs – it is human to make mistakes.
• Good software engineering techniques can reduce the occurrence of errors.
• Testing is necessary, and testing and debugging skills are essential.
• Automatetestingwhereverpossible. • Reduces tediousness.
• Reduces human error.
• Makes (re)testing easier.
• Practice a range of debugging skills.
-
Testing
47