Test-Driven Development (TDD) with JUnit
EECS2030 B: Advanced Object Oriented Programming Fall 2018
CHEN-WEI WANG
Motivating Example: Two Types of Errors (1)
Consider two kinds of exceptions for a counter:
public class ValueTooLargeException extends Exception { ValueTooLargeException(String s) { super(s); }
}
public class ValueTooSmallException extends Exception {
ValueTooSmallException(String s) { super(s); } }
Any thrown object instantiated from these two classes must be handled ):
X Either throws . . . in the method signature (i.e., propagating it to other caller)
X Or handle it in a try-catch block 2 of 41
( catch-specify requirement
specify
Motivating Example: Two Types of Errors (2)
Approach 1 – Specify: Indicate in the method signature that a specific exception might be thrown.
Example 1: Method that throws the exception
3 of 41
class C1 {
void m1(int x) throws ValueTooSmallException {
if(x < 0) {
throw new ValueTooSmallException("val " + x);
} }
}
Example 2: Method that calls another which throws the exception
class C2 { C1 c1;
void m2(int x) throws ValueTooSmallException { c1.m1(x);
} }
Motivating Example: Two Types of Errors (3) Approach 2 – Catch: Handle the thrown exception(s) in a
try-catch block.
4 of 41
class C3 {
public static void main(String[] args) {
Scanner input = new Scanner(System.in); int x = input.nextInt();
C2 c2 = new c2();
try {
c2.m2(x); }
catch(ValueTooSmallException e) { ... } }
}
A Simple Counter (1)
Consider a class for keeping track of an integer counter value:
public class Counter {
public final static int MAX_VALUE = 3; public final static int MIN_VALUE = 0; private int value;
public Counter() {
this.value = Counter.MIN_VALUE; }
public int getValue() { return value;
}
... /* more later! */
X Access private attribute value using public accessor getValue. X Two class-wide (i.e., static) constants (i.e., final) for lower and
upper bounds of the counter value.
X Initialize the counter value to its lower bound.
X Requirement :
The counter value must be between its lower and upper bounds.
5 of 41
Exceptional Scenarios
Consider the two possible exceptional scenarios:
An attempt to increment above the counter’s upper bound.
An attempt to decrement below the counter’s lower bound.
6 of 41
A Simple Counter (2)
/* class Counter */
public void increment() throws ValueTooLargeException { if(value == Counter.MAX_VALUE) {
throw new ValueTooLargeException("counter value is " + value); }
else { value ++; } }
public void decrement() throws ValueTooSmallException { if(value == Counter.MIN_VALUE) {
throw new ValueTooSmallException("counter value is " + value); }
else { value --; } }
}
7 of 41
X Change the counter value via two mutator methods.
X Changes on the counter value may trigger an exception:
Attempt to increment when counter already reaches its maximum. Attempt to decrement when counter already reaches its minimum.
Components of a Test
Manipulate the relevant object(s).
e.g., Initialize a counter object c, then call c.increment(). What do you expect to happen ?
e.g., value of counter is such that Counter.MIN VALUE + 1 What does your program actually produce ?
e.g., call c.getValue to find out.
A test:
X Passes if expected value matches actual value
X Fails if expected value does not match actual value
So far, you ran tests via a tester class with the main method. 8 of 41
Testing Counter from Console (V1): Case 1
Consider a class for testing the Counter class:
public class CounterTester1 {
public static void main(String[] args) {
Counter c = new Counter(); println("Init val: " + c.getValue()); try {
c.decrement();
println("ValueTooSmallException NOT thrown as expected."); }
catch (ValueTooSmallException e) { println("ValueTooSmallException thrown as expected.");
}}}
Executing it as Java Application gives this Console Output:
9 of 41
Init val: 0
ValueTooSmallException thrown as expected.
Testing Counter from Console (V1): Case 2 Consider another class for testing the Counter class:
public class CounterTester2 {
public static void main(String[] args) {
Counter c = new Counter();
println("Current val: " + c.getValue());
try { c.increment(); c.increment(); c.increment(); } catch (ValueTooLargeException e) {
println("ValueTooLargeException thrown unexpectedly."); } println("Current val: " + c.getValue());
try {
c.increment();
println("ValueTooLargeException NOT thrown as expected."); } catch (ValueTooLargeException e) {
println("ValueTooLargeException thrown as expected."); } } }
Executing it as Java Application gives this Console Output:
10 of 41
Current val: 0
Current val: 3
ValueTooLargeException thrown as expected.
Testing Counter from Console (V2)
Consider a different class for testing the Counter class:
import java.util.Scanner; public class CounterTester3 {
public static void main(String[] args) {
Scanner input = new Scanner(System.in);
String cmd = null; Counter c = new Counter(); boolean userWantsToContinue = true; while(userWantsToContinue) {
println("Enter \"inc\", \"dec\", or \"val\":"); cmd = input.nextLine();
try {
if(cmd.equals("inc")) {
else if(cmd.equals("dec")) {
}
catch(ValueTooLargeException e){ println("Value too big!"); } catch(ValueTooSmallException e){ println("Value too small!"); }
}}}
; }
; }
); } else { userWantsToContinue = false; println("Bye!"); }
c.increment()
c.decrement()
else if(cmd.equals("val")) { println(
c.getValue()
11 of 41
Testing Counter from Console (V2): Test 1 Test Case 1: Decrement when the counter value is too small.
12 of 41
Enter "inc", "dec", or "val":
val
0
Enter "inc", "dec", or "val": dec
Value too small!
Enter "inc", "dec", or "val": exit
Bye!
Testing Counter from Console (V2): Test 2 Test Case 2: Increment when the counter value is too big.
13 of 41
Enter "inc", "dec", or "val":
inc
Enter "inc", "dec", or "val":
inc
Enter "inc", "dec", or "val":
inc
Enter "inc", "dec", or "val":
val
3
Enter "inc", "dec", or "val": inc
Value too big!
Enter "inc", "dec", or "val": exit
Bye!
Limitations of Testing from the Console
Do Test Cases 1 & 2 suffice to test Counter’s correctness? X Is it plausible to claim that the implementation of Counter is
correct because it passes the two test cases? What other test cases can you think of?
c.getValue() c.increment() c.decrement()
0 1 ValueTooSmall 120 231
3 ValueTooLarge 2
So in total we need 8 test cases. 6 more separate
X CounterTester classes to create (like CounterTester1)! X Console interactions with CounterTester3!
Problems? It is inconvenient to:
X Run each TC by executing main of a CounterTester and
comparing console outputs with your eyes.
X manually all TCs whenever Counter is changed.
14 of 41
: Any change introduced to your software must not compromise its established correctness.
Re-run
Regression Testing
Why JUnit?
Automate the testing of correctness of your Java classes. Once you derive the list of tests, translate it into a JUnit test
case, which is just a Java class that you can execute upon.
JUnit tests are helpful callers/clients of your classes, where each test may:
X Either attempt to use a method in a legal way (i.e., satisfying its precondition), and report:
Success if the result is as expected
Failure if the result is not as expected
X Or attempt to use a method in an illegal way (i.e., not satisfying
its precondition), and report: Success if the expected exception
(e.g., ValueTooSmallException) occurs. Failure if the expected exception does not occur.
15 of 41
How to Use JUnit: Packages
Step 1:
X In Eclipse, create a Java project ExampleTestingCounter X Separation of concerns :
Group classes for implementation (i.e., Counter) into package implementation.
Group classes classes for testing (to be created) into package tests.
16 of 41
How to Use JUnit: New JUnit Test Case (1) Step 2: Create a new JUnit Test Case in tests package.
Create one JUnit Test Case to test one Java class only.
If you have n Java classes to test, create n JUnit test cases. 17 of 41
How to Use JUnit: New JUnit Test Case (2) Step 3: Select the version of JUnit (JUnit 4); Enter the name of
test case (TestCounter); Finish creating the new test case.
18 of 41
How to Use JUnit: Adding JUnit Library
Upon creating the very first test case, you will be prompted to add the JUnit library to your project’s build path.
19 of 41
How to Use JUnit: Generated Test Case
X Lines 6 – 8: test is just an ordinary mutator method that has a one-line implementation body.
X Line 5 is critical: Prepend the tag @Test verbatim, requiring that the method is to be treated as a JUnit test.
When TestCounter is run as a JUnit Test Case, only those methods prepended by the @Test tags will be run and reported.
X Line 7: By default, we deliberately fail the test with a message “Not yet implemented”.
20 of 41
How to Use JUnit: Running Test Case Step 4: Run the TestCounter class as a JUnit Test.
21 of 41
X
How to Use JUnit: Generating Test Report
A report is generated after running all tests (i.e., methods prepended with @Test) in TestCounter.
22 of 41
X
How to Use JUnit: Interpreting Test Report
A test is a method prepended with the @Test tag. The result of running a test is considered:
X Failure if either
an assertion failure (e.g., caused by fail, assertTrue, assertEquals) occurs; or
an unexpected exception (e.g., NullPointerException, ArrayIndexOutOfBoundException) is thrown.
X Success if neither assertion failures nor unexpected exceptions occur.
After running all tests:
X A green bar means that all tests succeed.
Keep challenging yourself if more tests may be added. X A red bar means that at least one test fails.
Keep fixing the class under test and re-runing all tests, until you
receive a green bar.
Question: What is the easiest way to making test a success?
Answer: Delete the call fail("Not yet implemented"). 23 of 41
How to Use JUnit: Revising Test Case
Now, the body of test simply does nothing.
Neither assertion failures nor exceptions will occur.
The execution of test will be considered as a success.
ì There is currently only one test in TestCounter.
We will receive a green bar!
Caution: test which passes at the moment is not useful at all!
24 of 41
How to Use JUnit: Re-Running Test Case
A new report is generated after re-running all tests (i.e., methods prepended with @Test) in TestCounter.
25 of 41
X
How to Use JUnit: Adding More Tests (1)
Recall the complete list of cases for testing Counter: c.getValue() c.increment() c.decrement()
0 10 21 32
Let’s turn the two cases in the 1st row into two JUnit tests: X Test for the green cell succeeds if:
No failures and exceptions occur; and
The new counter value is 1.
X Tests for red cells succeed if the expected exceptions occur
(ValueTooSmallException & ValueTooLargeException). Common JUnit assertion methods:
X void assertNull(Object o)
X void assertEquals(expected, actual)
X void assertArrayEquals(expecteds, actuals) X void assertTrue(boolean condition)
X void fail(String message) 26 of 41
1
ValueTooSmall
2 3
ValueTooLarge
How to Use JUnit: Assertion Methods
27 of 41
How to Use JUnit: Adding More Tests (2.1)
1 2 3 4 5 6 7 8 9
10
@Test
public void testIncAfterCreation() {
Counter c = new Counter(); assertEquals(Counter.MIN_VALUE, c.getValue()); try {
c.increment();
assertEquals(1, c.getValue());
} catch(ValueTooBigException e) {
/* Exception is not expected to be thrown. */
("ValueTooBigException is not expected."); } }
fail
X Lines 5 & 8: We need a try-catch block because of Line 6. Method increment from class Counter may throw the ValueTooBigException.
X Lines 4, 7 & 10 are all assertions:
Lines 4 & 7 assert that c.getValue() returns the expected values. Line 10: an assertion failure ì unexpected ValueTooBigException
X Line 7 can be rewritten as assertTrue(1 == c.getValue()). 28 of 41
How to Use JUnit: Adding More Tests (2.2)
Don’t lose the big picture!
JUnit test in previous slide automates this console interaction:
Enter "inc", "dec", or "val":
val
0
Enter "inc", "dec", or "val": inc
Enter "inc", "dec", or "val": val
1
Enter "inc", "dec", or "val": exit
Bye!
Automation is exactly rationale behind using JUnit! 29 of 41
How to Use JUnit: Adding More Tests (3.1)
1 2 3 4 5 6 7 8 9
@Test
public void testDecFromMinValue() {
Counter c = new Counter(); assertEquals(Counter.MIN_VALUE, c.getValue()); try {
c.decrement();
fail ("ValueTooSmallException is expected."); }
catch(ValueTooSmallException e) {
/* Exception is expected to be thrown. */ } }
X Lines 5 & 8: We need a try-catch block because of Line 6. Method decrement from class Counter may throw the ValueTooSmallException.
X Lines 4 & 7 are both assertions:
Lines 4 asserts that c.getValue() returns the expected value (i.e.,
Counter.MIN VALUE).
Line 7: an assertion failure ì expected ValueTooSmallException not thrown
30 of 41
How to Use JUnit: Adding More Tests (3.2)
Again, don’t lose the big picture!
JUnit test in previous slide automates CounterTester1 and
the following console interaction for CounterTester3:
Enter "inc", "dec", or "val":
val
0
Enter "inc", "dec", or "val": dec
Value too small!
Enter "inc", "dec", or "val": exit
Bye!
Again, automation is exactly rationale behind using JUnit! 31 of 41
How to Use JUnit: Adding More Tests (4.1)
1 2 3 4 5 6 7 8 9
10
11
12
13
14
15
@Test
public void testIncFromMaxValue() { Counter c = new Counter();
try {
c.increment(); c.increment(); c.increment(); } catch (ValueTooLargeException e) {
fail("ValueTooLargeException was thrown unexpectedly."); }
assertEquals(Counter.MAX_VALUE, c.getValue()); try {
c.increment();
fail("ValueTooLargeException was NOT thrown as expected."); } catch (ValueTooLargeException e) {
/* Do nothing: ValueTooLargeException thrown as expected. */
}}
X Lines4–8:
We use a try-catch block to express that a VTLE is not expected.
X Lines 9 – 15:
We use a try-catch block to express that a VTLE is expected. 32 of 41
How to Use JUnit: Adding More Tests (4.2)
JUnit test in previous slide automates CounterTester2 and the following console interaction for CounterTester3:
33 of 41
Enter "inc", "dec", or "val":
inc
Enter "inc", "dec", or "val":
inc
Enter "inc", "dec", or "val":
inc
Enter "inc", "dec", or "val":
val
3
Enter "inc", "dec", or "val": inc
Value too big!
Enter "inc", "dec", or "val": exit
Bye!
How to Use JUnit: Adding More Tests (4.3) Q: Can we rewrite testIncFromMaxValue to:
1 2 3 4 5 6 7 8 9
10 11 12
@Test
public void testIncFromMaxValue() { Counter c = new Counter();
try {
c.increment();
c.increment();
c.increment();
assertEquals(Counter.MAX_VALUE, c.getValue()); c.increment();
fail("ValueTooLargeException was NOT thrown as expected."); } catch (ValueTooLargeException e) { }
}
No!
At Line 9, we would not know which line throws the VTLE:
X If it was any of the calls in L5 – L7, then it’s not right.
X If it was L9, then it’s right. 34 of 41
How to Use JUnit: Adding More Tests (5)
Loops can make it effective on generating test cases:
1 2 3 4 5 6 7 8 9
10
11
12
13
14
15
16
17
18
19
@Test
public void testIncDecFromMiddleValues() { Counter c = new Counter();
try {
for(int i = Counter.MIN_VALUE; i < Counter.MAX_VALUE; i ++) { int currentValue = c.getValue();
c.increment();
assertEquals(currentValue + 1, c.getValue());
}
for(int i = Counter.MAX_VALUE; i > Counter.MIN_VALUE; i –) {
int currentValue = c.getValue(); c.decrement();
assertEquals(currentValue – 1, c.getValue());
}
} catch(ValueTooLargeException e) {
fail(“ValueTooLargeException is thrown unexpectedly”); } catch(ValueTooSmallException e) {
fail(“ValueTooSmallException is thrown unexpectedly”); }}
35 of 41
Exercises
1. Run all 8 tests and make sure you receive a green bar.
2. Now, introduction an error to the implementation: Change the line value ++ in Counter.increment to –.
X Re-run all 8 tests and you should receive a red bar. [ Why? ] X Undo the error injection, and re-run all 8 tests. [ What happens? ]
36 of 41
Test-Driven Development (TDD)
37 of 41
fix the Java class under test when some test fails
Java Classes (e.g., Counter)
derive
JUnit Test Case (e.g., TestCounter)
extend, maintain
(re-)run as junit test case
JUnit Framework
when all tests pass add more tests
Maintain a collection of tests which define the correctness of your Java class under development (CUD):
Derive and run tests as soon as your CUD is testable .
i.e., A Java class is testable when defined with method signatures. Red bar reported: Fix the class under test (CUT) until green bar. Green bar reported: Add more tests and Fix CUT when necessary.
Resources
Official Site of JUnit 4:
http://junit.org/junit4/
API of JUnit assertions:
http://junit.sourceforge.net/javadoc/org/junit/Assert.html
Another JUnit Tutorial example:
https://courses.cs.washington.edu/courses/cse143/11wi/ eclipse- tutorial/junit.shtml
38 of 41
Index (1)
Motivating Example: Two Types of Errors (1) Motivating Example: Two Types of Errors (2) Motivating Example: Two Types of Errors (3) A Simple Counter (1)
Exceptional Scenarios
A Simple Counter (2)
Components of a Test
Testing Counter from Console (V1): Case 1 Testing Counter from Console (V1): Case 2 Testing Counter from Console (V2)
Testing Counter from Console (V2): Test 1 Testing Counter from Console (V2): Test 2 Limitations of Testing from the Console
Why JUnit?
39 of 41
Index (2)
How to Use How to Use How to Use How to Use How to Use How to Use How to Use How to Use How to Use How to Use How to Use How to Use How to Use
How to Use
40 of 41
JUnit: Packages
JUnit: New JUnit Test Case (1) JUnit: New JUnit Test Case (2) JUnit: Adding JUnit Library JUnit: Generated Test Case JUnit: Running Test Case JUnit: Generating Test Report JUnit: Interpreting Test Report JUnit: Revising Test Case JUnit: Re-Running Test Case JUnit: Adding More Tests (1) JUnit: Assertion Methods JUnit: Adding More Tests (2.1) JUnit: Adding More Tests (2.2)
Index (3) How to Use
How to Use How to Use How to Use How to Use How to Use Exercises Test-Driven
Resources
JUnit: Adding More JUnit: Adding More JUnit: Adding More JUnit: Adding More JUnit: Adding More JUnit: Adding More
Development (TDD)
Tests (3.1) Tests (3.2) Tests (4.1) Tests (4.2) Tests (4.3) Tests (5)
41 of 41