Test-Driven Development (TDD)
EECS3311 A: Software Design Fall 2018
CHEN-WEI WANG
DbC: Supplier
DbC is supported natively in Eiffel for supplier:
class ACCOUNT create
make
feature — Attributes owner : STRING
balance : INTEGER feature — Constructors
make(nn: STRING; nb: INTEGER) require — precondition
end feature — Commands
withdraw(amount: INTEGER) require — precondition
non negative amount: amount > 0
affordable amount: amount <= balance -- problematic, why?
2 of 35
do
positive balance: nb > 0 owner := nn
balance := nb
do
ensure — postcondition
balance := balance – amount
balance deducted: balance = old balance – amount
end
invariant — class invariant
positive balance: balance > 0
end
DbC: Contract View of Supplier
Any potential client who is interested in learning about the kind of services provided by a supplier can look through the
(without showing any implementation details):
contract view
class ACCOUNT create
make
feature — Attributes owner : STRING
balance : INTEGER feature — Constructors
make(nn: STRING; nb: INTEGER) require — precondition
positive balance: nb > 0
end feature — Commands
withdraw(amount: INTEGER) require — precondition
non negative amount: amount > 0
affordable amount: amount <= balance -- problematic, why? ensure -- postcondition
balance deducted: balance = old balance - amount
end
invariant -- class invariant
3 of 35
positive balance: balance > 0
end
DbC: Testing Precondition Violation (1.1) The client need not handle all possible contract violations:
class BANK_APP inherit
ARGUMENTS
create
make
feature — Initialization make
— Run application.
local
alan: ACCOUNT do
— A precondition violation with tag “positive_balance”
create {ACCOUNT} alan.make (“Alan”, -10) end
end
By executing the above code, the runtime monitor of Eiffel Studio
will report a contract violation (precondition violation with tag
“positive balance”). 4 of 35
DbC: Testing for Precondition Violation (1.2)
5 of 35
DbC: Testing for Precondition Violation (2.1)
class BANK_APP inherit
ARGUMENTS
create
make
feature — Initialization make
— Run application.
local
mark: ACCOUNT do
create {ACCOUNT} mark.make (“Mark”, 100)
— A precondition violation with tag “non_negative_amount” mark.withdraw(-1000000)
end end
By executing the above code, the runtime monitor of Eiffel Studio
will report a contract violation (precondition violation with tag
“non negative amount”). 6 of 35
DbC: Testing for Precondition Violation (2.2)
7 of 35
DbC: Testing for Precondition Violation (3.1)
class BANK_APP inherit
ARGUMENTS
create
make
feature — Initialization make
— Run application.
local
tom: ACCOUNT do
create {ACCOUNT} tom.make (“Tom”, 100)
— A precondition violation with tag “affordable_amount” tom.withdraw(150)
end end
By executing the above code, the runtime monitor of Eiffel Studio
will report a contract violation (precondition violation with tag
“affordable amount”). 8 of 35
DbC: Testing for Precondition Violation (3.2)
9 of 35
DbC: Testing for Class Invariant Violation (4.1)
class BANK_APP inherit
ARGUMENTS
create
make
feature — Initialization make
— Run application.
local
jim: ACCOUNT do
create {ACCOUNT} tom.make (“Jim”, 100)
jim.withdraw(100)
— A class invariant violation with tag “positive_balance”
end end
By executing the above code, the runtime monitor of Eiffel Studio
will report a contract violation (class invariant violation with tag
“positive balance”). 10 of 35
DbC: Testing for Class Invariant Violation (4.2)
11 of 35
DbC: Testing for Class Invariant Violation (5.1)
class BANK_APP
inherit ARGUMENTS
create make
feature — Initialization
make
— Run application.
local
jeremy: ACCOUNT do
— Faulty implementation of withdraw in ACCOUNT: — balance := balance + amount
create {ACCOUNT} jeremy.make (“Jeremy”, 100) jeremy.withdraw(150)
— A postcondition violation with tag “balance_deducted”
end end
By executing the above code, the runtime monitor of Eiffel Studio
will report a contract violation (postcondition violation with tag
“balance deducted”). 12 of 35
DbC: Testing for Class Invariant Violation (5.2)
13 of 35
TDD: Test-Driven Development (1)
How we have tested the software so far:
X Executed each test case manually (by clicking Run in EStudio).
X Compared with our eyes if actual results (produced by program)
match expected results (according to requirements). Software is subject to numerous revisions before delivery.
Testing manually, repetitively, is tedious and error-prone. We need in order to be cost-effective.
X:
normal scenario (expected outcome)
14 of 35
Test-Driven Development
X
abnormal scenario (expected contract violation).
: Collection of test cases.
A test suite is supposed to measure “correctness” of software. The larger the suite, the more confident you are.
automation
Test Case
Test Suite
TDD: Test-Driven Development (2)
Start writing tests as soon as your code becomes executable : X with a unit of functionality completed
X or even with headers of your features completed
class TEST_STACK …
test_lifo: BOOLEAN
local s: STACK[STRING] do create s.make
s.push (“Alan”) ; s.push (“Mark”) Result := s.top “Mark”
check Result end
s.pop
Result := s.top “Alan” end
end
class STACK[G] create make
— No implementation feature — Queries
top: G do end feature — Commands
make do end
push (v: G) do end pop do end
end
Writing tests should not be an isolated, last-staged activity.
Tests are a precise, executable form of documentation that
can guide your design.
15 of 35
TDD: Test-Driven Development (3)
The ESpec (Eiffel Specification) library is a framework for:
X Writing and accumulating test cases
Each list of relevant test cases is grouped into an ES TEST class, which is just an Eiffel class that you can execute upon.
X Executing the test suite whenever software undergoes a change e.g., a bug fix
e.g., extension of a new functionality
ESpec tests are helpful client of your classes, which may:
X Either attempt to use a feature in a legal way (i.e., satisfying its precondition), and report:
Success if the result is as expected
Failure if the result is not as expected:
e.g., state of object has not been updated properly
e.g., a postcondition violation or class invariant violation occurs
X Or attempt to use a feature in an illegal way (e.g., not satisfying its precondition), and report:
Success if precondition violation occurs. Failure if precondition violation does not occur.
16 of 35
TDD: Test-Driven Development (4)
17 of 35
extend, maintain
Elffel Classes
(e.g., ACCOUNT, BANK)
derive
ESpec Test Suite (e.g., TEST_ACCOUT, TEST_BANK)
fix the Eiffel class under test when some test fails
(re-)run as espec test suite
ESpec Framework
when all tests pass add more tests
Adding the ESpec Library (1) Step 1: Go to Project Settings.
18 of 35
Adding the ESpec Library (2)
Step 2: Right click on Libraries to add a library.
19 of 35
Adding the ESpec Library (3) Step 3: Search for espec and then include it.
This will make two classes available to you:
ES TEST for adding test cases
ES SUITE for adding instances of ES TEST.
X To run, an instance of this class must be set as the root. 20 of 35
ES TEST: Expecting to Succeed (1)
1 2 3 4 5 6 7 8 9
10
11
12
13
14
15
16
17
18
19
20
21
class TEST_ACCOUNT
inherit ES TEST
create make
feature — Add tests in constructor
make
do
add boolean case (agent test_valid_withdraw)
end
feature — Tests
test_valid_withdraw: BOOLEAN local
acc: ACCOUNT do
comment(“test: normal execution of withdraw feature”) create {ACCOUNT} acc.make (“Alan”, 100)
Result := acc.balance = 100
check Result end
acc.withdraw (20)
Result := acc.balance = 80 end
end
21 of 35
ES TEST: Expecting to Succeed (2)
L2: A test class is a subclass of ES TEST.
L10 – 20 define a BOOLEAN test query . At runtime:
X Success: Return value of test valid withdraw (final value of variable Result) evaluates to true upon its termination.
X Failure:
The return value evaluates to false upon termination; or
Some contract violation (which is
L7 calls feature
expects to take as input a
) occurs.
from ES TEST, which that returns a Boolean value.
unexpected
add boolean case
query
X We pass query test valid withdraw as an input. X Think of the keyword agent acts like a function pointer.
alone denotes its return value denotes address of query
L14: Each test feature must call (inherited
from ES TEST) to include the description in test report.
L17: Check that each intermediate value of Result is true. 22 of 35
test invalid withdraw
agent test invalid withdraw
comment(. . . )
ES TEST: Expecting to Succeed (3)
Why is the statement at L7 necessary?
X When there are two or more assertions to make, some of which (except the last one) may temporarily falsify return value Result.
X As long as the last assertion assigns true to Result, then the entire test query is considered as a success.
A false positive is possible!
For the sake of demonstrating a false positive, imagine:
X Constructor make mistakenly deduces 20 from input amount. X Command withdraw mistakenly deducts nothing.
1 2 3 4 5 6 7 8 9
10
23 of 35
Fix? [ insert check Result end ] between L6 and L7.
check Result end
test_query_giving_false_positive: BOOLEAN
local acc: ACCOUNT
do comment(“Result temporarily false, but finally true.”)
create {ACCOUNT} acc.make (“Jim”, 100) — balance set as 80 Result := acc.balance = 100 — Result assigned to false acc.withdraw (20) — balance not deducted
Result := acc.balance = 80 — Result re-assigned to true
— Upon termination, Result being true makes the test query
— considered as a success ==> false positive! end
ES TEST: Expecting to Fail Precondition (1)
1 2 3 4 5 6 7 8 9
10
11
12
13
14
15
16
17
18
19
20
21
class TEST_ACCOUNT
inherit ES TEST
create make
feature — Add tests in constructor
make
do
add violation case with tag (“non_negative_amount”, agent test_withdraw_precondition_violation)
end
feature — Tests
test_withdraw_precondition_violation
local
acc: ACCOUNT do
comment(“test: expected precondition violation of withdraw”) create {ACCOUNT} acc.make (“Mark”, 100)
— Precondition Violation
— with tag “non_negative_amount” is expected.
acc.withdraw (-1000000) end
end
24 of 35
ES TEST: Expecting to Fail Precondition (2)
L2: A test class is a subclass of ES TEST. L11 – 20 define a test command . At runtime:
X Success: A precondition violation (with tag
“non negative amount”) occurs at L19 before its termination.
X Failure:
No contract violation with the expected tag occurs before its termination; or
Some other contract violation (with a different tag) occurs.
L7 calls feature from ES TEST, which expects to take as input a .
X We pass command test invalid withdraw as an input. X Think of the keyword agent acts like a function pointer.
alone denotes a call to it
denotes address of command
L15: Each test feature must call (inherited
from ES TEST) to include the description in test report. 25 of 35
add violation case with tag
command
test invalid withdraw
agent test invalid withdraw
comment(. . . )
ES TEST: Expecting to Fail Postcondition (1) model
tests
feature Test Commands for Contract Violations test_withdraw_postcondition_violation
local
acc: BAD_ACCOUNT_WITHDRAW
do
create acc.make (“Alan”, 100)
Violation of Postcondition
with tag “balance_deduced” expected acc.withdraw (50)
end
ACCOUNT
feature Commands
withdraw (amount: INTEGER)
require
non_negative_amount: amount > 0
affordable_amount: amount ≤ balance do
balance := balance amount
ensure
balance_deduced: balance = old balance amount
end
BAD_ACCOUNT_WITHDRAW
feature Redefined Commands withdraw (amount: INTEGER) ++
do
Precursor (amount)
Wrong Implementation balance := balance + 2 * amount
end
TEST_ACCOUNT
acc
26 of 35
ES TEST: Expecting to Fail Postcondition (2.1)
1 2 3 4 5 6 7 8 9
10
11
12
13
14
15
class
BAD_ACCOUNT_WITHDRAW
inherit
ACCOUNT
redefine withdraw end
create
make
feature — redefined commands withdraw(amount: INTEGER)
do
Precursor(amount)
— Wrong implementation
balance := balance + 2 * amount
end end
X L3–5: BAD ACCOUNT WITHDRAW.withdraw inherits postcondition from ACCOUNT.withdraw: balance = old balance – amount.
X L11 calls correct implementation from parent class ACCOUNT.
X L13 makes overall implementation incorrect. 27 of 35
ES TEST: Expecting to Fail Postcondition (2.2)
1 class TEST_ACCOUNT
2 inherit ES TEST
3 create make
4 feature — Constructor for adding tests 5 make
6 do
7
8
9 end
add violation case with tag (“balance_deducted”, agent test_withdraw_postcondition_violation)
10 feature — Test commands (test to fail) 11 test_withdraw_postcondition_violation 12 local
13
14 do
acc: BAD_ACCOUNT_WITHDRAW
15
16
17
18
19 end 20 end
comment (“test: expected postcondition violation of withdraw”) create acc.make (“Alan”, 100)
— Postcondition Violation with tag “balance_deduced” to occur. acc.withdraw (50)
28 of 35
Exercise
Recall from the “Writing Complete Postconditions” lecture:
class BANK
deposit_on_v5 (n: STRING; a: INTEGER)
do … — Put Correct Implementation Here. ensure
…
others unchanged :
across old accounts.deep twin as cursor all cursor.item.owner / n implies
cursor.item account_of (cursor.item.owner) end
end end
How do you create a “bad” descendant of BANK that violates this postcondition?
class BAD_BANK_DEPOSIT
inherit BANK redefine deposit end feature — redefined feature
deposit_on_v5 (n: STRING; a: INTEGER) do Precursor (n, a)
accounts[accounts.lower].deposit(a)
end end
29 of 35
ES SUITE: Collecting Test Classes
1 2 3 4 5 6 7 8 9
10 11
L2: A test suite is a subclass of ES SUITE.
L7 passes an anonymous object of type TEST ACCOUNT to
add test inherited from ES SUITE).
L8 & L9 have to be entered in this order! 30 of 35
class TEST_SUITE
inherit ES SUITE
create make
feature — Constructor for adding test classes
make
do
add test (create {TEST_ACCOUNT}.make) show_browser
run_espec
end end
Running ES SUITE (1)
Step 1: Change the root class (i.e., entry point of execution) to be
TEST SUITE.
31 of 35
Running ES SUITE (2)
Step 2: Run the Workbench System.
32 of 35
Running ES SUITE (3)
Step 3: See the generated test report.
33 of 35
Beyond this lecture…
Study this tutorial series on DbC and TDD:
https://www.youtube.com/playlist?list=PL5dxAmCmjv_
6r5VfzCQ5bTznoDDgh__KS
34 of 35
Index (1)
DbC: Supplier
DbC: Contract View of Supplier
DbC: Testing for Precondition Violation (1.1) DbC: Testing for Precondition Violation (1.2) DbC: Testing for Precondition Violation (2.1) DbC: Testing for Precondition Violation (2.2) DbC: Testing for Precondition Violation (3.1) DbC: Testing for Precondition Violation (3.2) DbC: Testing for Class Invariant Violation (4.1) DbC: Testing for Class Invariant Violation (4.2) DbC: Testing for Class Invariant Violation (5.1) DbC: Testing for Class Invariant Violation (5.2) TDD: Test-Driven Development (1)
TDD: Test-Driven Development (2)
35 of 35
Index (2)
TDD: Test-Driven Development (3) TDD: Test-Driven Development (4) Adding the ESpec Library (1) Adding the ESpec Library (2) Adding the ESpec Library (3)
ES TEST: Expecting to Succeed (1)
ES TEST: Expecting to Succeed (2)
ES TEST: Expecting to Succeed (3)
ES TEST: Expecting to Fail Precondition (1) ES TEST: Expecting to Fail Precondition (2) ES TEST: Expecting to Fail Postcondition (1) ES TEST: Expecting to Fail Postcondition (2.1) ES TEST: Expecting to Fail Postcondition (2.2)
Exercise
36 of 35
Index (3)
ES SUITE: Collecting Test Classes
Running ES SUITE (1)
Running ES SUITE (2)
Running ES SUITE (3)
Beyond this lecture…
37 of 35