Eiffel Testing Framework (ETF): Automated Regression & Acceptance Testing
EECS3311 A & E: Software Design Fall 2020
CHEN-WEI WANG
Required Tutorial
All technical details of ETF are discussed in this tutorial series:
https://www.youtube.com/playlist?list=PL5dxAmCmjv_
5unIgLB9XiLwBey105y3kI
3 of 21
Learning Objectives
Upon completing this lecture, you are expected to understand: 1. User Interface: Concrete vs. Abstract
2. Use Case: Interleaving Model, Events & (Abstract) States 3. Acceptance Tests vs. Unit Tests
4. Regression Tests
2 of 21
Take-Home Message
● Your remaining assignments are related to ETF: Lab3 & Project. ● You are no longer just given partially implemented classes:
○ Design decisions have already been made for you. ○ You are just to fill in the blanks (to-do’s).
● ETF is in Eiffel, but try to see beyond what it allows you do: 1. Design your own classes and routines.
2. Practice design principles:
e.g., DbC, modularity, information hiding, single-choice, cohesion.
3. Practice design patterns:
e.g., iterator, singleton.
4. Practice acceptance testing and regression testing.
4 of 21
Bank ATM: Concrete User Interfaces
An ATM app has many concrete (implemented, functioning) UIs. PHYSICAL INTERFACE MOBILE INTERFACE
5 of 21
Prototyping System with Abstract UI
● For you to quickly a working system, you do not need to spend time on developing a elaborate, full-fledged GUI.
● The Eiffel Testing Framework (ETF ) allows you to:
○ Generate a starter project from the specification of an abstract UI. ○ Focus on developing the business model .
○ Test your business model as if it were a real app.
● Q. What is an abstract UI?
Events abstracting observable interactions with the concrete GUI (e.g., button clicks, text entering).
● Q. Events vs. Features (attributes & routines)? 7 of 21
Events
Features
interactions
computations
external
internal
observable
hidden
acceptance tests
unit tests
users, customers
programmers, developers
prototype
UI, Model, TDD
● Separation of Concerns
○ The (Concrete) User Interface
Users typically interact with your application via some GUI.
e.g., web app, mobile app, or desktop app ○ The Model (Business Logic)
Develop an application via classes and features.
e.g., a bank storing, processing, retrieving accounts & transactions
● Test Driven Development (TDD) In practice:
○ The model should be independent of the UI or View.
○ Do not wait to test the model when the concrete UI is built.
⇒ Test your software as if it was a real app
way before dedicating to the design of an actual GUI.
⇒ Use an abstract UI (e.g., a cmd-line UI) for this purpose. 6 of 21
Bank ATM: Abstract UI
AbstractUI isthelistofeventsabstractingobservableinteractions with the concrete GUI (e.g., button clicks, text entering).
system bank
new(id: STRING)
— create a new bank account for “id”
deposit(id: STRING; amount: INTEGER)
— deposit “amount” into the account of “id”
withdraw(id: STRING; amount: INTEGER)
— withdraw “amount” from the account of “id”
transfer(id1: STRING; id2: STRING; amount: INTEGER) — transfer “amount” from “id1” to “id2”
8 of 21
Bank ATM: Abstract States
is a representation of the system:
○ Including relevant details of functionalities under testing
9 of 21
○ Excluding other irrelevant details
e.g., An abstract state may show each account’s owner:
{alan, mark, tom}
e.g., An abstract state may also show each account’s balance: e.g., An abstract state may show account’s transactions:
{alan: 200, mark: 300, tom: 700}
Account Owner: alan
List of transactions:
+ deposit (Oct 15): $100
– withdraw (Oct 18): $50
Account Owner: mark
List of transactions:
Abstract State
Bank ATM: Outputs of Acceptance Tests (1)
Output from running an acceptance test is a sequence interleaving abstract states and abstract events:
S0->e1->S1->e2->S2-> . . .
where:
○ S0 is the initial state.
○ Si isthepre-stateofeventei+1
e.g., S0 is the pre-state of e1, S1 is the pre-state of e2 ○ Si isthepost-stateofeventei
e.g., S1 is the post-state of e1, S2 is the post-state of e2 11 of 21
[i≥0] [i≥1]
Bank ATM: Inputs of Acceptance Tests
An acceptance test is a use case of the system under test, characterized by sequential occurrences of abstract events.
For example:
new(“alan”)
new(“mark”)
deposit(“alan”, 200)
deposit(“mark”, 100)
transfer(“alan”, “mark”, 50)
10 of 21
Bank ATM: Outputs of Acceptance Tests (2)
Consider an example acceptance test output:
{}
->new(“alan”) {alan: 0} ->new(“mark”)
{alan: 0, mark: 0} ->deposit(“alan”, 200) {alan: 200, mark: 0} ->deposit(“mark”, 100)
{alan: 200, mark: 100} ->transfer(“alan”, “mark”, 50)
{alan: 150, mark: 150}
● Initial State? {}
● What role does the state {alan: 200, mark: 0} play?
○ Post-State of deposit(“alan”, 200)
○ Pre-State of deposit(“mark”, 100) 12 of 21
Bank ATM: Acceptance Tests vs. Unit Tests
Q. Difference between an acceptance test and a unit test?
A.
{}
->new(“alan”) {alan: 0}
->deposit(“alan”, 200) {alan: 200}
test: BOOLEAN
local acc: ACCOUNT
do create acc.make(“alan”)
acc.add(200)
Result := acc.balance = 200 end
○ Writing a unit test requires knowledge about the programming language and details of implementation.
⇒ Written and run by developers
○ Writing an acceptance test only requires familiarity with the abstract UI and abstract state.
⇒ Written and run by customers [ for communication ]
13 of 21⇒ Written and run by developers [ for testing ]
Workflow: Develop-Connect-Test
define
monitored
events
(re)new
implement
derive
use cases
business
model
connect to
generate Code Skeleton
redefine
debug
test run
fix or add
Abstract
State
ETF
15 of 21
ETF in a Nutshell
● Eiffel Testing Framework (ETF) facilitates engineers to write and execute input-output-based acceptance tests.
○ Inputs are specified as traces of events (or sequences).
○ The abstract UI of the system under development (SUD) is defined by declaring the list of input events that might occur.
○ Outputs are interleaved states and events logged to the terminal, and their formats may be customized.
● An executable ETF project tailored for the SUD can already be generated, using these event declarations (specified in a plain
text file), with a default .
business model
○ Once the is implemented, there is a small number of steps to follow for developers to connect it to the generated ETF.
business model
14 of 21
○ Once connected, developers may re-run all acceptance tests and observe if the expected state effects occur.
ETF: Abstract UI and Acceptance Test
16 of 21
ETF: Generating a New Project
17 of 21
ETF: Implementing an Abstract Command
19 of 21
ETF: Architecture
*+
EFCOMMAND
EFMODEL
+ EFMODELACCESS
● Classes in the model cluster are hidden from the users.
● All commands reference to the same model (bank) instance. ● When a user’s request is made:
○ A command object of the corresponding type is created, which invokes relevant feature(s) in the model cluster.
○ Updates to the model are published to the output handler. 18 of 21
++++ EFNE EFDEPOSI EFIHDRA EFRANSFER
Beyond this lecture
The singleton pattern is instantiated in the ETF framework:
● ETF MODEL (shared data) ● ETF MODEL ACCESS (exclusive once access) ● ETF COMMAND and its effective descendants:
deferred class
ETF_COMMAND
feature — Attributes model: ETF_MODEL
feature {NONE} make(. . .)
local
ma: ETF_MODEL_ACCESS do
…
model := ma.m end
end
class
ETF_DEPOSIT
inherit
ETF_DEPOSIT_INTERFACE
— which inherits ETF_COMMAND feature — command
deposit(. . .) do
…
model.some_routine (…) …
end end
20 of 21
Index (1)
Learning Objectives
Required Tutorial
Take-Home Message
Bank ATM: Concrete User Interfaces
UI, Model, TDD
Prototyping System with Abstract UI
Bank ATM: Abstract UI
Bank ATM: Abstract States
Bank ATM: Inputs of Acceptance Tests
Bank ATM: Outputs of Acceptance Tests (1)
Bank ATM: Outputs of Acceptance Tests (2)
21 of 21
Index (2)
ETF in a Nutshell
Workflow: Develop-Connect-Test
ETF: Abstract UI and Acceptance Test
ETF: Generating a New Project
ETF: Architecture
ETF: Implementing an Abstract Command
Beyond this lecture
22 of 21
Bank ATM: Acceptance Tests vs. Unit Tests