Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Usability and Quality Assurance
Session 1 2021
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Outline
• Software Quality Assurance and Software Testing
• Relating the popular Agile methods with: • Software Quality
• ISO9000 Quality Standards
• CMM-I Process Maturity Levels
2
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Software Development Process – Getting from Problems to Solutions
• Generalised activities in all software processes include:
• Specification – what the system should do and any development constraints • Development – production of the software system
• Validation – checking that the software meets the customer’s needs
• Evolution – changing the software in response to changing demands
New or changed requirements (Problem)
New or changed software product
(Solution)
Software Development Process
3
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Importance of Software/IT
• More and more systems are software controlled
• Including an increasing number of safety-critical and mission-critical systems, with high demands on dependability
• More and more businesses depend on software for their success
• Software and Information Systems are critical success factors in an increasing
number of businesses and organizations
• The economies of nations are dependent on software
• Software expenditure (in the development and maintenance of software products) is significant
4
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
What is Quality?
• Conformance to a stated and documented requirement (quantifiable)
• Fitness for Purpose (open to misunderstanding and argument – expectation)
• Degree of Excellence (different level definition of excellence – expectation)
5
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Time
(Management, User)
Budget (Management)
Financial Strength
Functionality (User)
Fudged Process
Insufficient Modeling
Software
Quality Priority!!
(Process QA for UML-based Projects, by B. Unhelkar, Addison-Wesley, USA)
AdHoc Testing
Quality
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
The Quality Context Triangle (Process QA for UML-based Projects, by B. Unhelkar, Addison-Wesley, USA)
Quality Environment
Management Quality
Process Quality
Model Quality
Code Quality
Primary Roles operating in a slice
Of the triangle. Relevant Quality Roles Such as Tester, Auditor etc. will Accompany the primary roles
Data Quality
Quality Manager
Project Manager Process Engineer
Modeler, Architect Programmer
Data Manager, User
Technology
MethodologySociology
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Quality Processes Reduce Costs of error correction, thereby Increasing Productivity (Process QA for UML-based Projects, by B. Unhelkar, 2003, Addison-Wesley, USA)
Quality Process
20$ – 200$
1$
Time
AGILE software processes aim to enhance QUALITY
Implementation
Cost of Error Correction $
Business Analysis
Requirements Modeling
System Design
Running
Deployment Testing
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Quality – Strategic versus Tactical aspects of quality
Quality Management
Quality Assurance
Quality Control
Strategic (project size and type issues)
Process issues
Tactical (unit tests, acceptance tests, test beds, results)
(Practical OO Analysis, B. Unhelkar, 2005, Cengage, Melbourne)
9
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Organization of testing (based on Unhelkar, 1999)
Strategic: Created for the entire Project.
Test Plan
Test Design 1
Test Design 2
Created per Package;
Tactical: Created per Element (like a Use case or a Class)
Test Case 2
Test Case 1
Test Case 1
Test Case 2
Test Case 3
Test Case 3
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Some Valid and Invalid Inputs in Test Cases
INPUT for “Patient Registration DATE” field
Comment
30 Sep, 2004
Typical valid standard input. Accept; Pass.
31 Dec, 2004.
Another valid input on the boundary. Accept; Pass.
29 Feb, 2003.
An Invalid input. System should reject for Pass.
00 00 0000.
Another Invalid Input. System should reject for test to Pass.
11
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Product Quality Attributes
• Attributes of good software (beyond delivering the required functionality) include:
• Efficiency – Software should not make wasteful use of system resources (disk and memory space, CPU time, etc.) and should present appropriate response times
• Usability (ease of use) – Software must be usable by the users for which it was designed
• Dependability (reliability, availability, security, safety etc.) – Software must be trustworthy
• Maintainability (ease of maintenance) – Software must evolve to meet changing needs. Software costs more to maintain than it does to develop. For systems with a long life, maintenance costs may be several times development costs
12
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Main Dimensions of Dependability
• Reliability – The probability of failure-free system operation over a specified time in a given environment for a given purpose
• Availability – The probability that a system, at a point in time, will be operational and able to deliver the requested services.
• It’s possibly to have high availability with low reliability if failures are repaired quickly
• Safety – The system’s ability to operate, normally or abnormally, without danger of causing human injury or death and without damage to the system’s environment
• Security – The system’s ability to protect itself from accidental or deliberate external attack
13
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Dependability of Critical Systems
• For critical systems, it is usually the case that the most important system property is the dependability of the system
• Types of critical systems:
• Safety-critical system – a system whose failure may result in injury, loss of life or major environment damage
•
• Mission-critical system – a system whose failure may result in the failure of some goal-directed activity
•
• Business-critical system – a system whose failure may result in the failure of the business using the system
•
14
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Dependability of Critical Systems
• For critical systems, it is usually the case that the most important system property is the dependability of the system
• Types of critical systems:
• Safety-critical system – a system whose failure may result in injury, loss of life or major environment damage
• e.g. an insulin delivery system
• Mission-critical system – a system whose failure may result in
the failure of some goal-directed activity
• e.g. a navigational system for a space aircraft
• Business-critical system – a system whose failure may result in the failure of the business using the system
• e.g. a customer account system in a bank
15
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Quality Assurance and Testing
• How to think about quality
• https://www.youtube.com/watch?v=hMfPCdF07hA
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Quality Degrades with Time
• Software becomes obsolete due to:
• Fast evolution of technology, changing requirements or increased knowledge
• It is used for longer than originally anticipated (remember the Y2K bug?)
• Continuous Innovation and Evolution
• Withdrawal of Manufacturer’s Support
17
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Process and Product Quality
• The quality of a developed product is influenced by the quality of the production process
• This is important in software development as some product quality attributes are hard to assess
• However, there is a very complex and poorly understood relationship between software processes and product quality
18
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Process-based quality
• There is a straightforward link between process and product in manufactured goods.
• More complex for software because:
• The application of individual skills and experience is particularly important in
software development;
• External factors such as the novelty of an application or the need for an accelerated development schedule may impair product quality.
• Care must be taken not to impose inappropriate process standards – these could reduce rather than improve the product quality.
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Process-based Quality
Define Process
Develop Product
Improve Process
Assess Product Quality
Is Quality OK?
Standardise Process
21
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Practical Process Quality
• Define process standards such as how reviews should be conducted, configuration management, etc.
• Monitor the development process to ensure that standards are being followed.
• Report on the process to project management and software procurer.
• Don’t use inappropriate practices simply because standards have been established.
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Process Quality
• A good process is usually required to produce a good product
• For manufactured goods, process is the principal quality determinant
• For design-based activity (like software development), other factors are also involved especially the capabilities of the designers
• For large projects with ‘average’ capabilities, the development process determines product quality
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
POPULAR AGILE METHODS
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Extreme Programming (XP) (Beck 2001; esic-solutions.com)
MethodScience.com, 1998- 2010;
Composite Agile Method & Strategy
User Stories (Cards) – Code – Unit Tests Stresses Customer Satisfaction
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Lean Development
Greater Focus on project management rather than technical; Evolved from Deming’s TQM approach
MethodScience.com, 1998-2010; Composite Agile Method & Strategy
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Feature-Driven Development (FDD)
Domain Model based on SMEs; other Models. Translated into Features
SMEs = subject matter experts
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Scrum Lifecycle
[http://www.methodsandtools.com/archive/scrum1.gif – (based on Ken Schwaber and Mike Beedle’s work)
Based on Rugby: huddled team moving the ball towards the goal Claims to be a Project Management Framework Self-organization vital; 30-day iterations (Sprints)
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Agile Quality Assurance in Software Testing
• https://www.youtube.com/watch?v=AWgYBkvU8mY
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Relating Agile Methods with CMM (Capability Maturity Model) and ISO
9000
ISO = International Organisation for Standardisation https://www.iso.org/home.html
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
ISO 9000
• International set of standards for quality management (ISO 9000:2015, ISO 9001:2015, ISO 9004:2018, etc.)
• Applicable to a range of organisations from manufacturing to service industries
• ISO 9001:2015 specifies requirements for a quality management system for any organization that needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements and aims to enhance customer satisfaction, in all business sectors
• ISO 9001 is a generic model that must be instantiated for each organisation
33
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
ISO 9000 (cont’d):
• ISO 9004:2018 provides guidance for continual improvement of a quality management system to benefit all parties (employees, owners, suppliers, society in general etc.) through sustained customer satisfaction
• It should be used to extend the benefits obtained from ISO 9001:2000 to all parties that are interested in or affected by the business operations
34
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
ISO 9000 and quality management
ISO 9000 quality models
instantiated as
do cumen ts
Organization quality process
instantiated as
Project quality management
Or g an iz a t io n quality manual
is used to develop
Project 1 quality plan
Project 2 quality plan
Project 3 quality plan
S u p p or t s
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
ISO 9000 certification
• Quality standards and procedures should be documented in an organisational quality manual
• External body may certify that an organisation’s quality manual conforms to ISO standards
• Customers are, increasingly, demanding that suppliers are ISO 9000 certified
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
The Software Engineering Institute (SEI) Capability Maturity Model for Software (CMM)
Is a model for
• Judging the maturity of the software
processes of an organization
• Identifying the key practices that are required to increase the maturity of these processes
Level 5 Optimizing
Level 4 Managed
Level 3 Defined
Level 2 Repeatable
Level 1 Initial
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
CMM Levels
1. Initial – The software process is characterised as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics
2. Repeatable – Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications
3. Defined – The software process for both management and engineering activities is documented, standardised, and integrated into a standard software process for the organisation. All projects use an approved, tailored version of the organisation’s standard software process for developing and maintaining software
4. Managed – Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled
5. Optimizing – Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies
38
Professional Development (300578)
O p t i mi zi n g
Lecture 11 – Usability and Quality Assurance
Process change management Technology change management Defect prevention
CMM key process areas
Managed
Software quality management Quantitative process management
Defi ned
Peer reviews
Intergroup coordination Software product engineering Integrated software management Training programme Organization process definition Organization process focus
Repeatabl e
Software configuration management Software quality assurance
Software subcontract management Software project tracking and oversight Software project planning Requirements management
I niti al
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
The CMM and ISO 9000
• There is correlation between the key processes in the CMM and the quality management processes in ISO 9000
• The CMM is more detailed and prescriptive and includes a more detailed framework for improvement
• Organisations rated as level 2 in the CMM are likely to be ISO 9000 compliant
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
Key points
• Software quality management is concerned with ensuring that software meets its required standards
• Software Methods & Lifecycles play an important part in Quality assurance & Testing
• Agile methods like SCRUM and XP are popular as they manage Expectations and enhance Quality
• Agile needs to relate to ISO9000 & CMM
Professional Development (300578)
Lecture 11 – Usability and Quality Assurance
?