Announcements
• You should be able to see the correct answers/feedback in your Midterm on CourseLink now
• Exemplar answers are now posted under Midterm Prep & Results folder
CIS 3760
Software Engineering
1
Some QA Humour…from Steph Beach, QA Geek
• https://www.youtube.com/watch?v=o2Qxf07kggM • https://www.youtube.com/watch?v=imrp-xFOx_I
CIS 3760
Software Engineering
2
CIS 3760: Software Engineering Software Quality –
Defect Prevention
Instructor: Prof. Stacey Scott
School of Computer Science
University of Guelph https://www.uoguelph.ca/computing/people/stacey-scott stacey.scott@uoguelph.ca
© 2017-21 Stacey D. Scott – with content from J. McCuaig, used with permission
Preventing Defects (Reviews & Other Static Analysis Approaches)
4
Taxonomy for Defect Handling Techniques
Methodology
Configuration Management
Exception Handling
Modular Redundancy
Static Analysis Methods
Verification/ Formal Methods
Unit Testing
Integration Testing
System Testing
Note: this is an incomplete taxonomy; other techniques exist, including usability testing
Defect Prevention
Defect Detection
Defect Tolerance
CIS 3760
Software Engineering
Slides adapted from Bruegge & Dutoit, Pearson online resources
5
Testing
Debugging
Defect Handling
Assigned Reading: Code Review White Paper
• PeerCodeReview:AnAgile Process from SmartBear Software
• https://www.slideshare.net/SmartBear_Software/peer- code-revew-an-agile-process
• Describes how Code Reviews can be incorporated into the Agile Software Engineering Process
• Also presents several real-world case studies of the value gained by employing code review
CIS 3760
Software Engineering
6
Static Analysis Approaches
• Inspections and code reviews – formalandinformal
– reviewmetrics
• Usability inspection techniques (optional) – E.g.,CognitiveWalkthrough
CIS 3760
Software Engineering
7
Inspection (aka formal technical reviews)
CIS 3760
Software Engineering
8
Slides adapted from Pressman, McGraw-Hill online resources
What Are Inspections / Formal Technical Reviews?
• a meeting conducted by technical people for technical people for the purpose of finding defects
• a technical assessment of a work product created during the software engineering process
– Designs,code,tests,documents,etc.
• a software quality assurance mechanism
• a context for training
CIS 3760
Software Engineering
9
Slides adapted from Pressman, McGraw-Hill online resources
What Inspections Are Not!
• A project summary or progress assessment
• A meeting intended solely to impart information
• A mechanism for political or personal reprisal (revenge or retaliation)!
CIS 3760
Software Engineering
10
Slides adapted from Pressman, McGraw-Hill online resources
Formal Technical Review Objectives
• To uncover errors in function, logic, or implementation for any representation of the software
• To verify that the software under review meets its requirements
• To ensure that the software has been represented according to predefined standards
• To achieve software that is developed in a uniform manner
• To make projects more manageable
CIS 3760
Software Engineering
11
Slides adapted from Pressman, McGraw-Hill online resources
FTR / Inspection Meeting Basics
• 2-5 people (typically) should be involved in the review
• Advance preparation should occur but should require no more than two hours of work for each person
• Duration of the review meeting should be less than two hours
• Focus is on a work product (e.g., a portion of a requirements model, a detailed component design, source code for a component)
• Inspections specifically involve using a checklist to guide the defect detection
CIS 3760
Software Engineering
12
Slides adapted from Pressman, McGraw-Hill online resources
Inspection Demonstration and Discussion (Watch at home: Video 1: 0-3:28, 4:21-7:23, 11:50-14:29)
Recorder Reviewer
CIS 3760
Software Engineering
Moderator Producer
Reviewer
Reviewer
Video #1: https://www.youtube.com/watch?v=Sn6gcvtpMhg
Video#2: https://www.youtube.com/watch?v=BvIvh5RD4Bk
13
Players in a FTR / Inspection
• Author/Producer: individual who created work product
https://www.dreamstime.com/stock-illustration-meeting-conference-round- table-no-gradients-image44789005
– Informsmoderatorwhenworkproductisreadyforreview – Providesclarificationofworkproductduringmeeting
• (LEADS REVIEW) Moderator: evaluates the product for readiness, distributes work product to two or three reviewers for advance preparation, moderates review meeting
• Reviewer(s): expected to spend between one and two hours reviewing the product, making notes, (or annotating a predefined checklist) and otherwise becoming familiar with the work
• Scribe/Recorder: reviewer who records (in writing) all important issues raised during the review
CIS 3760
Software Engineering
14
Slides adapted from Pressman, McGraw-Hill online resources
Conducting the Review
• Before meeting:
– AllocateresourcesandscheduletimeforFTRs
– Develop a checklist for each product that is likely to be reviewed
– Insist on advance preparation
– Conduct meaningful training for all reviewers
• During meeting:
– Setanagendaandmaintainit
– Limitnumberofparticipants;technicalstaffnotmanagement
– Review the product, not the author
– Focusondefectdetection,notsolvingtheidentifiedproblems
– Limitdebateandrebuttal
– Takewrittennotes
CIS 3760
Software Engineering
15
Slides adapted from Pressman, McGraw-Hill online resources
Common Review Metrics
• Totalrevieweffortand
total number of errors discovered are defined as:
• Ep : Preparation effort* • Ea : Assessment effort* • Er : Rework effort*
Ereview = Ep + Ea + Er
Errtot = Errminor + Errmajor
• Errminor : Minor errors** found • Errmajor : Major errors** found
• Defectdensityrepresentstheerrors found per unit of work product reviewed
• WPS : Work product size
(# of UML models, # of doc pages, # lines of code, etc.)
defect density = Errtot / WPS
*in person hours
**Minor/Major: error requiring less/more than some pre-specified effort to correct16
CIS 3760
Software Engineering
Slides adapted from Pressman, McGraw-Hill online resources
Why Bother??
• IBMfoundthateachhourofcodeinspectedpreventedabout 100 hours of related work (testing, defect correction, etc.)*
• Studyoflargeprogramsfoundeachhourspentoninspections saved an average of 33 hours of maintenance work and that inspections were up to 20 times more efficient than testing*
• Reviews&inspectionsare widely used in industry; often occur before testing begins*
Recall: the relative cost of fixing a defect substantially increases as the later the defect is discovered
• 2017 SmartBear study
of 550 s/w professionals
across 30 industries
found 74.1% of
respondents use code reviews
CIS 3760
Software Engineering
17
*from McConnell, Code Complete 2nd ed.
Code Reviews
• Code author writes/modifies code
• 1+ other developers review code changes by the author either using a checklist (formal) or not (informal), sends author any issues found
• Author addresses changes, asks reviewer to approve fixes
• Author’s code is approved and checked into the common code base (e.g., main dev branch)
CIS 3760
Software Engineering
https://www.michaelagreiler.com/code-review-best-practices/ 18
From SmartBear’s Article
“In some Scrum teams, developers invest a target percentage of their overall capacity performing code reviews. … They also apply best practices around defining tasks that are granular enough so that the code requiring review is in manageable chunks and does not overwhelm the review process.”
CIS 3760
Software Engineering
19
Code Review Case Study: $200K+ savings
• 10,000 LOC project, produced over 3 months
• Company tracked how many bugs were found by QA and customers in following 6 months
• Then, went back and had other developers peer review same code
• Result: Code review would have saved more than half the cost of
fixing bugs, AND they would have found 162 additional bugs.
463
463
180
321
194
113
132
20
Advice for Successful Code Reviews*
• Author should read the changes carefully
• Aim for small, incremental changes
• Cluster related changes
• Describe the purpose and motivation of the change
• Run tests before submitting a code review; automate when possible
• Minimize number of reviewers
• Add experienced reviewers to get insightful feedback
• Add junior developers to let them learn
• Be open to suggested changes
CIS 3760
Software Engineering
*https://www.michaelagreiler.com/code-review-best-practices/
21
Potential “Checklist” Topics for Code Review…Why the human is needed
CIS 3760
Software Engineering
Source: https://www.evoketechnologies.com/blog/code-review-
22
checklist-perform-effective-code-reviews/
Informal Review: Walk-Through
A walk-through involves two or more people discussing a design or code
Typical characteristics:
• Usually hosted by author of the design or code under review
• Focuses on technical issues—it’s a working meeting
• Participants prepare by reading the design/code and looking for errors
• Is a chance for senior programmers to pass on experience and corporate culture to junior programmers, and for junior programmers to present new methodologies and challenge assumptions
• Lasts 30-60 minutes
• Emphasis is on error detection, not correction
CIS 3760
Software Engineering
23
Informal Review: Code Reading / (informal) Code Review
A code reading usually involves two or more people reading code independently and then meeting with the author of the code to discuss it
Typical characteristics:
• Author distributes source code listings to code readers. Listings are from 1000-10,000 lines of code, 4000 lines is typical
• 2+ people read the code
• Code reading done independently, estimate ~1000 lines / day
• After code reading, readers meet with author for 1-2 hours (optional)
• Meeting focuses on code defects identified by readers, not all code is discussed
• Author fixes the identified defects by the reviewers.
CIS 3760
Software Engineering
24
…
McConnell, Code Complete 2nd ed., Ch 21 “Comparison of Collaborative Construction Techniques
25
Additional Materials (Optional)
26
Static Analysis Approaches
Reviews (inspections and code review) – formalandinformal
– reviewmetrics
Usability inspection techniques (optional) – E.g.,CognitiveWalkthrough
CIS 3760
Software Engineering
27
Usability inspection techniques
• Methods that involve expert evaluators inspecting the interface against some set of criteria
• Typically quicker and less expensive than experimental methods involving data collection from actual user
• Can often be applied early in the system development cycle to help identify defects before costly implementation and/or formal evaluations occur
CIS 3760
Software Engineering
28
Sample usability inspection technique: Cognitive Walkthrough
Cognitive walkthrough (CW)
• an explicit procedure is used by a usability professional to simulate a user’s problem solving process through the system to check if the user’s goals and memory capabilities would likely lead to the next correct action
• Participants:
– Designer, who provides the system / prototype, and information about the users, task goals, and usage scenarios
– Usability expert, who will conduct the CW
CIS 3760
Software Engineering
29 (Nielsen, 1995)
Cognitive Walkthrough Details
• Focuses on ease of learning
• Designer presents a partial or full design & one or
more descriptive usage scenarios
• Expert is told the assumptions about the user
population, context of use, task details
• One or more expert/evaluator walks through the design prototype with the scenario
• Experts/evaluators are guided by 4 questions
CIS 3760
Software Engineering
30
Slides adapted from www.id-book.com online resource.
Recall: Static Analysis Approaches
Reviews
– formalandinformal – reviewmetrics
Usability inspection techniques – E.g.,CognitiveWalkthrough
CIS 3760
Software Engineering
31
Cognitive Walkthrough: The 4 Questions*
1. Will the user realistically be trying to do this action?
2. Is the control for the action visible?
3. Is there a strong link between the correct action and what the user is trying to achieve? (i.e. between the control and the action)
4. If the correct action is performed, is the feedback appropriate?
As the experts work through the scenario they record problems.
CIS 3760
Software Engineering
32
Slides adapted from www.id-book.com online resource.
Video Demonstration
• https://www.youtube. com/watch?v= Edqjao4mmxM
• And
• https://www.youtube.com/watch?v=
BNO5OkA0R44
CIS 3760
Software Engineering
33
Given the following “relative costs of fixing defects” graph, explain the benefits of usability inspection techniques such as Cognitive Walkthrough to a Software Project
CIS 3760
Software Engineering
34
Review
• What is a static analysis and how can it help improve software quality?
• What is a usability inspection technique? – Purpose?
– Whoconductsthem?
• What is a cognitive walkthrough?
– Howisitconducted?
– Whattypesofworkproductscanbeevaluated?
CIS 3760
Software Engineering
35
Review
• What metrics are commonly collected to assess the value of a formal technical review (aka inspection)?
• What type of work products are typically reviewed?
• What is a code reading (aka code review) and how
does it differ from a code inspection?
CIS 3760
Software Engineering
36
Review
• What are some example responsibilities of an SQA professional?
• What is the difference between a static and dynamic analysis?
– Whatisauniqueadvantageofstaticanalysismethods?
• What is a key defining characteristic of a formal technical
review (also called an inspection)?
CIS 3760
Software Engineering
37