程序代写代做代考 Excel IOS graph html go User behaviour analysis a.k.a. Usability testing

User behaviour analysis a.k.a. Usability testing
1

Interface Hall of Fame or Shame?
Submit these to the Canvas Discussion Board “Interface Hall of Fame or Shame” and I’ll pick a couple to be discussed in the lecture
RMIT University©2020 CS&IT

Quick Recap
3

Iterative Design Process
• Ideo’s human centred design process: http://www.designkit.org/resources/1

Feedback types?
• • • •
•
•
•
(Paper) Prototypes
Surveys Interviews Usability tests
Other (see Usability Engineering) Focus groups
Log analysis
5

Formative vs summative
Formative (evaluative)
— >
Build
— >
Summative
Ethnography
Interviews
Surveys
Cultural Probes
Focus Groups
Diary Studies
Experience Sampling Studies
Studying Similar Products
Interaction Logging of Past Product / Early Prototype Studying Past Product Documentation
Ethnography
Interviews
Surveys
Focus Groups
Diary Studies
Experience Sampling Studies Interaction Logging
RMIT University©2020 CS&IT
6

Rapid Prototyping and Mockups
7

Fidelity
• Low fidelity
– more like a sketch missing details.
• High fidelity
– similar to finished product.
8

Fidelity
“The low-versus high-fidelity debate lies in the fidelity of prototype required to illustrate a concept, model design alternatives, or test an application. The debate rages to whether prototypes need to be complete, realistic, or reusable to be effective.”
Jim Rudd, Ken Stern, Scott Isensee, Low vs. High Fidelity Prototyping Debate, Interactions Magazine, 1996
9

Fidelity
Jim Rudd, Ken Stern, Scott Isensee, Low vs. High Fidelity Prototyping Debate, Interactions Magazine, 1996
10

Horizontal and Vertical Prototypes
• Horizontal prototypes
–Prototype across a breadth of features.
• Vertical prototypes
–Exploring a specific feature in detail.
11

Look and Feel Fidelity
• Look – prototype appearance.
• Feel – how the user interacts with the prototype.
• Aesthetics is not a concern. However the prototype should accurately and clearly portray the interaction options.
12

Building a Paper Prototype
• Cut up paper pieces for screen elements: – windows
– buttons
– menus
– dialogue boxes
• Hand draw the elements as necessary. • Get acquainted with pens and pencils.
13

Paper Prototype Interaction
• Point with finger = click
• Writing = typing input
• Someone simulates the computer’s response to user’s interactions:
– rearranging pieces
– writing custom responses
– verbal description of complex effects

Benefits
• A valuable testing tool
• Even if just the designer/implementors are doing paper
prototyping:
– puts the designer in the role of the user
– shows present and missing functionality
– shows how navigation and task flow will work
15

Limitations
• Look and feel difficult to be evaluated.
• Response time cannot be simulated.
• Users are more deliberate – they don’t bash the system, or be rude, mostly.
16

Mock ups
• Same role as paper prototypes. • Created digitally or drawn.
• Balsamiq
• Mockingbird
• Axure
17

Digital and Software Prototyping
• Becoming increasingly popular and useful as the software matures in terms of capability and ease of use.
• Examples: Balsamiq, Axure.
• These allow you to create high fidelity mock ups with scripted interfaces so you could test them.
18

Balsamiq
19

Axure
20

Invision
Prototyping with Invision at Uber https://www.youtube.com/watch?v=xwlCt_eM8BM
21

Figma
https://www.figma.com/
22

“Digital Paper” prototyping
• Example is “Pop”on iOS • Simulates paper prototyping.
23

Video prototyping
Prototyping for Elmo’s Monster Maker iPhone App. (IDEO)
https://www.youtube.com/watch?v=- SOeMA3DUEs&feature=youtu.be
24

Video prototyping
• Behind the scene:
• http://www.slate.com/blogs/the_eye/2013/10/23/the_importance_of
_prototyping_creative_confidence_by_tom_and_david_kelley.html
25

3-dimensional framework
• Attitudinal vs. Behavioral
• Qualitative (formative) vs. Quantitative (summative)
• Context of Use
26
https://www.nngroup.com/articles/wh

3-dimensional framework
• Attitudinal vs. Behavioral
• Qualitative (formative) vs. Quantitative (summative)
• Context of Use
https://www.nngroup.com/articles/wh ich-ux-research-methods/
27

Usability problems – can you spot it?
Stanstead airport toilet, https://flic.kr/p/9bjhd
RMIT University©2016 CS&IT 29

Usability problems – can you spot it?
• Where do I press to select the pump I want to use?

RMIT University©2016 CS&IT 30

How to get evidence of usability problems in UI?
• Eye tracking
• Interaction logging
• Those are tools in user testing
RMIT University©2016 CS&IT 31

Conducting Usability Studies
32

What are Usability Studies?
• Also known as user testing.
• Measured observation of users’ behaviour as they engage
with your application.
• Uses the scientific methods like metrics, data and measurements to prove assumptions.
33

What is the goal of a usability study?
• To measure the effectiveness of a feature set within your application.
• It all really depends on the development model or the requirements of the clients.
• To do this you establish metrics such as time taken, or number of errors during a test.
34

Usability Testing – examples
Usability testing of fruit https://www.youtube.com/watch?v=3Qg80qTfzgU
RMIT University©2016 CS&IT 35

Components of a test
•Test plan
• Screener
• Script
• Test resources
• Location
• Equipment • People
• Post test survey
36

Creating A Test Plan
• Just like how you need to create a design plan, a usability test requires a solid plan as well.
• So what is required in that plan? • Well other than the introduction.
37

Creating A Test Plan
The 1-Page Usability Test Plan by David Travis (Image Source: UserFocus)
38

A Good Screener
• This means an excellent survey to identify the exact users you wanted to test.
• Just like how you need to poll the ideal or targeted users for information during the design phase.
• Make sure to avoid selection bias in the questions.
39

Screener questions
• Bad
–List your favourite Web sites
–Do you check online news every time you surf the Web?
–What do you love most about Wired News?
• Good
–Write a list of the Web addresses (URLs) of sites that you go to often, or that you really like. Write up to 10 Web addresses.
–How often do you check online news?
–Several times a day –Once a day
–Which of these features in Wired News are important to you? (check all that apply)
–Number of stories on a given topic –Number of different topics covered
RMIT University©2016 CS&IT
40

A Tight Script
• During the test you will want to keep it as structured as possible and read/memorize off a readied script.
• This includes introduction, explaining the test, the equipment and the tasks involved.
• Important because you will want to provide the same test environment to all the testers for consistency.
41

Lab-based user testing
42

Formal usability lab
• Facility design
–Low-traffic low-noise area or good sound
insulation in walls
–Good acoustics and lighting
–Electrical, ventilation & ceiling requirements
–Double-pane one-way mirror • System design
–Digital audio/video/screen-activity monitor/capture
–Event-logging software –Editing system
RMIT University©2016
CS&IT
43

The Tasks
• The plan should include each tasks that will be given to the testers. This ideally should be in the form of a scenario.
• In the plan all the metrics and information pertaining to the task should be provided.
• For example, ideal time taken to complete task, starting state, finished states, extra information required, possible deviations.
44

Observation Charts/Tables
• Included in the plan should be the observation documents that will be used for note taking.
• A structured approach will make data analysis less painful and could easily provide connected themes.
45

Post Test Surveys
• Aside from observations being the main source of information, testers can also be provided with a questionnaire at the end of the survey.
• The results will affirm your observations and can provide you with Likert scale data.
• Open ended questions here can be used to gain further insights into the testers’ thoughts.
46

Conducting the test.
• Each task/scenario should be printed on separate pieces of paper and be provided to the user one at a time.
• Some users tend to get ahead of themselves or be confused if you give them a thick stack of documents.
• Be polite, friendly but firm.
47

Conducting the test.
• Encourage the tester to think aloud; silence is a wasted test. (Think aloud protocol)
• Vocalize their actions, thoughts and feelings.
• Ensure the testers feel comfortable voicing out their hatred
for your interface.
• Just ask them politely why.
48

Conducting the test.
• Don’t guide the tester through tasks • unless it is considered a fail.
• It’s not a training session.
• Impress upon the testers that it’s the application being
tested, not them.
• A usability test is not an opportunity to show off your shiny program, but to help you get your program shiny.
49

Conducting the test.
• Tester asks you if that is the right way to complete the task?
• Reply in the form of another question asking why they think that is so.
• Tester encounters a problem? • Ask them why.
50

Conducting the test.
• Could include a short questionnaire after each task.
• Time each attempt at a task, even if you are not focused on that aspect.
51

Usability Testing – examples
Usability testing of Windows 8 https://www.youtube.com/watch?v=v4boTbv9_nU
RMIT University©2016 CS&IT 52

Compiling your findings.
By organizing your comments, findings and measurements you can come up with justifications for your design or a change in design.
A very important technique is to identify grouped/themed issues from multiple comments and data.
53

Sample findings from eye tracking
• Banner blindness
• https://www.nngroup.com/articles/banner-blindness-old-and-new-findings/
RMIT University©2016 CS&IT 54

Sample findings from eye tracking
• F-shaped pattern for reading web content
• https://www.nngroup.com/articles/f-shaped-pattern-reading-web-content/
RMIT University©2016 CS&IT 55

Guerrilla user testing
56

Guerrilla user testing
• Guerrilla testing: ““the art of pouncing on lone people in cafes and public spaces, and quickly filming them whilst they use a website for a couple of minutes.” (Martin Belam)
• A type of usability testing intended for teams with limited resources — limited time, money, or people.
• https://uxmastery.com/guerrilla-ux-research/
RMIT University©2016 CS&IT 57

When to use guerrilla testing?
Best uses for guerrilla research
• You have a general consumer-facing product which requires no previous experience or specialty knowledge OR you can easily recruit your target participants
• You want to gather general first-impressions and see if people understand your product’s value
• You want to see if people can perform very specific tasks without prior knowledge
• You can get some value out of the sessions and the alternative is no research at all
RMIT University©2016 CS&IT 58

Where?
http://www.uxbooth.com/articles/the-art-of-guerrilla-usability-testing/ Check this video https://www.youtube.com/watch?v=0YL0xoSmyZI
RMIT University©2016 CS&IT 59

5-seconds usability test
• To capture first impressions and immediate reaction • To learn more see:
• https://measuringu.com/five-second-tests/
• Check this video https://www.youtube.com/watch?v=X0FG0jCqLYQ
RMIT University©2016 CS&IT 60

The number of users in usability testing
61

How many users needed?
• How many?
• Not very helpful answer: Depends.
• A little more helpful: What sort of information are you trying to obtain? How much value would this information bring?
• Who?
• Depends
• See earlier lectures
62

Nielsen suggests
63

How many users needed?
• Using 5 users to find 84% problems with each of three design iterations
• Using 15 users to find 99% of problems with one design iteration
(Nielsen)
RMIT University©2013 CS&IT 64

How many users needed?
• The bigger issue is actually approaching the right users. • Have a good mix.
• Balanced users
• Do not test them with fellow developers. You will end up spending hours debating the merits of using some CSS API like bootstrap or semantic UI…
65

Remote user testing
66

Web services for user testing
• http://www.usertesting.com/
• Specify the tasks you want users to perform. –Test on all devices: mobile, tablet, desktop.
• We recruit participants
–Users are available to test 24/7.
• You get results in 1 hour
67

Web services user testing
• Let’s see those videos http://www.usertesting.com/ • Another similar service: http://www.trymyui.com/
RMIT University©2016 CS&IT 68

Controlled experiment
69

Controlled experiment: A/B testing
• From Ronny Kohavi, Microsoft’s General Manager of the Analysis and Experimentation team & Stanford Professor http://ai.stanford.edu/~ronnyk/
• Concept is trivial
–Set your OEC = Overall Evaluation Criterion. –Randomly split traffic between two (or more)
versions
–A (Control) –B (Treatment)
–Collect metrics of interest –Analyze
http://www.exp-platform.com/Pages/KDD2015KeynoteExPKohavi.aspx
RMIT University©2016 CS&IT 70

The First Controlled Experiment
• The earliest controlled experiment was a test for
vegetarianism, suggested in the Old Testament’s Book of
Daniel
Test your servants for ten days. Give us nothing but vegetables to eat and water to drink. Then compare our appearance with that of the young men who eat the royal food, and treat your servants in accordance with what you see
http://www.exp-platform.com/Pages/KDD2015KeynoteExPKohavi.aspx
Ronny Kohavi 71

Real Example: MSN Home Page Search Box
• OEC: Clickthrough rate for Search box and popular searches A
B
Differences: A has taller search box (overall size is the same), has magnifying glass icon, “popular searches”
B has big search button, provides popular searches without calling them out
• Raise your left hand if you think A Wins (top)
• Raise your right hand if you think B Wins (bottom)
• Don’t raise your hand if they are the about the same
Ronny Kohavi 72

Real Example: Bing Ads with Site Links
• Should Bing add “site links” to ads, which allow advertisers to offer several destinations on ads?
• OEC: Revenue, ads constraint to same vertical pixels on avg
AB
• Pro adding: richer ads, users better informed where they land
• Cons: Constraint means on average 4 “A” ads vs. 3 “B” ads Variant B is 5msc slower (compute + higher page weight)
• Raise your left hand if you think A Wins (left)
• Raise your right hand if you think B Wins (right)
• Don’t raise your hand if they are the about the same
Ronny Kohavi 73

User testing in product development phase
74

Google Analytics – Behaviour Flow
• https://blog.crazyegg.com/2016/01/07/5-ways-expose-split-testing-ideas/
RMIT University©2016 CS&IT 77

Quantitative and Qualitative Research in Product Development Phase
1.STRATEGIZE: In the beginning phase of the product development, you typically consider new ideas and opportunities for the future. Research methods in this phase can vary greatly. 2.EXECUTE: Eventually, you will reach a “go/no- go” decision point, when you transition into a period when you are continually improving the design direction that you have chosen. Research in this phase is mainly formative and helps you reduce the risk of execution.
3.ASSESS: At some point, the product or service will be available for use by enough users so that you can begin measuring how well you are doing. This is typically summative in nature, and might be done against the product’s own historical data or against its competitors.
https://www.nngroup.com/articles/which-ux-research-methods/
RMIT University©2016
CS&IT 78

Quantitative and Qualitative Data in UX research
• Both are useful and complementary
• Qualitative data describes (e.g. give insights to nuance) • Quantitative data counts (measures the significance)
• UX/UCD research does both
• https://www.youtube.com/watch?v=qjsfblXdocg
RMIT University©2016 CS&IT 79

Week 6 Guest lecture
80

Next week’s guest lecture on Personas
Speaker: Rob Roe, Managing Director ANZ, OneTrust Topic: Mission Statement, Personas, and Use cases
Bio: Rob Roe was the Managing Director for AirWatch Australia & New Zealand from January 2011 until July 2015. AirWatch is an Atlanta based company and provides mobile device management solutions. During this time AirWatch in Australia grew from 0% to 50% market share and was recognised in 2013 and 2014 as the leading provider for mobile device management solutions in Australia. VMware acquired AirWatch in 2014 for $1.54billion. Rob is also a producer at the blog site FAQs for Orgs. Previously Roe spent nine years with Telstra in a variety of leadership positions, before joining AirWatch. He has also led teams in North America, Europe and Asia for IBM and Gartner.
At present time Rob is the country director of OneTrust, which provides data privacy services and awareness to companies from various industries.
RMIT University©2016 CS&IT 81

Next: Guest talk by Sam Nolan
Talk summary. Sam will give a talk about his recent work on exploring the impact of the lockdown on social roles and emotion, and also his pet project — Dot collector.
Sam Nolan’s brief bio
President of the RMIT Programming Club
Treasurer of the RMIT CSIT Society
Computer Science Honours Student, working under Flora with her PhD students. Currently working with Maria Spichkova on his honours thesis
Nolan SW, Rumi S, Anderson C, David K, Salim F. Exploring the Impact of COVID-19 Lockdown on Social Roles and Emotions while Working from Home. New Future of Work Symposium. 2020. https://arxiv.org/abs/2007.12353; https://www.microsoft.com/en-us/research/publication/exploring-the-impact-of-covid-19-lockdown-on-social-roles-and- emotions-while-working-from-home/
Dot collector is an application inspired by an tool from Ray Dalio’s “Principles” book by the same name. The application is based off the concept of micro feedback, where large quantities of very small pieces of feedback are collected. This application will give you the opportunity to give feedback with a single click many times during the lecture to evaluate Sam’s performance as a guest lecturer. Your feedback will be anonymous. This software is very much in alpha form, but feedback in regards to the application itself is welcome.
RMIT University©2020 CS&IT 82

Related Posts