CMP2019M Human-Computer Interaction
Week 10 – Data Analysis I
Last Week
• Quantitative Evaluation in HCI – Part 1: Research methods
– Part 2: Data types
This Week
• DataAnalysis
– Recap: Data types
– Descriptive statistics
• Distributions
• Central tendency (averages) • Spread
– Discovering relationships • Scatterplots
– Task: Describing data
Reading
• InteractionDesign:Beyond Human-Computer Interaction, by Sharp, Rogers, & Preece
• Chapter 8.3: Simple Quantitative Analysis
Descriptive Statistics
Assuming you have just finished an evaluation…
Sample Data
PID
Gender
Age
Time (s)
Errors
SUS
1
F
22
45
3
71
2
M
23
34
4
73
3
F
21
30
4
71
4
M
22
58
3
72
5
F
24
44
2
73
6
M
21
35
5
65
7
F
23
18
7
60
8
M
20
49
3
69
9
F
25
43
3
68
10
M
26
15
8
71
How do you make sense of the data that you gathered?
How can you describe it in a meaningful way?
1. Distributions
2. Averages 3. Spread
Descriptive Stats: Distributions
• Distribution of data can give insights & will tell you which mathematical operations and statistical tests you can apply
• Uniform, bimodal, normal / Gaussian, and skewed distributions
• Many statistical tests require normal distribution of data – even things like the arithmetic mean!
Imagine you asked people to rate system usability on a 5-point Likert scale.
(You would need more than one item to assess usability – this is just for demo purposes!)
Remember: Likert Scale
I enjoyed using the interface.
12345
strongly agree O O O O O strongly disagree
• Asks user to judge a specific statement on a numeric scale that corresponds to agreement or disagreement with a statement
13
People can choose from five options, and if you ask several people, that can lead to different distributions of their answers.
Let’s look at what could happen with ten respondents.
Distributions: Uniform
Participant ID
Score
1
1
2
1
3
2
4
2
5
3
6
3
7
4
8
4
9
5
10
5
# of persons
3
2
1
0
Distributions: Uniform
12345
answer options
Distributions: Bimodal
Participant ID
Score
1
1
2
1
3
1
4
2
5
2
6
4
7
4
8
5
9
5
10
5
# of persons
4
3
2
1
0
Distributions: Bimodal
12345
answer options
Distributions: Normal
Participant ID
Score
1
1
2
2
3
2
4
3
5
3
6
3
7
3
8
4
9
4
10
5
# of persons
5 4 3 2 1 0
Distributions: Normal
12345
answer options
Distributions: Skewed
Participant ID
Score
1
1
2
2
3
3
4
3
5
4
6
4
7
4
8
4
9
5
10
5
# of persons
5 4 3 2 1 0
Distributions: Skewed
12345
answer options
Normal
54
Bimodal
4 3 2 1 0
vs
3 2 1 0
12345
12345
The average of both distributions is 3.
The question we asked was “I enjoyed using the interface.”
(1 = strongly agree, 5 = strongly disagree).
Normal
54
Bimodal
4 3 2 1 0
vs
3 2 1 0
12345
= some people enjoyed it, some didn’t, and most thought it was average.
12345
= half of the people really liked it, the other half really didn’t.
What are the implications for developers?
Think game vs. government web service.
Task: Distributions
Draw and name the following distribution.
Participant ID
Score
1
1
2
2
3
3
4
1
5
4
6
4
7
2
8
3
Task: Distributions
Uniform distribution:
3 2
1
0
1234
answer options
# of persons
How can we incorporate these differences in the way we report our results?
1. Distributions
2. Averages
3. Spread
Descriptive Stats: Averages
• “A value that describes the entire distribution.”
• Mode
• Median
• Arithmeticmean(“average”)
Averages: Mode
• Most frequent value in dataset
Normal
54
Bimodal
4 3 2 1 0
3 2 1 0
12345
Mode = 3
12345
Mode = 1 AND Mode = 5
Averages: Median
• Value that splits dataset at 50%
Normal
54
Bimodal
4 3 2 1 0
3 2 1 0
12345
Median = 3
12345
Median = 2 OR Median = 4* Median = 3**
* “Rolling the dice” approach ** Calculation, imaginary value (2+4/2=3)
Averages: Arithmetic Mean
• Sum of all measurements divided by number of observations that were made (usually participants)
Normal
54
Bimodal
4 3 2 1 0
3 2 1 0
12345
Mean = 3
12345
Mean = 3
Descriptive Stats: Averages
• “A value that describes the entire distribution.”
• Mode – works on all types of data
• Median – ordinal data and up
• Arithmeticmean(“average”)–continuousdata
• Dependingondistribution,calculationmethodleads to different results and is more or less appropriate
• Arithmetic mean is vulnerable regarding extremes
Descriptive Stats: Averages
2005/2006, http://www.gov.scot/Publications/2007/07/18083820/4
Remember: We’re trying to quantify reality.
Think of the average as a statistical model – we need to assess whether it is a good fit.
1. Distributions 2. Averages
3. Spread
Descriptive Stats: Spread
• Reporting the spread of data allows you to give additional information, e.g., outline extremes, and demonstrate whether your model of choice is a good fit for sample (e.g., whether arithmetic mean accurately describes data)
• Range
• Deviances: Total Error, Sum of Squared Errors, Variance, and Standard Deviation
Spread: Range
• Highest and lowest value in a set of observations
RangeT = 13 to 44
RangeE = 2 to 9
Useful when reporting data where extremes are of interest, e.g., age, …
PID
Time
Errors
1
33
2
2
17
7
3
44
2
4
37
3
5
14
9
6
42
2
7
13
8
8
15
5
Spread: Deviances
• Is the model (arithmetic mean) a good representation of our data?
• What is the deviation between model and observed data?
Average # of Errors: (2 + 7 + 2 + 3) / 4 = 3.5
PID
Time
Errors
1
33
2
2
17
7
3
44
2
4
37
3
Spread: Deviances MErrors = (2 + 7 + 2 + 3) / 4 = 3.5
8 7 6 5 4 3 2 1 0
1234
Spread: Total Error
MErrors = (2 + 7 + 2 + 3) / 4 =
Total error = sum of deviances = (𝒙𝒊−𝒙)
= (-1.5) + (3.5) + (-1.5) + (-0.5) = 0
3.5
Task: Total Error
Calculate the total error.
PID
Time
Errors
1
33
2
2
17
3
3
44
3
4
37
3
Average # of Errors:
(2 + 3 + 3 + 3) / 4 = 2.75
Total Error = (-0.5) + 0.25 + 0.25 + 0.25 = 0
Spread: Sum of Squared Errors
MErrors = (2 + 7 + 2 + 3) / 4 =
Sum of squared errors (SS) accounts for differences in direction:
SS= 𝒙𝒊−𝒙 (𝒙𝒊−𝒙)= 𝒙𝒊−𝒙
= (-1.5)2 + (3.5)2 + (-1.5)2 + (-0.5)2 = 17
3.5
𝟐
Spread: Variance
When estimating error in population:
Variance s2 = 𝑺𝑺 = 𝒙𝒊−𝒙 𝟐 𝑵−𝟏 𝑵−𝟏
Our example:
= ((-1.5)2 + (3.5)2 + (-1.5)2 + (-0.5)2) / 3 = 5.67
Degrees of Freedom
Degrees of freedom are the number of observations that are free to vary.
Example: Sample mean = 10, four observations: 8, 9, 11, 12
Change three values to 7, 15, 8 remaining value must be 10 to arrive at mean.
Spread: Variance
Average # of Errors: (2 + 7 + 2 + 3) / 4 = 3.5
PID
Time
Errors
1
33
2
2
17
7
3
44
2
4
37
3
s2 = ((-1.5)2 + (3.5)2 + (-1.5)2 + (-0.5)2) / 3 = 5.67
Spread: Standard Deviation
Variance gives squared error – not always realistic.
Standard Deviation takes root:
𝒙𝒊−𝒙 𝟐 𝑵−𝟏
s=
Spread: Standard Deviation
Average # of Errors: (2 + 7 + 2 + 3) / 4 = 3.5
PID
Time
Errors
1
33
2
2
17
7
3
44
2
4
37
3
−𝟏.𝟓 𝟐+ 𝟑.𝟓 𝟐+ −𝟏.𝟓 𝟐+ −𝟎.𝟓 𝟐 𝟑
s=
=
2.38
Spread: Standard Deviation
MErrors = (2 + 7 + 2 + 3) / 4 = 3.5 SD = 2.38
8 7 6 5 4 3 2 1 0
1234
Remember: Averages
2005/2006, http://www.gov.scot/Publications/2007/07/18083820/4
Descriptive Stats: Spread
• Reporting the spread of data allows you to give additional information
• Range – ordinal data
• Variance–continuousdata
• Standard Deviation – continuous data
• For our example, arithmetic mean is not a great fit – does not represent skewed distribution, always check distribution of data before choosing model
How can you explore relationships within that data?
Sample Data
PID
Gender
Age
Time (s)
Errors
SUS
1
F
22
45
3
71
2
M
23
34
4
73
3
F
21
30
4
71
4
M
22
58
3
72
5
F
24
44
2
73
6
M
21
35
5
65
7
F
23
18
7
60
8
M
20
49
3
69
9
F
25
43
3
68
10
M
26
15
8
71
Scatterplots
PID
Time
Errors
1
33
2
2
17
7
3
44
2
4
37
3
5
14
9
6
42
2
7
13
8
8
15
5
10 8 6 4 2 0
Time vs. Errors
0 20 40 60
Regression analysis and hypothesis testing follow up on that – more next week, and in Project Preparation in year 3 and Research Methods in year 4.
Exercise:
Exemplary data analysis
Sample Data
PID
Gender
Age
Time (s)
Errors
SUS
1
F
55
45
6
71
2
M
23
34
3
73
3
F
53
30
5
71
4
M
22
58
3
72
5
F
24
44
2
73
6
M
46
35
5
65
7
F
55
43
7
60
8
M
20
49
1
69
9
F
25
43
2
68
10
M
58
57
8
71
Task 1: Descriptives
Give an overview of participant gender (frequency diagram) and age (average and range).
Task 1: Sample solution Average age:
55 (Mode)
25 / 46 (Median)
38.1 (Arithmetic mean)
Age range: 20 to 58
Frequency (Gender):
6 5 4 3 2 1 0
Female Male
Task 2: Descriptives and interpretation
Calculate averages for error rates, completion time, and SUS score. Describe the spread of data.
For each variable, choose the most appropriate approach: Mode, median or arithmetic mean?
Range, variance, or Standard Deviation?
What does the data tell you about the system that was tested?
Task 2: Descriptives and interpretation
Error rates:
3 / 5 (Median), Range 1-8
4.2 (Arithmetic mean), SD = 2.35
Completion time:
43 / 44 (Median), Range 30-58 43.8 (Arithmetic mean), SD = 9.25
SUS score:
71 (Median), Range 60-73
69.3 (Arithmetic mean), SD = 4.08
Task 3: Relationships
Create a scatterplot for errors & time, and for errors & age.
10 8 6 4 2 0
Time vs. Errors
PID
Time
Errors
1
33
2
2
17
7
3
44
2
4
37
3
5
14
9
6
42
2
7
13
8
8
15
5
0 20 40 60
Task 3: Relationships
Errors vs. Age
Errors vs. Time
70 60 50 40 30 20 10
0
0 5 10
70 60 50 40 30 20 10
0
0 5 10
What are the difficulties when trying to determine relationships within the data?
Interactions, confounds, correlation and causality, and asking the right research question.
Be honest about your results.
The goal of statistical data analysis is to describe data and explore relationships within it.
It is important to choose right description and analysis approaches to accurately represent and interpret results.
Next week:
Data Analysis II – Hypothesis testing etc.