CMP2019M Human-Computer Interaction
Workshop 10: Data Analysis II
Part 1: Theory
1. Explain the concept of statistical significance. Write no more than 2-3 sentences.
2. What does a p-value of 0.032 tell us, and what about a p-value of 0.500? Write no more than 2-3 sentences.
3. Under which conditions can you apply a t-test? Write no more than 2-3 sentences.
4. Name one thing a t-test cannot do for you. Write no more than 1-2 sentences.
Part 2: Practical Exercises
1. You run a user study with 120 participants, who are asked to answer a range of usability-related questions for two different systems on five-point scales (1=worst, 5=best). You run a t-test on your data.
Interpret the following outcomes in 1-2 sentences each.
a) You calculate a p-value of 0.002, and a confidence interval of 1.9 (lower bound) and 2.2 (upper bound).
b) You calculate a p-value of 0.035, and a confidence interval of 0.1 (lower bound) and 0.2 (upper bound).
c) You calculate a p-value of 0.051, and a confidence interval of 0.8 (lower bound) and 2.3 (upper bound).
d) You calculate a p-value of 0.798, and a confidence interval of 0.1 (lower bound) and 2.9 (upper bound).
2. For each of the previous results, write 1-2 sentences on their implications for your system variants, and the actions developers should take (if any).
3. Working in pairs, research parametric and non-parametric statistical tests. What is the key difference? When would you choose a parametric test, and when would you choose a non- parametric option? Write no more than 7-8 sentences.