CMP2019M Human-Computer Interaction
Week 12 – Qualitative Methods
•
– Research methods
Last few Weeks
Quantitative Evaluation in HCI
– Data types
– Data Analysis
– Statistical Significance
• • •
This Week
What is Qualitative Research?
Empathy in Design
Qualitative Methods
– Interviews, Focus Groups and Questionnaires – Thematic Analysis & Grounded Theory
Quantitative – Concerning “quantity” Qualitative – Concerning “quality”
Quantitative result: passengers rate the design of the train as 6.2/10 average
Quantitative result: passengers rate the design of the train as 6.2/10 average
Qualitative result: passengers wonder why the train is so sad :*(
What is qualitative research?
•
than simply their quantity.
Understanding the quality of phenomena rather Typically less participants than quantitative analysis,
•
but more in-depth data gathering and analysis.
•
made, as well as what, when where etc.
Aimed at understanding why and how decisions are
•
unstructured interviews, focus groups, diaries etc.
Methods such as ethnography, semi-structured &
What is qualitative research?
•
Useful when the data you are interested in are:
•
meaningfully reduced to numbers.
Not easily observable – can’t be easily or
– Feelings, values, abstract concepts
•
– Ethics, feasibility
Cant be experimentally manipulated
•
– more variables than you can measure
– known variables interact in unpredictable ways
complexity
What is qualitative research?
•
– Quotes from participants
Qualitative data can come in the form of:
– In-depth descriptions of observed behaviour – Images
– Narrative
Why is qualitative research useful for HCI?
• •
Helps us design for Empathy and Experience Helps us design for Cultures and Contexts
Empathy in Design
•
Rain sensor in the farm in Cyprus
•
years
•
Only happens once!!!!
Blossom
The “flower” opens once enough rain has fallen – may take
•
experiences and the lasting quality of many of our feelings
“echoes and values the fleeting quality of many of our
for other people. It uses digital technologies as a way to harness the ephemeral characteristics of a flower blossoming, rather than for the more common uses of digital technologies of repeatability and immediacy.”
http://www.northumbria.ac.uk/static/5007/2008pdf/wallace.pdf
Poultry Internet
• • •
Your pet chicken wears a haptic waistcoat You have a chicken doll in your office Stroke your doll → stroke your chicken
•
questionnaires.
Poultry Internet
How do we design for chickens? They can’t fill out
– Poultry Internet is informed by the rich relationship between pet and owner. It is not technically complex, but is built from an empathetic understanding of that relationship.
http://dl.acm.org/citation.cfm?id=1149823
Ok, we can Design for qualities, but what about Evaluation?
•
the users
Qualitative Evaluation
Extracting data on the “qualities” of a system from
•
aspects make them feel this way?
How does the system make them feel? What – How/Why is Fallout 4 fun?
– How/Why is Blackboard frustrating?
•
subjective experience
We can’t predict the answers without understanding
Gathering Qualitative Data
•
– Focus groups
Common approaches:
– Interviews
– Open Questionnaires
•
requirements.
Ring a bell?
We’ve done this before as part of gathering
•
explore opportunities for design.
In that activity we used lo- and mid-fi prototypes to
•
understand qualities of the experience!
We can do the same with the real system to
•
conclusions
This time, we’ll do some analysis to draw out
•
Discussing experience with specific user groups
Focus Groups
•
that?”) but letting users explain their experience
Steering discussion and probing (“why do you say
•
– Social, personal interaction
These are very commonly used
– Public discussion
– Highlights agreement, disagreement – People express their understanding
Real example – The black cat
Un/Semi-Structured Interviews
•
• • •
Exploratory conversations around a particular topic
– Can be guided, or based on probing questions by experimenter
Questions are open
No expectation of the format or content of answers
Useful when you want to explore the range of possible opinions
Open Questionnaires
• Askingopenquestionsandgettinglongform answers.
• You will be familiar with these:
– “What was the best thing about CMP2019M HCI and why?”
• Theresultsforyourmodulefeedbackareanalysed in exactly these ways!
– The qualitative responses are more useful than knowing students rate the module as a 8/10
•
messy data
Data
These methods result in lots of unstructured and
•
consuming)
•
Audio must be transcribed (Extremely time How do we make sense of this data?
Thematic Analysis
• This allows us to draw together data based on abstract themes.
• The themes are always emergent from the data. The process is inductive
• First, we have to read and understand the data
• Then we break it down into “chunks”
• Thenwecodethedata
Reading and Chunking
• Separate big blocks of text into smaller ones.
• Eachblockshouldbeanidea – Usually sentences
• Readmultipletimes – Read closely
• We aren’t interested in the specific words that are used, but rather the meaning of them. What is this person telling us?
Coding (the other kind)
•
philosophy behind the approach & what you are trying to find out
Many ways of doing this, depending on the
– Identifying recurring patterns or themes – Categorising data
– Examining critical incidents
•
& results are compared for agreement.
This is done independently by at least 2 researchers – (we won’t bother with that bit)
Identifying recurring themes
•
• • •
This is simplest type of analysis
– Often this is done first, then built upon with other methods
It is a high level analysis
You are intending to get a broad picture
It should be completely inductive
– You haven’t decided in advance what the data may contain
– Allow yourself to be surprised / challenged
Identifying recurring themes
•
or themes that emerge
Read the text, noting on the transcript any patterns
– Behaviour
– Places / contexts where important events happen – Values
– Cool / not cool
•
•
(one participant, an excerpt of the transcript)
Identify themes and patterns from a sample of data Then try and apply those to the rest
Identifying recurring themes
•
important themes from your data.
At the end you should have a list of the most
•
up your point.
•
– Because sentences will be tagged with themes This will help you report your data
But also, you will have easy to find raw data to back
•
Much more detailed and technical analysis
•
existence in advance
Categorising data
Often (but not necessarily) there are categories in
– Coding for usability issues
– Could use schneidermans golden rules etc.
•
structured categories
Appropriate when you want the data in the form of
– Problems with menu navigation
– Frustration over interface features
Examining critical incidents
•
necessary to code all of it.
If you have massive quantities of data, often it is not
•
behaviours that make a significant contribution to the activity
This type of analysis limits analysis to those
– Problem with definition of this – especially if novice
•
critical incidents, with the remainder of data serving as context.
In HCI – helps focus the analysis around these
•
What you report depends on the analysis done
•
your analysis.
• • • •
Thematic Analysis:
State a theme
An explanation of that theme
A quote / quotes illustrating that theme
Reporting
Generally, use evidence from the text to support
•
conducted recently.
•
Instead of interview transcript, we have tweets
Thematic Analysis
We will use the example of a study that we
•
communication on Twitter
Interested in peoples health disclosure &
•
A., & Lawson, S. I can’t get no sleep: discussing
Jamison-Powell, S., Linehan, C., Daley, L., Garbett,
#insomnia on twitter. In Proceedings of ACM CHI (2012) pp. 1501-1510.
•
Insomnia tweets
•
communication technology
•
themes.
Exercise
Understanding peoples use of twitter as health
Step 1: Read through page 1, noting any patterns or
– Be as specific as possible – Make a list of all themes
•
– Refine themes if necessary (it will be)
Step 2: Apply themes to analysis of page 2
• •
You have now done a thematic analysis You must report those themes
•
– State a theme
•
Lets look back at the paper
Step 3: Reporting
Exercise
– An explanation of that theme
– A quote / quotes illustrating that theme
•
This will help you identify the challenges and values
•
to participants
Summary
•
is to make sense of it, in terms of subjective experiences, but in an objective way.
The point of doing qualitative analysis on your data
Particular types of interaction may seem preferable
– Some types are not
•
not (values)
Some behaviours (habits) can be changed, some
Remember
• Qualitativeevaluationisforunderstandingthequalitiesofthe experience
– Focus on smaller numbers of users, or specific aspects of a system
– Helps understand value in a richer sense
• Quantitativeevaluationisforunderstandingthebroaderpicture
– Larger number of users
– Efficiency, general effectiveness of a system – Understanding the “average” user
• Theyarecomplementaryapproaches.Oftenamixedapproach is the best one!