PowerPoint Presentation
Basic VUI Design
Principles
Copyright By PowCoder代写 加微信 powcoder
Human-AI Interaction
Lecture 14
▪ Conversational design
▪ Command-and-control vs. conversational
▪ Conversational markers
▪ VUI Design Process
▪ Design tools
▪ Confirmations
▪ Error handling
This lecture – key concepts of VUI design
COMP3074-HAI Lecture 14, Basic VUI design principles
Part 1. Conversational design
(2017). Designing Voice User Interfaces
▪One shot interaction vs. multiple turns
▪Current smartspeakers mostly one shot
▪ The wake word needed to explicitly invoke the interaction “Alexa…”
▪ The device then ‘listens’ for a predetermined period, e.g., until 2s
after it ‘hears’ the last input
▪So what does conversational mean?
▪ The answer is complicated, but for the purposes of VUI it means
things like
▪Acknowledging ”thank you” with “you’re welcome”
▪ These are called conversational markers
Command-and-control vs. conversational
COMP3074-HAI Lecture 14, Basic VUI design principles
▪ Timelines (“First”, “Halfway
there” and “Finally”)
▪Acknowledgments (“Thanks,”
“Got it,” “Alright,” and “Sorry
about that”)
▪Positive feedback (“Good job,”
and “Nice to hear that”)
▪Greetings (Hi, Hello, Bye etc.)
▪BUT this may raise concerns of
trying to be too human, needs
to fit VUI persona
Conversational markers
COMP3074-HAI Lecture 14, Basic VUI design principles
VIRTUAL ASSISTANT
I’ll be asking you a few questions about your health.
first, how many hours of sleep did you get last night?
about seven.
VIRTUAL ASSISTANT.
Good job. and how many servings of fruits and
vegetables did you eat yesterday?
Maybe four.
VIRTUAL ASSISTANT
Got it. last question—were you able to take your
medication last night?
VIRTUAL ASSISTANT
Alright. That’s it for now. I’ll talk to you again tomorrow.
▪Moving towards multiple turn-based interactions
▪Bad example:
Conversational design
COMP3074-HAI Lecture 14, Basic VUI design principles
Ok Google. When’s my next appointment?
You have a calendar entry tomorrow.
The title is “Chinatown field trip.”
Ok Google. Can you please repeat that?
▪A better example:
Conversational design
COMP3074-HAI Lecture 14, Basic VUI design principles
Ok Google. Who was the 16th president of the United States?
abraham lincoln was the 16th president of the United States.
How old was he when he died?
abraham lincoln died at the age of 56.
Where was he born?
Hodgenville, KY
What is the best restaurant there?
Here is Paula’s Hot Biscuit:
▪Some principles
▪Keep track of ‘context’
▪Note, what context means is a hard problem, see pragmatics
Conversational design
COMP3074-HAI Lecture 14, Basic VUI design principles
Ok Google. Who was the 16th president of the United States?
abraham lincoln was the 16th president of the United States.
How old was he when he died?
abraham lincoln died at the age of 56.
Where was he born?
▪Coreference resolution
▪Different words
pointing to the same
▪Pronouns and
indexicals
▪Need NLP to do this
▪Some principles
▪Keep track of ‘context’
▪ Let the user decide
▪ how long the conversation should be
▪ also, when to initiate the conversation, see unpopularity of
recent Alexa ‘upgrade’ → https://www.bbc.co.uk/news/technology-54915777
▪Set user expectations
▪ “don’t ask a question if you won’t be able to understand the
Conversational design
COMP3074-HAI Lecture 14, Basic VUI design principles
https://www.bbc.co.uk/news/technology-54915777
▪Some principles
▪Provide clear, actionable prompts
▪E.g., “Do you want to send it or change it?” (after the user has
finished writing an email…)
▪Be concise, speech is ephemeral
▪Usually avoid long prompts with many options, etc.
▪Help the user understand what they can do
▪Discoverability is an important principle – we’ll get back to this
▪Help the user recover from errors
▪E.g., by providing status updates on connectivity
▪ but also by providing contextual help
Conversational design
COMP3074-HAI Lecture 14, Basic VUI design principles
Part 2. VUI Design Process and Tools
(2017). Designing Voice User Interfaces
▪How do you do effective VUI design?
▪VUI design process, and tools to create VUIs
▪VUI from early concept to fully working prototype
▪VUIs, like other UIs best created following a UX design process
▪User-centred design (UCD), human-centred design (HCD)
▪ HCD (ISO standard 9241) is an approach to make systems usable and useful
by focusing on the users, their needs and requirements and by applying
human factors/ergonomics, and usability knowledge and techniques. This
approach enhances effectiveness and efficiency, improves human well-
being, user satisfaction, accessibility and sustainability; and counteracts
possible adverse effects of use on human health, safety and performance.
▪How do we do that for VUIs?
VUI Design Process
COMP3074-HAI Lecture 14, Basic VUI design principles
▪Support process from early concept to fully working prototype
▪Sample dialogs
▪Visual mockups
▪Prototyping Tools
▪Voiceflow, Dialogflow (Google), Wit.ai (Facebook), Nuance Mix
▪Platforms
▪Rasa, IBM Watson, Amazon Alexa, Google Assistant, MS
Cortana, Bixby, FB Messenger, WeChat, Slack etc.
Design Tools
COMP3074-HAI Lecture 14, Basic VUI design principles
▪Write a sample dialog between VUI and a user
▪Avoids writing prompts in isolation, which can lead to stilted
user experience
▪Pick 5 most common use cases for your VUI
▪Start with “blue sky” best path dialogs
▪Also write for when things go wrong, focus on recovery
▪Can be useful to illustrate early concept to stakeholders (client,
developers, etc.)
▪Can be enacted in a “table read” with another person, good way
to improve your wording
▪Good VUIs use clear language; it can take time to get this right!
Sample dialogs
COMP3074-HAI Lecture 14, Basic VUI design principles
▪Similar to wireframes and mockups
▪Especially useful if your VUI experience has a GUI (hybrid)
▪Can be turned into a visual storyboard together with your sample
▪VUI design teams may consist of graphical and language
designers, storyboards to bind this together
▪Again, good way to present and refine early design concept
▪ Iterate with colleagues
▪Present to stakeholders
Visual mockups
COMP3074-HAI Lecture 14, Basic VUI design principles
▪Decision tree style
diagrams showing
possible paths
through your VUI
▪But language
interaction isn’t
hierarchical, so
careful with
nesting levels
Flow diagram
COMP3074-HAI Lecture 14, Basic VUI design principles
Part 3. Confirmations
(2017). Designing Voice User Interfaces
▪A key part of VUI design
▪ Getting the user to confirm an action
▪ Over-confirmation can be a problem
▪ Considerations for choosing a confirmation strategy
▪ What is the consequence for getting it wrong?
(Would the wrong flight end up being booked or just a wrong song played?
▪ What modalities will the system have to provide feedback?
(Will it be audio-only? Will there be non-textual visual feedback such as the light ring
on the Amazon Echo?)
▪ [If GUI] What is the screen size, does it have a small screen, such as a
smartwatch, or a medium-sized screen, such as an iPhone?
▪ What type of confirmation is most appropriate?
(Explicit confirmation? Implicit? A hybrid?)
Confirmations
COMP3074-HAI Lecture 14, Basic VUI design principles
▪Explicit confirmation
▪ “I think you want to set a reminder to ‘buy insurance before going
skydiving next week .’ Is that right?”
▪→ requires the user to take another turn to explicitly confirm
▪ Implicit confirmation
▪ “OK, setting a reminder to buy insurance…”
▪→ does not require the user to say anything else
Types of confirmations
COMP3074-HAI Lecture 14, Basic VUI design principles
▪ Three-tiered confidence
▪ Implicit confirmation
▪Nonspeech confirmation
▪Generic confirmation
▪Visual confirmation
Confirmation methods
COMP3074-HAI Lecture 14, Basic VUI design principles
▪ Three-tiered confidence
▪ Take into account system confidence level that the user request
has been interpreted correctly in selecting the response, e.g.,
Confirmation methods
COMP3074-HAI Lecture 14, Basic VUI design principles
Please buy more paper towels.
[>80 percent confidence, implicit confirmation] OK, ordering more paper towels…
[45–79 percent confidence, explicit confirmation] I thought you said you’d like to order
more paper towels. Is that correct?
[<45 percent confidence] I’m sorry, I didn’t get that. What would you like to buy? ▪ Implicit confirmation ▪ just give the answer, but it may be a good idea to build in some of the request to ensure the user can be confident in the answer ▪Consider the responses to a possible request “What’s the weather tomorrow”. ▪E.g., “the weather tomorrow is….” vs. ▪ “the weather in Nottingham tomorrow is...” Confirmation methods COMP3074-HAI Lecture 14, Basic VUI design principles ▪Nonspeech confirmation ▪E.g., for a request “turn on the lights” the response could simply be to turn them on ▪Generic confirmation ▪Something that works most of the time, other examples are ▪ I’m sorry to hear that, tell me more, that’s interesting etc. Confirmation methods COMP3074-HAI Lecture 14, Basic VUI design principles How are you feeling today, Cathy? Well, pretty good today, I guess. Thank you for sharing that with me. How did you sleep last night? ▪Visual confirmation ▪Good for hybrid experiences ▪Screens are better for communicating several items of information ▪Remember, speech is ephemeral Confirmation methods COMP3074-HAI Lecture 14, Basic VUI design principles Ok Google. When’s my next meeting? You have a calendar entry tomorrow. The title is “Chinatown field trip.” Part 4. Error handling (2017). Designing Voice User Interfaces ▪ “When you talk to a human being, there is never an unrecoverable error state”. – Abi Jones ▪Reprompt is the traditional response if user was not heard or understood,, e.g., “I’m sorry, I didn’t understand” ▪But reprompts not always necessary (they can be quite annoying!) ▪Silence can work in much the same way as a reprompt ▪Silence in human conversation is sign of trouble ▪People respond to silent VUIs as sign of trouble too ▪BUT silence doesn’t tell the user what the problem is… Error handling COMP3074-HAI Lecture 14, Basic VUI design principles ACTIVITY. Think, Pair, Share (2 minutes) How can a VUI go wrong/malfunction? ▪ There are a variety of ways VUIs can make mistakes: ▪ No speech detected ▪ Pre-ASR error → no response. ▪ Speech detected, but nothing recognized ▪ Transcription failure → ‘didn’t understand’ response. ▪ Something was recognized incorrectly ▪ Transcription error → unexpected/unintended response. May be avoidable by confirmation ▪ Something was recognized correctly, but the system does the wrong thing with it ▪ Semantic error / no matching intent → unexpected/unintended response. May be avoidable by disambiguation Kinds of VUI errors COMP3074-HAI Lecture 14, Basic VUI design principles ▪No speech detected ▪Pre-ASR error → no response. Kinds of VUI errors COMP3074-HAI Lecture 14, Basic VUI design principles ▪Way to handle: ▪ Indicate that your system ’is listening’, e.g., Alexa’s light ring ▪Absence of light indicates that system isn’t listening, therefore unable to detect speech ▪Speech detected, but nothing recognized ▪ Transcription failure → ‘didn’t understand’ response. Kinds of VUI errors COMP3074-HAI ▪Ways to handle: ▪Call it out explicitly (e .g ., “I’m sorry, I didn’t get that. What was the name of the restaurant?” or “Sorry, what was that?”). But “didn’t understand” is ambiguous – is it a transcription error or a semantic error? Better to be precise, e.g., ‘couldn’t parse what you said’ ▪Do nothing Poor strategy, as it doesn’t give the user a way to recover ▪ Indicate that your system has processed the query, but isn’t returning a response e.g., Alexa’s Google’s lights ‘spinning’, but then disappearing, maybe with a ’bleep’ error sound ▪Something was recognized incorrectly ▪ Transcription error → unexpected/unintended response. May be avoidable by confirmation ▪Very frequent that ASR returns a partially incorrect transcription Kinds of VUI errors COMP3074-HAI Lecture 14, Basic VUI design principles ▪Ways to handle ▪Present partial response for confirmation to allow user to correct when VUI is less certain ▪E.g., Alexa: “You want me to play a station for BB Intro, right?” ▪Building the (potential) transcription error into the response to the query so that user can move on, progress ▪Something was recognized correctly, but the system does the wrong thing with it ▪Semantic error / no matching intent → unexpected/unintended response. May be avoidable by disambiguation Kinds of VUI errors COMP3074-HAI Lecture 14, Basic VUI design principles MEDICAL VIRTUAL ASSISTANT How are you feeling? I have a cold. MEDICAL VIRTUAL ASSISTANT How long have you been feeling cold? What’s the weather in Springfield? Did you mean the one in Illinois, or Maryland? It’s 65 degrees... ▪Ways to handle ▪Confirmation ▪Homonym resolution for disambiguation (see L3!) ▪Reprompt users progressively ▪ Instead of a generic error message, tell the user what’s expected ▪ This is sometimes called “escalating error strategy” Reprompt strategies for error handling COMP3074-HAI Lecture 14, Basic VUI design principles AIRLINE APP Please tell me your flight number, and I’ll look up the reservation. Uhh...576782. AIRLINE APP I’m sorry, I don’t recognize that. Your flight number is three digits long and follows the letters Ua. Oh, that! It’s 375. AIRLINE APP Thank you. Getting your reservation... ▪Don’t blame the user ▪People don’t like to be blamed; they will like your VUI less ▪Design for both novice and expert users (and other kinds of users) ▪New users need more instructions than experienced users ▪Experienced users may wish to use shortcuts Error handling – general points COMP3074-HAI Lecture 14, Basic VUI design principles ▪ Universals: ‘global’ commands the user can always say ▪ E.g., help, stop ▪ Help has specific significance in VUIs, due to invisibility of the affordances of the app ▪ It’s often not clear which commands are supported ▪ It’s unclear what the app can do ▪ Discoverability – the ability for the user to discover what they can do – is a problem in VUIs ▪ E.g., ”Siri what can you do?” ▪ Contextual help (e.g., specific to a step the user’s in) is responsibility of the app developer “Help” and other Universals COMP3074-HAI Lecture 14, Basic VUI design principles ▪ May be useful to write lists of the (recurring) things a system says (for TTS, voice actors, stakeholder sign off etc.) ▪ Key phrases ▪ As above, but smaller “building blocks”, e.g., yep, yeah, sure, etc. ▪ Accessibility ▪ Cater for different abilities, how do you support visually impaired? ▪ Time-efficiency ▪ VUI interactions can get tedious if they take too long to complete ▪ Keep it short ▪ Don’t overload the system prompts with information, speech is ephemeral Finally, some more things to think about COMP3074-HAI Lecture 14, Basic VUI design principles ▪ Talk faster ▪ Consider different speeds, especially important for accessibility ▪ Interrupt me at any time ▪ Current smartspeakers always listen for the wake word ▪ Provide actionable prompts ▪ Should be clear what users can/should do next ▪ Where am I? ▪ No visual ‘breadcrumb’ in process, provide status/process help ▪ TTS personalization ▪ Allow users to configure aspects such as the VUI’s voice, dialect, or language, as well as taking into account the user’s geographical region Finally, some more things to think about COMP3074-HAI Lecture 14, Basic VUI design principles 程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com