Speech-Based Interaction Using Speech as a Natural Data Type - - PowerPoint PPT Presentation
Speech-Based Interaction Using Speech as a Natural Data Type Speech as Input Chief decision: Recognition versus Raw Data Recognition Translate into other information (words) Must deal with errors Useful for
2
3
4
ambient noise audio feedback
Noise reduction Enhance audio quality Echo cancellation
5
6
7
8
9
10
11
Big idea: move away from voice as a replacement for menus (easy to implement but painful to use), toward more conversational interfaces
“Designing SpeechActs: Issues in Speech User Interfaces,” Yankelovich, Levow, Marx, CHI’95
Mail:
SpeechActs: You have 14 new messages and 47 old
Eric Baatz regarding "report completed."
User: Let me hear it.
SpeechActs: "The first draft is ready for your comments.
Eric."
User: Reply and include the current message.
SpeechActs: Preparing message to Eric Baatz. Begin
recording after the tone. When finished,
pause for several seconds.
User: Eric, I'll get those to you this afternoon.
SpeechActs: Say cancel, send, or review.
User: Send.
SpeechActs: Message sent. What now?
User: Next message.
SpeechActs: New message two from Stuart Adams,
subject "Can we meet at 3:00 today?"
User: Switch to calendar...
Other commands:
What do I have tomorrow?
What about Bob?
What did he have last Wednesday?
And next Thursday?
What was Paul doing three days after Labor Day?
What's the weather in Seattle?
How about Texas?
I'd like the extended forecast for Boston.
12
13
Avoid prompting wherever possible
Build context around subdialogs
Output prosodics: system asks “huh?”
Pacing: people often have to speak more slowly when talking to machines; need a way to “barge in” to machine output
Vocabulary: need wide, domain-dependent vocabulary
Information organization: how to present content like email messages, flags, message numbers, etc., with consistency and w/o overwhelming the user
Information flow: speech “dialog boxes” (force users into a small set of choices) don’t fit well into conversational style (Users ignore or may produce unexpected answers: “Do you have the time?” not always answered by yes/no)
14
Recognition errors
Rejection errors (utterance not recognized) are frustrating. Can yield “brick wall” of “I don’t understand” messages. Solution: provide progressive assistance
Substitution errors are damaging. Don’t want to verify every utterance. Approach: commands that present data are verified implicitly; commands that destroy data or are undoable are verified explicitly
Insertion errors (background audio picked up as commands or data). Solution: key to turn off recognizer
The Nature of Speech
Lack of visual feedback. Users feel less in control; users can be faced with silence if they don’t do anything; long pauses in conversations are uncomfortable so users may feel a need to respond quickly; less information transmitted to hte user at one time
Speed and persistence: although speech is easy for humans to produce it is hard to
15
16
17
18
19
20
”What were we talking about…?” ”What was that phone number I heard?”
compressed or discarded in some cases
21
22
How do I do this math homework?
I want to talk to Jamie. Jamie, have you finished your homework? He is alone in his room.
23
What is this little two above the number? … Power of 2. When you finish, come set the dinner table. Bye.
24
25