Using Pen-based (Sonically- enhanced) Gestures to Maximise PDA - PowerPoint PPT Presentation

School of Informatics Sonically-Enhanced Gestures Using Pen-based (Sonically- enhanced) Gestures to Maximise PDA Screen Space Mark Nicholson & Paul Vickers Mobile HCI '04 1

School of Informatics Sonically-Enhanced Gestures Introduction • The Palm V screen is small – 160 × 160 pixels – 2% of the area of a 1280 × 1024 desktop screen Display area – 10% or more is used by buttons • Q: Can we get rid of the Buttons buttons? Graffiti area Mobile HCI '04 2

School of Informatics Sonically-Enhanced Gestures Alternative to buttons? • Gestures – Stylus strokes interpreted by computer – Palm’s Graffiti is a gesture system, though not a whole- screen system • Use whole-screen gestures instead of buttons? – Instead of a "Confirm" button have a "Confirm" gesture • Cinema listing application built to test the idea Mobile HCI '04 3

School of Informatics Sonically-Enhanced Gestures The gesture set (a) Confirm (b) Cancel (c) Previous (d) Next (e) Showing (f) Book (g) F/slash (h) B\slash Mobile HCI '04 4

School of Informatics Sonically-Enhanced Gestures Problems? • No visual cues as to available actions • Memorability of the gestures • Recognition algorithms not always reliable enough for all users Mobile HCI '04 5

School of Informatics Sonically-Enhanced Gestures A solution • Use audio cues to inform user of available actions • 3 groups of gestures – Navigation group: "Next" and "Previous" – Decision group: "Confirm" and "Cancel" – Cinema-specific group: "Showing" and "Book" – (F/slash & B/slash not sonified) • Earcons for three groups – Single, double, and triple tones respectively Mobile HCI '04 6

School of Informatics Sonically-Enhanced Gestures Evaluation • Four modes of operation With gestures & With sonically-enhanced With sonically-enhanced gestures and permanent With buttons gestures screen prompts Mobile HCI '04 7

School of Informatics Sonically-Enhanced Gestures Tasks Task Description 1 Identify two pieces of information for a selected cinema. 2 Identify two pieces of information for a selected film. 3 Place and confirm a booking for a selected film. 4 Place and cancel a booking for a selected film Place and confirm a booking for a selected film, but before confirming 5 the booking, make an electronic payment. Place and confirm a booking for a selected film, but before confirming 6 the booking, cancel an attempted electronic payment. Place and cancel a booking for a selected film, but before cancelling the 7 booking, make an electronic payment. Mobile HCI '04 8

School of Informatics Sonically-Enhanced Gestures User responses • Buttons Mode: – "Easy to use but had to stop myself trying to use gestures … buttons mode felt cumbersome", "Easy, but tried to use gestures first" • Earcons difficult remember and to recognize – More a function of poor earcon design and the Palm’s audio engine • Gestures modes required fewer stylus presses • Familiarity led to decreased dependence on backslash and auditory cues Mobile HCI '04 9

School of Informatics Sonically-Enhanced Gestures Next • Look at platform with better sound • Improve earcon design • Extend earcons to give feedback during use • Apply to desktop environments? – e.g. Wacom graphics tablets Mobile HCI '04 10

School of Informatics Sonically-Enhanced Gestures Questions Mobile HCI '04 11