Using Pen-based (Sonically- enhanced) Gestures to Maximise PDA - - PowerPoint PPT Presentation

Mobile HCI '04 1

School of Informatics

Sonically-Enhanced Gestures

Using Pen-based (Sonically- enhanced) Gestures to Maximise PDA Screen Space

Mark Nicholson & Paul Vickers

Mobile HCI '04 2

Sonically-Enhanced Gestures

School of Informatics

Introduction

• The Palm V screen is small

– 160×160 pixels – 2% of the area of a 1280×1024 desktop screen – 10% or more is used by buttons

• Q: Can we get rid of the

buttons?

Graffiti area Display area Buttons

Mobile HCI '04 3

Sonically-Enhanced Gestures

School of Informatics

Alternative to buttons?

• Gestures

– Stylus strokes interpreted by computer – Palm’s Graffiti is a gesture system, though not a whole- screen system

• Use whole-screen gestures instead of buttons?

– Instead of a "Confirm" button have a "Confirm" gesture

• Cinema listing application built to test the idea

Mobile HCI '04 4

Sonically-Enhanced Gestures

School of Informatics

The gesture set

(a) Confirm (b) Cancel (c) Previous (d) Next (e) Showing (f) Book (g) F/slash (h) B\slash

Mobile HCI '04 5

Sonically-Enhanced Gestures

School of Informatics

Problems?

• No visual cues as to available actions
• Memorability of the gestures
• Recognition algorithms not always

reliable enough for all users

Mobile HCI '04 6

Sonically-Enhanced Gestures

School of Informatics

A solution

• Use audio cues to inform user of available actions
• 3 groups of gestures

– Navigation group: "Next" and "Previous" – Decision group: "Confirm" and "Cancel" – Cinema-specific group: "Showing" and "Book" – (F/slash & B/slash not sonified)

• Earcons for three groups

– Single, double, and triple tones respectively

Mobile HCI '04 7

Sonically-Enhanced Gestures

School of Informatics

Evaluation

• Four modes of operation

With buttons With gestures & With sonically-enhanced gestures With sonically-enhanced gestures and permanent screen prompts

Mobile HCI '04 8

Sonically-Enhanced Gestures

School of Informatics

Tasks

Task Description 1 Identify two pieces of information for a selected cinema. 2 Identify two pieces of information for a selected film. 3 Place and confirm a booking for a selected film. 4 Place and cancel a booking for a selected film 5 Place and confirm a booking for a selected film, but before confirming the booking, make an electronic payment. 6 Place and confirm a booking for a selected film, but before confirming the booking, cancel an attempted electronic payment. 7 Place and cancel a booking for a selected film, but before cancelling the booking, make an electronic payment.

Mobile HCI '04 9

Sonically-Enhanced Gestures

School of Informatics

User responses

• Buttons Mode:

– "Easy to use but had to stop myself trying to use gestures … buttons mode felt cumbersome", "Easy, but tried to use gestures first"

• Earcons difficult remember and to recognize

– More a function of poor earcon design and the Palm’s audio engine

• Gestures modes required fewer stylus presses
• Familiarity led to decreased dependence on

backslash and auditory cues

Mobile HCI '04 10

Sonically-Enhanced Gestures

School of Informatics

Next

• Look at platform with better sound
• Improve earcon design
• Extend earcons to give feedback during use
• Apply to desktop environments?

– e.g. Wacom graphics tablets

Mobile HCI '04 11

Sonically-Enhanced Gestures

School of Informatics

Questions