Natural Interfaces Charlie Albright & Thomas Griffin Gestures - - PowerPoint PPT Presentation

Natural Interfaces

Charlie Albright & Thomas Griffin

Gestures

• Use of gestures already exists
• Commonly seen in video game technology

○ Kinect ○ Wii Remotes ○ PSEye

• Smartphones
• “The Clapper”

Johnny Lee

• Author of our first paper, “In Search of a

Natural Gesture”

• Led development on the Xbox Kinect when it

was still an R&D project

• Had a TED talk in 2008 about repurposing

Wii Remotes for “natural gesture” sensing

In Search of a Natural Gesture

• What makes an interaction natural?
• Interfaces for computers have not changed

much in 40 years

• “The real problem with the interface is that it

is an interface. Interfaces get in the way. I don’t want to focus my energies on an

• interface. I want to focus on the job.” - Don

Norman

Complexity

• Everyone wants a “Natural User Interface”, but that has different meanings

for different people

• Human interaction is typically multimodal

○ speech + gestures, etc ■ Put-That-There System

• Gestures are hard to generalize across all

human population (diversity in cultures)

Problems

• Interacting in the air with a system is awkward

○ Kinect gesture recognition

• Natural gestures vary a lot between people

○ Temporary Fix: define a set of gestures a user must learn ■ is this really natural?

• Interference from outside “noise”

○ No intuitive way to “filter” between intentional input and normal activity

Mobile & Micro Interactions

• People aren’t very good at multi-tasking
• Threshold of 4-5 seconds
• Avoid complex control and gestures
• Instead, focus on simple repeatable tasks

Examples?

• From the paper, tapping on arm or muscle

contracting

• Lift up phone to call
• The clapper
• Others?

Electromyography

• Sensing the contractions of muscles
• Discreet
• Tied to the user, not computer
• Natural

The Myo

• An armband that measures muscle

contraction as well as motion.

• Via Bluetooth and a defined API

Voice Recognition

• Probably the most common and successful

alternate interaction

• Phones, touchtone systems, tvs/consoles,

etc.

• Require either lots of training, or a limited

vocabulary.

• Generally not “smart”

“Depth” Imaging

• Microsoft conducted a research project on

creating a depth camera using normal imaging overlaid with infrared imaging to create depth models

“Depth” Imaging

• This adds a new dimension (literally) to the

way we visually sense motion and gestures

• How do you think it would improve gesture

recognition?

Other Controls?

• The Emotiv - EEG control
• Uses brain activity as input