Natural Interfaces Charlie Albright & Thomas Griffin Gestures - PowerPoint PPT Presentation

Natural Interfaces Charlie Albright & Thomas Griffin

Gestures ● Use of gestures already exists ● Commonly seen in video game technology ○ Kinect ○ Wii Remotes ○ PSEye ● Smartphones ● “The Clapper”

Johnny Lee ● Author of our first paper, “In Search of a Natural Gesture” ● Led development on the Xbox Kinect when it was still an R&D project ● Had a TED talk in 2008 about repurposing Wii Remotes for “natural gesture” sensing

In Search of a Natural Gesture ● What makes an interaction natural? ● Interfaces for computers have not changed much in 40 years ● “The real problem with the interface is that it is an interface. Interfaces get in the way. I don’t want to focus my energies on an interface. I want to focus on the job.” - Don Norman

Complexity ● Everyone wants a “Natural User Interface”, but that has different meanings for different people ● Human interaction is typically multimodal ○ speech + gestures, etc ■ Put-That-There System ● Gestures are hard to generalize across all human population (diversity in cultures)

Problems ● Interacting in the air with a system is awkward ○ Kinect gesture recognition ● Natural gestures vary a lot between people Temporary Fix: define a set of gestures a user must learn ○ ■ is this really natural? ● Interference from outside “noise” No intuitive way to “filter” between intentional input and normal ○ activity

Mobile & Micro Interactions ● People aren’t very good at multi-tasking ● Threshold of 4-5 seconds ● Avoid complex control and gestures ● Instead, focus on simple repeatable tasks

Examples? ● From the paper, tapping on arm or muscle contracting ● Lift up phone to call ● The clapper ● Others?

Electromyography ● Sensing the contractions of muscles ● Discreet ● Tied to the user, not computer ● Natural

The Myo ● An armband that measures muscle contraction as well as motion. ● Via Bluetooth and a defined API

Voice Recognition ● Probably the most common and successful alternate interaction ● Phones, touchtone systems, tvs/consoles, etc. ● Require either lots of training, or a limited vocabulary. ● Generally not “smart”

“Depth” Imaging ● Microsoft conducted a research project on creating a depth camera using normal imaging overlaid with infrared imaging to create depth models

“Depth” Imaging ● This adds a new dimension (literally) to the way we visually sense motion and gestures ● How do you think it would improve gesture recognition?

Other Controls? ● The Emotiv - EEG control ● Uses brain activity as input