WRIST: Wearables for Rich, Subtle, Gestural Interactions in - - PowerPoint PPT Presentation

▶
wrist
SMART_READER_LITE
LIVE PREVIEW

WRIST: Wearables for Rich, Subtle, Gestural Interactions in - - PowerPoint PPT Presentation

Apr 23, 2023 355 likes •940 views

WRIST: Wearables for Rich, Subtle, Gestural Interactions in Ubiquitous Environments Edward Lank With: Yang Li, Sylvain Malacria, Mathieu Nancel, Keiko Katsuragawa, James Wallace, Dan Vogel, Jaime Ruiz, Krzysztof Pietroszek, Matt Negulescu,

slide-1
SLIDE 1

WRIST:

Wearables for Rich, Subtle, Gestural Interactions in Ubiquitous Environments

Edward Lank

With: Yang Li, Sylvain Malacria, Mathieu Nancel, Keiko Katsuragawa, James Wallace, Dan Vogel, Jaime Ruiz, Krzysztof Pietroszek, Matt Negulescu, Ankit Kamal, Alec Azad, Shaishav Siddhpuria, Edmund Liu, Jay Henderson

slide-2
SLIDE 2

Edward Lank

Motion Gestures

A gesture performed by physically translating or/and rotating the device.

slide-3
SLIDE 3

Edward Lank

Mapping: Motion Gesture Design Eyes-Free Input: Tap/Swipe/Motion Gestures Scaffolding: Training Mechanisms Multi-Display Interactions

slide-4
SLIDE 4

Edward Lank

Smartphones + Motion Gestures + UBICOMP/IoT

slide-5
SLIDE 5

Edward Lank

Killer App for Wearables?

Josh Constantine, “Apple Watch Review: After 2 Months…”

https://techcrunch.com/2015/07/08/apple-watch-review-after-2-months/

slide-6
SLIDE 6

Edward Lank

Killer App for Wearables?

slide-7
SLIDE 7

Edward Lank

Embedded Interaction is the technological and conceptual phenomena of seamlessly integrating the means for interaction into everyday artifacts.

  • Sensing, actuation, processing, and

networking.

  • Interaction into users’ everyday tasks.
  • Kranz, Holleis, & Schmidt, (2010). Embedded interaction: Interacting

with the internet of things. IEEE Internet Computing, 14(2), 46-53.

slide-8
SLIDE 8

Edward Lank

Why Wearables?

  • Embedded devices (vs interactive devices)
  • Invisibility dilemma
  • Implicit vs Explicit interaction and the

Midas touch phenomenon

  • Sensing and tracking
slide-9
SLIDE 9

Edward Lank

Beyond being there

by Jim Hollan and Scott Stornetta CHI '92

“The mismatch between what we actually have and what we can deliver.”

  • Andy Wilson, Graphics Interface 2017 Keynote
slide-10
SLIDE 10

Edward Lank

Wearables (Smartwatches) for Rich, Subtle, Gestural Interactions

Manipulation and Text Input Ideographic Gestures Finger Gestures AVI 2016, CHI 2018 DIS 2017

slide-11
SLIDE 11

Edward Lank

Wearables for Transient, Subtle, Reliable Interactions

Manipulation and Text Input Ideographic Gestures Finger Gestures AVI 2016, CHI 2018 DIS 2017 TIIS 2018

slide-12
SLIDE 12

Edward Lank

Displays Everywhere

slide-13
SLIDE 13

Edward Lank

WatchPoint

slide-14
SLIDE 14

Edward Lank

Evaluation

slide-15
SLIDE 15

Edward Lank

Evaluation

slide-16
SLIDE 16

Edward Lank

CD-Gain/Cursor Acceleration

  • Trade-off
slide-17
SLIDE 17

Edward Lank

Stability and Small Targets

  • Advantage Watchpoint
slide-18
SLIDE 18

Edward Lank

Stability

slide-19
SLIDE 19

Edward Lank

Beyond Being There

Hollan and Stornetta

slide-20
SLIDE 20

“Pointing at a Distance with Everyday Smart Devices”

9am, Thursday, Public Large and Shared Displays 518C, first talk.

slide-21
SLIDE 21

Edward Lank

Wearables for Transient, Subtle, Reliable Interactions

Manipulation and Text Input Ideographic Gestures Finger Gestures AVI 2016, CHI 2018 DIS 2017 TIIS 2018

slide-22
SLIDE 22

Edward Lank

Gorilla Arm and Gestural Input

Consumed Endurance

Hincapié-Ramos, Guo, Moghadasian, & Irani, Consumed endurance: a metric to quantify arm fatigue of mid-air interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1063-1072). ACM.

slide-23
SLIDE 23

Edward Lank

slide-24
SLIDE 24

Edward Lank

Gorilla Arm and Gestural Input

Consumed Endurance

slide-25
SLIDE 25

Edward Lank

Side-of-Body, Rich Gesture Input

slide-26
SLIDE 26

Edward Lank

Elicitation Study

  • Given participant a task

– How would they perform the task? – Perform the task – Measure characteristics of performance

slide-27
SLIDE 27

Edward Lank

Tasks

slide-28
SLIDE 28

Edward Lank

Agreement

slide-29
SLIDE 29

Edward Lank

Ideographic & Alphabetic

slide-30
SLIDE 30

Edward Lank

Side-of-Body, Rich Gesture Input

slide-31
SLIDE 31

Edward Lank

Wearables for Transient, Subtle, Reliable Interactions

Manipulation and Text Input Ideographic Gestures Finger Gestures AVI 2016, CHI 2018 DIS 2017 TIIS 2018

slide-32
SLIDE 32

Edward Lank

Recognition Challenge

Gestures tend to be simple discrete gestures involving

  • ne

(maybe two) axis with low kinematic impulse

Ruiz, Li, and Lank, CHI 2011

+

slide-33
SLIDE 33

Edward Lank

Solutions for Reliable Input?

  • Better gestures

– But limited in number …

  • Delimiters

– Like a mode switch

  • Recognition strategies

– Tighter thresholding

slide-34
SLIDE 34

Edward Lank

Challenge => Tighter Thresholding

Likelihood of observing movement naturally Movements

slide-35
SLIDE 35

Edward Lank

How can we preserve both high recall and high precision when gestures collide with everyday motion?

slide-36
SLIDE 36

Edward Lank

Conceptualization: Eyes-Free Interaction

Tap or Swipe Motion gestures

“Tap, Swipe, Move” by Negulescu, Ruiz, Li and Lank, AVI 2012

slide-37
SLIDE 37

Edward Lank

Take-Aways

  • Main take-away:

– Even when touch/swipe designed for eyes- free, gestural input still better.

  • Additional observation:

– Effect of recognition errors – In particular, single error appears to have ~0 cost!

  • User just repeats gesture.
slide-38
SLIDE 38

Edward Lank

Cost of Errors

Attempt 1 Attempt 2 Attempt 3 Attempt 4

… Reliability vs Perceived Reliability

slide-39
SLIDE 39

Edward Lank

Bi-Level Thresholding

Observation: On false negative, user repeats gesture Safety Net

  • Observing two possible gestures = observing
  • ne highly probable gesture
  • One tightly thresholded initial model
  • One loosely thresholded double model
slide-40
SLIDE 40

Edward Lank

Bi-level Thresholding Recognition

slide-41
SLIDE 41

Edward Lank

Preliminary Study

  • Built a simple double-threshold recognizer

– 3 gestures – flick left, flick right, double-flip – Constructed HMM-based recognizer

slide-42
SLIDE 42

Edward Lank

Preliminary Study

  • Bi-level threshold recognizer achieved

93% recall with 95.3% precision for 2 attempts

  • 65% of gestures captured with double

threshold

– Vs 35% with single threshold (26% first attempt + 9% second attempt)

slide-43
SLIDE 43

Edward Lank

Evaluating Bi-Level Thresholding

  • Reliability or perceived reliability …

– Is it higher recognition or recognition strategy?

  • Bi-level behavior …

– How does bi-level thresholding perform on larger gesture sets?

slide-44
SLIDE 44

Edward Lank

User Reaction

  • 5-gesture walking

experiment

slide-45
SLIDE 45

Edward Lank

Just Noticeable Difference Experiment

  • Wizard-of-Oz Recognizer

–Controlled recognition rates of 50%, 60%, 70% for single threshold –Ensured identical recognition rates for bi-level thresholding

slide-46
SLIDE 46

Edward Lank

Experimental Design (2)

slide-47
SLIDE 47

Edward Lank

Results: TLX

slide-48
SLIDE 48

Edward Lank

Results: User Preference

slide-49
SLIDE 49

Edward Lank

Same Work, Varied Preference

slide-50
SLIDE 50

Edward Lank

Behavior: Hand Gestures

slide-51
SLIDE 51

Edward Lank

Subtle Hand/Finger Gestures

slide-52
SLIDE 52

Edward Lank

Empress, CHI 2016 WristFlex, UIST 2014 Serendipity, CHI2016 Tomo, UIST2016

Gesture Set

slide-53
SLIDE 53

Edward Lank

Gesture Set

slide-54
SLIDE 54

Edward Lank

Bi-Level Behavior

slide-55
SLIDE 55

Edward Lank

Wearables for Transient, Subtle, Reliable Interactions

Manipulation and Text Input Ideographic Gestures Finger Gestures AVI 2016, CHI 2018 DIS 2017 TIIS 2018

slide-56
SLIDE 56

Edward Lank

Colleagues + Students/Post- Docs

LOKI Waterloo Grad Students and Post-Docs

slide-57
SLIDE 57

Edward Lank

Questions?

Wearables for Rich Subtle Transient Interactions

Yang Li, Sylvain Malacria, Mathieu Nancel, Keiko Katsuragawa, Dan Vogel, Jim Wallace, Krzysztof Pietroszek, Shaishav Siddhpuria, Edmund Liu, Jaime Ruiz, Ankit Kamal, Alec Azad, Matei Negulescu, Jane Henderson.