PRE-TOUCH SENSING for MOBILE INTERACTION Ken Hinckley Seongkook - - PowerPoint PPT Presentation

▶
pre touch sensing
SMART_READER_LITE
LIVE PREVIEW

PRE-TOUCH SENSING for MOBILE INTERACTION Ken Hinckley Seongkook - - PowerPoint PPT Presentation

Apr 15, 2023 161 likes •747 views

PRE-TOUCH SENSING for MOBILE INTERACTION Ken Hinckley Seongkook Heo (KAIST & MSR) Michel Pahud Christian Holz Hrvoje Benko Abigail Sellen Gavin Smyth Richard Banks Kenton OHara Bill Buxton PRE-TOUCH SENSING for MOBILE

slide-1
SLIDE 1

Ken Hinckley Seongkook Heo (KAIST & MSR) Michel Pahud Christian Holz Hrvoje Benko Abigail Sellen Gavin Smyth Richard Banks Kenton O’Hara Bill Buxton

for

MOBILE INTERACTION

PRE-TOUCH SENSING

slide-2
SLIDE 2

Michel Pahud Christian Holz Hrvoje Benko Abigail Sellen Gavin Smyth Richard Banks Kenton O’Hara Bill Buxton Ken Hinckley

Seongkook Heo (KAIST & MSR)

for

MOBILE INTERACTION

PRE-TOUCH SENSING

slide-3
SLIDE 3
slide-4
SLIDE 4

TOUCH IS

ANALOG

and

CONTINUOUS…

IF NATURAL HUMAN

slide-5
SLIDE 5

(x,y)

Then why do we boil the down to flatland? entire vocabulary of mobile touch

slide-6
SLIDE 6

Much of what characterizes ‘touch’ starts

before contact

and originates

beyond the confines

  • f the screen.
slide-7
SLIDE 7

AN INSTANT IN TIME… FROM TOUCH AS A POINT,

slide-8
SLIDE 8

TO A WHOLE WORLD THAT SURROUNDS IT…

slide-9
SLIDE 9
slide-10
SLIDE 10

This treasury of contextual detail— the combination of hover and grip that we collectively refer to as

pre-touch

—has been lost to mobile interaction.

slide-11
SLIDE 11
slide-12
SLIDE 12

SENSOR /technology

– Self-capacitance is well-known. Not our contrib.

fogale-sensation.com

slide-13
SLIDE 13

You’ve probably heard about hover,

  • r seen examples of control gestures in-air.
slide-14
SLIDE 14

You’ve probably heard about hover,

  • r seen examples of control gestures in-air.

The focus of pre-touch isn’t on such

  • vert, ‘foreground’ gestures.
slide-15
SLIDE 15

You’ve probably heard about hover,

  • r seen examples of control gestures in-air.

The focus of pre-touch isn’t on such

  • vert, ‘foreground’ gestures.

Instead, pre-touch is primarily about the ‘background’ of the interaction—

slide-16
SLIDE 16

You’ve probably heard about hover,

  • r seen examples of control gestures in-air.

The focus of pre-touch isn’t on such

  • vert, ‘foreground’ gestures.

Instead, pre-touch is primarily about the ‘background’ of the interaction—

what happens before, the framing, the context.

slide-17
SLIDE 17

This is a design space paper.

slide-18
SLIDE 18

This is a design space paper. We explore an emerging modality, breadth-first across a number of possibilities.

slide-19
SLIDE 19

This is a design space paper. We explore an emerging modality, breadth-first across a number of possibilities. But we do not go depth-first into any one technique in exhaustive detail.

slide-20
SLIDE 20

WE EXPLORED: 1) ANTICIPATORY REACTIONS 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-21
SLIDE 21

WE EXPLORED: 1) ANTICIPATORY REACTIONS 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-22
SLIDE 22

WE EXPLORED: 1) ANTICIPATORY REACTIONS 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-23
SLIDE 23

WE EXPLORED: 1) ANTICIPATORY REACTIONS 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch, simultaneous with hover
slide-24
SLIDE 24

WE EXPLORED: 1) ANTICIPATORY REACTIONS 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-25
SLIDE 25

WE EXPLORED: 1) ANTICIPATORY REACTIONS

– when fingers away, no UI. Only content. – so we fade in (0.2s) … later fade out (1.2s) – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-26
SLIDE 26

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-27
SLIDE 27

WE EXPLORED:

– when fingers away, no UI. Only content. – so we fade in (0.2s) … later fade out (1.2s) – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-28
SLIDE 28

WE EXPLORED:

– when fingers away, no UI. Only content. – so we fade in (0.2s) … later fade out (1.2s) – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-29
SLIDE 29

WE EXPLORED:

– when fingers away, no UI. Only content. – so we fade in (0.2s) … later fade out (1.2s) – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-30
SLIDE 30

WE EXPLORED:

– when fingers away, no UI. Only content. – so we fade in (0.2s) … – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-31
SLIDE 31

WE EXPLORED:

– when fingers away, no UI. Only content. – so we fade in (0.2s) … later fade out (1.2s) – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-32
SLIDE 32

WE EXPLORED:

– when fingers away, no UI. Only content. – so we fade in (0.2s) … later fade out (1.2s) – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-33
SLIDE 33

WE EXPLORED:

– when fingers away, no UI. Only content. – so we fade in (0.2s) … later fade out (1.2s) – since we’re fading in anyway, take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-34
SLIDE 34

WE EXPLORED:

– when fingers away, no UI. Only content. – so we fade in (0.2s) … later fade out (1.2s) – since we’re fading in anyway, take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, ad-lib interface : key ideas

– approach suggests intent to engage

but…

slide-35
SLIDE 35

WE EXPLORED:

grip + hover

TO ILLUSTRATE THIS DESIGN SPACE,

multiple contexts

slide-36
SLIDE 36

(“dialing”) even adaptation of control gestures

slide-37
SLIDE 37

WE EXPLORED: 1) ANTICIPATORY REACTIONS calm revelation 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects

ad-lib interface

slide-38
SLIDE 38

WE EXPLORED: 1) ANTICIPATORY REACTIONS calm revelation 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects

ad-lib interface

slide-39
SLIDE 39

WE EXPLORED: 1) ANTICIPATORY REACTIONS calm revelation 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects

ad-lib interface

slide-40
SLIDE 40

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-41
SLIDE 41

WE EXPLORED: 1) ANTICIPATORY REACTIONS 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-42
SLIDE 42

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-43
SLIDE 43

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-44
SLIDE 44

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects

0ms (touch down) temporal trajectory for a small target temporal trajectory for a large target

  • 200ms

30mm single-finger distance above touchscreen 20mm 10mm

  • 400ms
  • 600ms
  • 800ms
  • 1s
slide-45
SLIDE 45

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-46
SLIDE 46

WE EXPLORED: 1) ANTICIPATORY REACTIONS 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch simultaneous with hover
slide-47
SLIDE 47

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-48
SLIDE 48

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-49
SLIDE 49

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects
slide-50
SLIDE 50

INFORMAL EVALUATION

slide-51
SLIDE 51

and they’re predictive.” “feels very natural to my hand,” allows – adapt to hand size? Ad-lib interface (video player) “I like the transparent controls, “a single hand in a comfortable position.” – controls should appear more quickly?

slide-52
SLIDE 52

– per user adaptations may also be helpful Overall – The one-handed adaptations stood out – But all techniques had proponents – Ballistic vs. fine taps: great for some users, – current heuristic doesn’t consider grip. – not so much for others.

slide-53
SLIDE 53

CONCLUSION

slide-54
SLIDE 54

CONCLUSION

slide-55
SLIDE 55

Pre-touch holds great promise for fresh approaches to many challenges in mobile touch interaction.

Our work has only just begun.

slide-56
SLIDE 56

Seongkook

TO ILLUSTRATE THIS DESIGN SPACE,

Heo

slide-57
SLIDE 57

WE EXPLORED: 2) RETROACTIVE INTERPRETATIONS 3) HYBRID TOUCH-HOVER GESTURES TO ILLUSTRATE THIS DESIGN SPACE,

interface contingent on grip and hover approach trajectory informs touch semantics

  • n-screen touch + above-screen aspects

Questions?