MMI 2: Mobile Human- Computer Interaction Small and Large Display - - PowerPoint PPT Presentation

▶
mmi 2 mobile human computer interaction
SMART_READER_LITE
LIVE PREVIEW

MMI 2: Mobile Human- Computer Interaction Small and Large Display - - PowerPoint PPT Presentation

Aug 16, 2023 522 likes •1.24k views

MMI 2: Mobile Human- Computer Interaction Small and Large Display Interaction Prof. Dr. Michael Rohs michael.rohs@ifi.lmu.de Mobile Interaction Lab, LMU Mnchen Lectures # Date Topic 1 19.10.2011 Introduction to Mobile Interaction,

slide-1
SLIDE 1

MMI 2: Mobile Human- Computer Interaction Small and Large Display Interaction

  • Prof. Dr. Michael Rohs

michael.rohs@ifi.lmu.de Mobile Interaction Lab, LMU München

slide-2
SLIDE 2

MMI 2: Mobile Interaction 2 WS 2011/12 Michael Rohs, LMU

Lectures

# Date Topic 1 19.10.2011 Introduction to Mobile Interaction, Mobile Device Platforms 2 26.10.2011 History of Mobile Interaction, Mobile Device Platforms 3 2.11.2011 Mobile Input and Output Technologies 4 9.11.2011 Mobile Input and Output Technologies, Mobile Device Platforms 5 16.11.2011 Mobile Communication 6 23.11.2011 Location and Context 7 30.11.2011 Mobile Interaction Design Process 8 7.12.2011 Mobile Prototyping 9 14.12.2011 Evaluation of Mobile Applications 10 21.12.2011 Visualization and Interaction Techniques for Small Displays 11 11.1.2012 Mobile Devices and Interactive Surfaces 12 18.1.2012 Camera-Based Mobile Interaction 13 25.1.2012 Sensor-Based Mobile Interaction 1 14 1.2.2012 Sensor-Based Mobile Interaction 2 15 8.2.2012 Exam

slide-3
SLIDE 3

MMI 2: Mobile Interaction 3 WS 2011/12 Michael Rohs, LMU

Aktuelles

  • Klausur am 8.2.2012

– Anmeldung

  • Fragen zur Klausur

– jeweils zu Beginn der nächsten Vorlesungen (ab 18.1.)

slide-4
SLIDE 4

MMI 2: Mobile Interaction 4 WS 2011/12 Michael Rohs, LMU

Review

  • How to visualize the relevant of a POI to a query?
  • Why is screen navigation important for small displays?
  • Mechanisms to indicate objects beyond the screen?
  • How to improve touch screen accuracy?
slide-5
SLIDE 5

MMI 2: Mobile Interaction 5 WS 2011/12 Michael Rohs, LMU

Preview

  • Touch Screen interaction techniques
  • Behind-the-device interaction
  • Mobile devices and interactive surfaces
slide-6
SLIDE 6

MMI 2: Mobile Interaction 6 WS 2011/12 Michael Rohs, LMU

TOUCH SCREEN INTERACTION TECHNIQUES

slide-7
SLIDE 7

MMI 2: Mobile Interaction 7 WS 2011/12 Michael Rohs, LMU

Precision Touch Input: ThumbSpace

Karlson, Bederson. ThumbSpace: Generalized One-Handed Input for Touchscreen-Based Mobile Devices. Interact 2007.

ThumbSpace: User-defined space which thumb can reach

– One-handed thumb operation of handheld touch interfaces – Not all of screen reachable à reduce thumb interaction space – Selection on lift-off (as with Offset Cursor)

slide-8
SLIDE 8

MMI 2: Mobile Interaction 8 WS 2011/12 Michael Rohs, LMU

Precision Touch Input: ThumbSpace

Karlson, Bederson. ThumbSpace: Generalized One-Handed Input for Touchscreen-Based Mobile Devices. Interact 2007.

slide-9
SLIDE 9

MMI 2: Mobile Interaction 9 WS 2011/12 Michael Rohs, LMU

Precision Touch Input: TapTap and MagStick

Roudaut, Huot, Lecolinet. TapTap and MagStick: Improving one- handed target acquisition on small touch-screens. AVI 2008.

TapTap: Tapping the screen twice

– tap 1: select area of interest – area zooms in, centered on screen – tap 2: select magnified target – zoomed target typically close to screen: fast selection – works in border areas (c.f. Shift) How to distinguish single touch?

slide-10
SLIDE 10

MMI 2: Mobile Interaction 10 WS 2011/12 Michael Rohs, LMU

Precision Touch Input: TapTap and MagStick

Roudaut, Huot, Lecolinet. TapTap and MagStick: Improving one- handed target acquisition on small touch-screens. AVI 2008.

MagStick: “magnetized telescopic stick”

– Initial touch position is reference point – Moving away from target extends stick in opposite direction – End of stick is “magnetically” attracted by target Is moving away from the target intuitive? Is MagStick better than simple Offset Cursor?

slide-11
SLIDE 11

MMI 2: Mobile Interaction 11 WS 2011/12 Michael Rohs, LMU

Precision Touch Input: Comparison Experiment

Roudaut, Huot, Lecolinet. TapTap and MagStick: Improving one-handed target acquisition on small touch-screens. AVI 2008.

  • Dependent variables

– Time – Error rate – Questionnaire results – Ranking of techniques

slide-12
SLIDE 12

MMI 2: Mobile Interaction 12 WS 2011/12 Michael Rohs, LMU

Precision Touch Input: Comparison Experiment

Roudaut, et al. TapTap and MagStick: Improving one-handed target acquisition on small touch-screens. AVI 2008.

slide-13
SLIDE 13

MMI 2: Mobile Interaction 13 WS 2011/12 Michael Rohs, LMU

Precision Touch Input: Comparison Experiment

  • Ranking (first to last): TapTap, MagStick, Shift, Offset

Cursor, Thumbspace, Direct Touch

Roudaut, et al. TapTap and MagStick: Improving one-handed target acquisition on small touch-screens. AVI 2008.

slide-14
SLIDE 14

MMI 2: Mobile Interaction 14 WS 2011/12 Michael Rohs, LMU

MicroRolls: Expanding Touch-Screen Input by Distinguishing Rolls vs. Slides of the Thumb

  • Input vocabulary for

touchscreens is limited

  • MicroRolls: thumb rolls

without sliding

– Roll vs. slide distinction possible – No interference

  • Enhanced input vocabulary

– Drags, Swipes, Rubbings and MicroRolls

Roudaut, Lecolinet, Guiard. MicroRolls: Expanding Touch-Screen Input Vocabulary by Distinguishing Rolls vs. Slides of the Thumb. CHI 2009.

slide-15
SLIDE 15

MMI 2: Mobile Interaction 15 WS 2011/12 Michael Rohs, LMU

MicroRolls: Expanding Touch-Screen Input by Distinguishing Rolls vs. Slides of the Thumb

Roudaut, Lecolinet, Guiard. MicroRolls: Expanding Touch-Screen Input Vocabulary by Distinguishing Rolls vs. Slides of the Thumb. CHI 2009.

slide-16
SLIDE 16

MMI 2: Mobile Interaction 16 WS 2011/12 Michael Rohs, LMU

Kinematic Traces of Different Touch Gestures

Roudaut, Lecolinet, Guiard. MicroRolls: Expanding Touch-Screen Input Vocabulary by Distinguishing Rolls vs. Slides of the Thumb. CHI 2009.

slide-17
SLIDE 17

MMI 2: Mobile Interaction 17 WS 2011/12 Michael Rohs, LMU

Mapping MicroRoll Gestures to Actions

  • Menu supports gesture learning

– Menu only appears after 300ms timeout – Experts execute gestures immediately

  • Precision: selecting small targets
  • Quasi-mode: modify subsequent operation

Roudaut, Lecolinet, Guiard. MicroRolls: Expanding Touch-Screen Input Vocabulary by Distinguishing Rolls vs. Slides of the Thumb. CHI 2009.

MicroRoll gestures Menu (300ms timeout)

slide-18
SLIDE 18

MMI 2: Mobile Interaction 18 WS 2011/12 Michael Rohs, LMU

Bezel Swipe: Conflict-Free Scrolling and Selection on Mobile Touch Screen Devices

  • Drag from screen edges

through thin bars

  • Edge bar encodes command
  • Multiple commands

without interference

– Selection, cut, copy, paste – Zooming, panning, tapping

Roth, Turner. Bezel Swipe: Conflict- Free Scrolling and Multiple Selection on Mobile Touch Screen

  • Devices. CHI 2009.

touch on bar: no activation touch on bezel: activation

slide-19
SLIDE 19

MMI 2: Mobile Interaction 19 WS 2011/12 Michael Rohs, LMU

Bezel Swipe: Conflict-Free Scrolling and Selection on Mobile Touch Screen Devices

Roth, Turner. Bezel Swipe: Conflict-Free Scrolling and Multiple Selection on Mobile Touch Screen Devices. CHI 2009.

slide-20
SLIDE 20

MMI 2: Mobile Interaction 20 WS 2011/12 Michael Rohs, LMU

Rubbing and Tapping: Multiple Fingers

  • n Single-Touch Screens
  • Zooming on single-touch displays

– Cursor “jumps” when second finger touches screen – Hardware averages touch point in center

  • Proposed interaction techniques

– Rub-Pointing: diagonal rubbing gesture for pointing and zooming in a single-handed technique – Zoom-Tapping: dominant hand points, non-dominant hand taps

Olwal, Feiner, Heyman. Rubbing and Tapping for Precise and Rapid Selection on Touch-Screen Displays. CHI 2008.

slide-21
SLIDE 21

MMI 2: Mobile Interaction 21 WS 2011/12 Michael Rohs, LMU

Rubbing and Tapping: Multiple Fingers

  • n Single-Touch Screens

Olwal, Feiner, Heyman. Rubbing and Tapping for Precise and Rapid Selection on Touch-Screen Displays. CHI 2008.

slide-22
SLIDE 22

MMI 2: Mobile Interaction 22 WS 2011/12 Michael Rohs, LMU

  • Concept of a future

rollable display

– Physical resizing of the display as an interaction technique – Semantic zooming

  • Metaphors

– Content locked in viewport – Content locked in hand

Khalilbeigi, Lissermann, Mühlhäuser, Steimle. Xpaaand: Interaction Techniques for Rollable Displays. CHI 2011.

Xpaaand: Interaction Techniques for Rollable Displays

slide-23
SLIDE 23

MMI 2: Mobile Interaction 23 WS 2011/12 Michael Rohs, LMU

Khalilbeigi, Lissermann, Mühlhäuser, Steimle. Xpaaand: Interaction Techniques for Rollable Displays. CHI 2011.

Xpaaand: Interaction Techniques for Rollable Displays

slide-24
SLIDE 24

MMI 2: Mobile Interaction 24 WS 2011/12 Michael Rohs, LMU

Effects of Structural Holds on Pointing and Dragging with Flexible Displays

  • How do users point and drag with

a paper-like display?

– Study: common holds and force patterns – Observed holds: grip zone, rigid / flexible zone

  • Efficiency: Rigid areas (produced by holds)

had 12% higher pointing and dragging performance

Dijkstra, ¡Perez, ¡Vertegaal. ¡Evalua5ng ¡Effects ¡of ¡Structural ¡Holds ¡on ¡ Poin5ng ¡and ¡Dragging ¡Performance ¡with ¡Flexible ¡Displays. ¡CHI ¡2011. ¡

slide-25
SLIDE 25

MMI 2: Mobile Interaction 25 WS 2011/12 Michael Rohs, LMU

Effects of Structural Holds on Pointing and Dragging with Flexible Displays

Dijkstra, ¡Perez, ¡Vertegaal. ¡Evalua5ng ¡Effects ¡of ¡Structural ¡Holds ¡on ¡ Poin5ng ¡and ¡Dragging ¡Performance ¡with ¡Flexible ¡Displays. ¡CHI ¡2011. ¡

slide-26
SLIDE 26

MMI 2: Mobile Interaction 26 WS 2011/12 Michael Rohs, LMU

Lahey, ¡Girouard, ¡Burleson, ¡Vertegaal. ¡PaperPhone: ¡Understanding ¡the ¡Use ¡of ¡Bend ¡ Gestures ¡in ¡Mobile ¡Devices ¡with ¡Flexible ¡Electronic ¡Paper ¡Display. ¡CHI ¡2011. ¡

PaperPhone: Bend Gestures in Mobile Devices with Flexible E-Paper Display

Use device as watch… …detach, use as PDA

slide-27
SLIDE 27

MMI 2: Mobile Interaction 27 WS 2011/12 Michael Rohs, LMU

Lahey, ¡Girouard, ¡Burleson, ¡Vertegaal. ¡PaperPhone: ¡Understanding ¡the ¡Use ¡of ¡Bend ¡ Gestures ¡in ¡Mobile ¡Devices ¡with ¡Flexible ¡Electronic ¡Paper ¡Display. ¡CHI ¡2011. ¡

PaperPhone: Bend Gestures in Mobile Devices with Flexible E-Paper Display

slide-28
SLIDE 28

MMI 2: Mobile Interaction 28 WS 2011/12 Michael Rohs, LMU

BEHIND-THE-DEVICE INTERACTION

slide-29
SLIDE 29

MMI 2: Mobile Interaction 29 WS 2011/12 Michael Rohs, LMU

LucidTouch

  • Behind-the-device multitouch input
  • Pseudo transparency

– Enabling back of the device pointing – 3 states + visual feedback = land-on selection

  • Form-factor

– Enabling multi-touch with all ten fingers

Wigdor, Forlines, Baudisch, Barnwell, Shen: LucidTouch: A See-Through Mobile Device. UIST'07.

slide-30
SLIDE 30

MMI 2: Mobile Interaction 30 WS 2011/12 Michael Rohs, LMU

Why Behind-the-Device Interaction?

  • Avoid occlusion
  • “Fat finger” problem

Wigdor, Forlines, Baudisch, Barnwell, Shen: LucidTouch: A See-Through Mobile Device. UIST'07.

slide-31
SLIDE 31

MMI 2: Mobile Interaction 31 WS 2011/12 Michael Rohs, LMU

Pseudo Transparency

Show finger “shadows” as cues

gray: “shadow” red: touch blue: hovering

Wigdor, Forlines, Baudisch, Barnwell, Shen: LucidTouch: A See-Through Mobile Device. UIST'07.

slide-32
SLIDE 32

MMI 2: Mobile Interaction 32 WS 2011/12 Michael Rohs, LMU

LucidTouch Camera See-Through

  • Finger shapes and positions tracked by camera

– Hovering / tracking state

  • (Multi-)touch detected by pad

– Dragging state physical see-through camera see-through

Wigdor, Forlines, Baudisch, Barnwell, Shen: LucidTouch: A See-Through Mobile Device. UIST'07.

slide-33
SLIDE 33

MMI 2: Mobile Interaction 33 WS 2011/12 Michael Rohs, LMU

Out of Range Tracking Dragging

State State 1 State 2

Lift Mouse Replace Mouse Depress Button Release Button

Buxton’s Three-State Model of Input

Buxton: A Three-State Model of Graphical Input. In Proc. of INTERACT ’90.

slide-34
SLIDE 34

MMI 2: Mobile Interaction 34 WS 2011/12 Michael Rohs, LMU

Passive Tracking Dragging

State 2 State

Contact Release Contact

Two-State Model for Touch Input

Buxton: A Three-State Model of Graphical Input. In Proc. of INTERACT ’90.

slide-35
SLIDE 35

MMI 2: Mobile Interaction 35 WS 2011/12 Michael Rohs, LMU

Out of Range Hovering / Tracking Dragging

State State 1 State 2

Contact Release Contact Finger Behind Back Remove Finger

LucidTouch: Three Input States

Buxton: A Three-State Model of Graphical Input. In Proc. of INTERACT ’90.

slide-36
SLIDE 36

MMI 2: Mobile Interaction 36 WS 2011/12 Michael Rohs, LMU

LucidTouch Applications: Map

Wigdor, Forlines, Baudisch, Barnwell, Shen: LucidTouch: A See-Through Mobile Device. UIST'07.

slide-37
SLIDE 37

MMI 2: Mobile Interaction 37 WS 2011/12 Michael Rohs, LMU

LucidTouch Applications: Text Input

Wigdor, Forlines, Baudisch, Barnwell, Shen: LucidTouch: A See-Through Mobile Device. UIST'07.

slide-38
SLIDE 38

MMI 2: Mobile Interaction 38 WS 2011/12 Michael Rohs, LMU

LucidTouch Issue: Finger Reachability

Wigdor, Forlines, Baudisch, Barnwell, Shen: LucidTouch: A See-Through Mobile Device. UIST'07.

slide-39
SLIDE 39

MMI 2: Mobile Interaction 39 WS 2011/12 Michael Rohs, LMU

Back-of-Device Interaction Works for Very Small Screens

  • Jewelry, watches, etc.
  • Pseudo transparency

– Capacitive touch pad – Clickable touch pad

Baudisch, Chi. Back-of-Device Interaction Allows Creating Very Small Touch Devices. CHI 2009. ¡

slide-40
SLIDE 40

MMI 2: Mobile Interaction 40 WS 2011/12 Michael Rohs, LMU

Back-of-Device Interaction Works for Very Small Screens

Baudisch, Chi. Back-of-Device Interaction Allows Creating Very Small Touch Devices. CHI 2009. ¡

slide-41
SLIDE 41

MMI 2: Mobile Interaction 41 WS 2011/12 Michael Rohs, LMU

Side-of-Device Interaction: SideSight

  • Useful if device is placed on table
  • Distance sensors along device edges

– Multipoint interactions

  • IR proximity sensors

– Edge: 10x1 pixel “depth” image

Left and right “depth” images

Butler, Izadi, Hodges. SideSight: Multi-“touch” Interaction Around Small Devices. UIST’08.

slide-42
SLIDE 42

MMI 2: Mobile Interaction 42 WS 2011/12 Michael Rohs, LMU

Side-of-Device Interaction: SideSight

Butler, Izadi, Hodges. SideSight: Multi-“touch” Interaction Around Small Devices. UIST’08.

slide-43
SLIDE 43

MMI 2: Mobile Interaction 43 WS 2011/12 Michael Rohs, LMU

Pressure-Sensitive Map Zooming

Essl, Rohs, Kratz: Squeezing the Sandwich: A Mobile Pressure- Sensitive Two-Sided Multi-Touch Prototype. Demo at UIST’09.

slide-44
SLIDE 44

MMI 2: Mobile Interaction 44 WS 2011/12 Michael Rohs, LMU

Pressure Sensitive Input: Multiple States

Out of range Tracking Dragging

State State 1 State 2

Release contact Contact Increase pressure Decrease pressure

State 2 State 2 State 2+

Increase pressure Increase pressure Decrease pressure

Buxton: A Three-State Model of Graphical Input. In Proc. of INTERACT ’90.

slide-45
SLIDE 45

MMI 2: Mobile Interaction 45 WS 2011/12 Michael Rohs, LMU

Extension of Virtual Trackball to Back of Device

x y z

  • Full sphere operated from both sides

instead of hemisphere operated from front

slide-46
SLIDE 46

MMI 2: Mobile Interaction 46 WS 2011/12 Michael Rohs, LMU

Virtual Trackballs for 3D Object Rotation

x,y,z-axis rotation z-axis rotation x,y,z-axis rotation θ θ z-axis rotation

slide-47
SLIDE 47

MMI 2: Mobile Interaction 47 WS 2011/12 Michael Rohs, LMU

slide-48
SLIDE 48

MMI 2: Mobile Interaction 48 WS 2011/12 Michael Rohs, LMU

MOBILE DEVICES AND INTERACTIVE SURFACES

slide-49
SLIDE 49

MMI 2: Mobile Interaction 49 WS 2011/12 Michael Rohs, LMU

Motivation for Combining Mobile and Large Displays

  • Support kinesthetic and spatial memory of users

– Locate information in space – Assign application semantics to spatial arrangements

  • Focus & context displays
  • Private & public displays
  • Drag’n’drop in the physical world

– Movement of data between devices – Carry out one task across multiple devices

  • Collaboration
  • Capturing information on public space
slide-50
SLIDE 50

MMI 2: Mobile Interaction 50 WS 2011/12 Michael Rohs, LMU

Fitzmaurice 1993: Spatially Aware Palmtop Computers

  • Fitzmaurice, G. W.: Situated information spaces and

spatially aware palmtop computers. Commun. ACM 36, 7 (Jul. 1993), pp. 39-49.

slide-51
SLIDE 51

MMI 2: Mobile Interaction 51 WS 2011/12 Michael Rohs, LMU

“Ubiquitous Graphics”

  • Focus & context displays

– Wall display for low resolution

  • verview

– Handheld display for high resolution details

  • Ultrasonic tracking

– Mimio XI ultrasonic pens attached to display – Pen emits ultrasonic signal when touching wall

  • Users can add annotations

and objects

Sanneblad, Holmquist. Ubiquitous graphics: Combining hand-held and wall-size displays to interact with large images. AVI '06. http://www.youtube.com/watch?v=uw0a7Zd1JVM

slide-52
SLIDE 52

MMI 2: Mobile Interaction 52 WS 2011/12 Michael Rohs, LMU

“Pick-and-Drop” and “Hyper Palette”

  • Pick-and-Drop

– Direct manipulation for smart environments – Extended “drag-and-drop” concept – Create text on PDA, pick-and-drop to whiteboard

  • Hyper Palette

– PDA as interaction device for table – Electromagnetic 6D trackers – Scoop-and-spread: tilting plus movement

  • Rekimoto. Pick-and-drop: a direct manipulation technique for

multiple computer environments. UIST '97. Ayatsuka, Matsushita, Rekimoto. HyperPalette: A hybrid computing environment for small computing devices. CHI '00.

slide-53
SLIDE 53

MMI 2: Mobile Interaction 53 WS 2011/12 Michael Rohs, LMU

“Pick-and-Drop”

  • Rekimoto. Pick-and-drop: a direct manipulation technique for multiple computer environments. UIST '97.
slide-54
SLIDE 54

MMI 2: Mobile Interaction 54 WS 2011/12 Michael Rohs, LMU

Augmented Surfaces

Rekimoto, Saitoh: Augmented surfaces: A spatially continuous work space for hybrid computing environments. CHI '99.

  • Interchanging information

between mobile devices, interactive surfaces, and physical objects

– Camera-based object recognition – Projected displays as extensions

  • f device screens
  • Hyperdragging

– Move information across boundary of devices and surfaces

slide-55
SLIDE 55

MMI 2: Mobile Interaction 55 WS 2011/12 Michael Rohs, LMU

Augmented Surfaces

Rekimoto, Saitoh: Augmented surfaces: A spatially continuous work space for hybrid computing environments. CHI '99.

slide-56
SLIDE 56

MMI 2: Mobile Interaction 56 WS 2011/12 Michael Rohs, LMU

Selection of menu item (code coordinate system)

Interaction with Large Public Displays

  • Train stations, air ports, museums, shopping malls

Content associated with menu item is transferred to the mobile phone

slide-57
SLIDE 57

MMI 2: Mobile Interaction 57 WS 2011/12 Michael Rohs, LMU

C-Blink: Visual Communication

  • Camera on top of display
  • Cell phone screen blinks

in different colors

  • Hue-difference signal

– Hue in HSV color space

Miyaoku, Higashino, Tonomura: C-Blink: A hue-difference-based light signal marker for large screen interaction via any mobile terminal. UIST 2004.

slide-58
SLIDE 58

MMI 2: Mobile Interaction 58 WS 2011/12 Michael Rohs, LMU

Touch Projector: Mobile Interaction-Through-Video

  • Touch Projector: Interact with remote screens through a

live video image on the mobile device

– Position tracking w.r.t. surrounding displays – Project image onto target display

  • Select targets, drag targets between displays

Boring, Baur, Butz, Gustafson, Baudisch: Touch Projector: Mobile Interaction-Through-Video. Proc. CHI 2010. http://www.youtube.com/watch?v=lTMAKHzbl1E

slide-59
SLIDE 59

MMI 2: Mobile Interaction 59 WS 2011/12 Michael Rohs, LMU

Touch Projector

Boring, Baur, Butz, Gustafson, Baudisch: Touch Projector: Mobile Interaction-Through-Video. Proc. CHI 2010.

slide-60
SLIDE 60

MMI 2: Mobile Interaction 60 WS 2011/12 Michael Rohs, LMU

Deep Shot: Migrating Tasks Across Devices Using Mobile Phone Cameras

  • User tasks often span

multiple devices

  • Deep Shot supports

migrating tasks across devices

– Take a picture – Recognize content – Recreate content on mobile

Chang, Li. Deep Shot: A Framework for Migrating Tasks Across Devices Using Mobile Phone Cameras. CHI 2011.

slide-61
SLIDE 61

MMI 2: Mobile Interaction 61 WS 2011/12 Michael Rohs, LMU

Deep Shot: Migrating Tasks Across Devices Using Mobile Phone Cameras

Chang, ¡Li. ¡Deep ¡Shot: ¡A ¡Framework ¡for ¡Migra5ng ¡Tasks ¡Across ¡Devices ¡Using ¡Mobile ¡Phone ¡Cameras. ¡CHI ¡2011. ¡

slide-62
SLIDE 62

MMI 2: Mobile Interaction 62 WS 2011/12 Michael Rohs, LMU

BlueTable: Connecting Wireless Mobile Devices on Interactive Surfaces

  • Association of a mobile device

with an interactive surface

  • Camera detects objects as

connected components (blobs) of a certain size and shape

  • BlueTable checks whether

detected blob is a mobile device

– Sends Bluetooth request to blink IRDA port to each device in turn – Downside: slow

Wilson and Sarin: BlueTable: Connecting Wireless Mobile Devices on Interactive Surfaces Using Vision-Based Handshaking. Graphics Interface 2007. http://research.microsoft.com/~awilson

slide-63
SLIDE 63

MMI 2: Mobile Interaction 63 WS 2011/12 Michael Rohs, LMU

Mobile Devices and Interactive Tabletops

  • Camera-projector system

– Works with regular tables – Pubs, cafés, meeting rooms

  • Map spatial configurations to

application-specific semantics

– Proximity regions around devices

  • Dynamic marker

Kray, Rohs, Hook, Kratz: Bridging the Gap between the Kodak and the Flickr Generations: A Novel Interaction Technique for Collocated Photo Sharing. IJHCS 2009.

slide-64
SLIDE 64

MMI 2: Mobile Interaction 64 WS 2011/12 Michael Rohs, LMU

  • Find corner stone candidates

– Convolve image with 7x7 “Gaussian” kernel – Separable in two 1D kernels

  • Thresholding (cross-correlation ≥0.72)
  • Non-maximum suppression
  • Find pairs of corner stone candidates
  • Compute homography and sample data area
  • Determine orientation
  • Decode value, compute position, rotation, size

Marker Recognition Algorithm

slide-65
SLIDE 65

MMI 2: Mobile Interaction 65 WS 2011/12 Michael Rohs, LMU

Camera Image (1024x768 pixels)

slide-66
SLIDE 66

MMI 2: Mobile Interaction 66 WS 2011/12 Michael Rohs, LMU

Convolution

slide-67
SLIDE 67

MMI 2: Mobile Interaction 67 WS 2011/12 Michael Rohs, LMU

Thresholding and Non-Maxima Suppression

slide-68
SLIDE 68

MMI 2: Mobile Interaction 68 WS 2011/12 Michael Rohs, LMU

Low Contrast Display Images

slide-69
SLIDE 69

MMI 2: Mobile Interaction 69 WS 2011/12 Michael Rohs, LMU

Low Contrast Display Images

slide-70
SLIDE 70

MMI 2: Mobile Interaction 70 WS 2011/12 Michael Rohs, LMU

The End