Touch Technologies Touching the World by Sara Kilcher Distributed - - PowerPoint PPT Presentation

▶
touch technologies
SMART_READER_LITE
LIVE PREVIEW

Touch Technologies Touching the World by Sara Kilcher Distributed - - PowerPoint PPT Presentation

Aug 25, 2023 283 likes •982 views

Touch Technologies Touching the World by Sara Kilcher Distributed Systems Seminar 30. April 2013 sakilche@student.ethz.ch Motivation Overview Skin as touchscreen Skinput Overview Skin as touchscreen Skinput OmniTouch

slide-1
SLIDE 1

Touch Technologies

Touching the World

by Sara Kilcher

Distributed Systems Seminar

  • 30. April 2013

sakilche@student.ethz.ch

slide-2
SLIDE 2

Motivation

slide-3
SLIDE 3

Overview

  • Skin as touchscreen

– Skinput

slide-4
SLIDE 4

Overview

  • Skin as touchscreen

– Skinput – OmniTouch

slide-5
SLIDE 5

Overview

  • Skin as touchscreen

– Skinput – OmniTouch

  • Touchscreen for virtual 3D

– Toucheo

slide-6
SLIDE 6

Skin as Touchscreen

Introduction

  • Mobile phones like computers
  • Small screen/keyboard size

User becomes bottleneck

slide-7
SLIDE 7

Skinput OmniTouch Toucheo

Harrison et al. 2010

Skinput

slide-8
SLIDE 8

Skinput OmniTouch Toucheo

Purpose

  • Extend interface
  • Using skin
slide-9
SLIDE 9

Skinput OmniTouch Toucheo

Video

Video: 01_skinput.avi

slide-10
SLIDE 10

Skinput OmniTouch Toucheo

Difficulty

  • Touch

where? →

slide-11
SLIDE 11

Skinput OmniTouch Toucheo

How it works

The Waves

slide-12
SLIDE 12

Skinput OmniTouch Toucheo

How it works

Processing the Waves

slide-13
SLIDE 13

Skinput OmniTouch Toucheo

How it works

Training SVM

Video: 02_skinput.avi

slide-14
SLIDE 14

Skinput OmniTouch Toucheo

How it works

Recognition

Video: 03_skinput.avi

slide-15
SLIDE 15

Skinput OmniTouch Toucheo

How well it works

  • User study with 13 participants
  • Touch event errors “negligible”
  • Position not very accurate

– Overall 87.6% (5 to 10 locations) – Up to 95.5% (5 locations)

slide-16
SLIDE 16

Skinput OmniTouch Toucheo

How well it works

Video: 04_skinput.avi

slide-17
SLIDE 17

Skinput OmniTouch Toucheo

Personal Opinion

  • Very innovative
  • Requires lots of calibration

– Did not work well in demos

  • Lots of “magic” that isn't entirely

understood

– Hard to tune gesture recognition? → – Completely individual

slide-18
SLIDE 18

Skinput OmniTouch Toucheo

Vision

  • Armband will get smaller
  • Use cases without projector

– Car – Jogging – Audio Feedback?

slide-19
SLIDE 19

Skinput OmniTouch Toucheo

Harrison et al. 2011

OmniTouch

slide-20
SLIDE 20

Skinput OmniTouch Toucheo

Purpose

  • Like Skinput, but

– Needs a projector – Needs a depth-camera – Can be used on additional flat surfaces

slide-21
SLIDE 21

Skinput OmniTouch Toucheo

Video

slide-22
SLIDE 22

Skinput OmniTouch Toucheo

Video

Video: 05_omnitouch.avi

slide-23
SLIDE 23

Skinput OmniTouch Toucheo

Difficulties

Projection

  • Surface is uneven

– Distortion – Focus

  • Surface is moving

– Tracking

slide-24
SLIDE 24

Skinput OmniTouch Toucheo

Difficulties

Input Recognition

  • Touch event

recognition

– Touch when? → – Touch where? →

slide-25
SLIDE 25

Skinput OmniTouch Toucheo

How it works

The Hardware

slide-26
SLIDE 26

Skinput OmniTouch Toucheo

How it works

Recognition of Fingers

Step 1: Depth map of scene

slide-27
SLIDE 27

Skinput OmniTouch Toucheo

How it works

Recognition of Fingers

Step 2: Derivatives

slide-28
SLIDE 28

Skinput OmniTouch Toucheo

How it works

Recognition of Fingers

Step 3: “Template matching”

slide-29
SLIDE 29

Skinput OmniTouch Toucheo

How it works

Recognition of Fingers

Step 4: Group slices & find fingertips

slide-30
SLIDE 30

Skinput OmniTouch Toucheo

How it works

Recognition of Touch Events

Flood fill the finger

slide-31
SLIDE 31

Skinput OmniTouch Toucheo

How it works

Recognition of Surfaces

slide-32
SLIDE 32

Skinput OmniTouch Toucheo

How it works

Displaying Interfaces

  • Projective texturing
slide-33
SLIDE 33

Skinput OmniTouch Toucheo

How well it works

Click Accuracy

  • User study with 12 participants
  • Click recognition: 96.5%
slide-34
SLIDE 34

Skinput OmniTouch Toucheo

How well it works

Location Accuracy

Touchscreen: 15mm Omnitouch, hand: 25mm

slide-35
SLIDE 35

Skinput OmniTouch Toucheo

Personal Opinion

  • Fascinating paper
  • Popular on web
  • No real applications yet
  • Use self-made algorithms

– possible to improve

slide-36
SLIDE 36

Skinput OmniTouch Toucheo

Vision

  • Entire world as touchscreen
  • Device will get smaller and more

comfortable

slide-37
SLIDE 37

Skinput OmniTouch Toucheo

Skinput OmniTouch

  • Touch accuracy
  • Use cases without

projector

  • Location accuracy
  • Use more surfaces

– More applications

slide-38
SLIDE 38

Skinput OmniTouch Toucheo

Skinput OmniTouch

  • “Impact” press needed

– No long press – No dragging

  • Setup & calibration
  • Stable arm position
  • Predefined points
  • Finger detection
  • Where to place

hardware?

  • Bright light
slide-39
SLIDE 39

Skinput OmniTouch Toucheo

Touchscreen World ⇔

Touchscreen

slide-40
SLIDE 40

Skinput OmniTouch Toucheo

Hachet et al. 2011

Toucheo

slide-41
SLIDE 41

Skinput OmniTouch Toucheo

Purpose

  • Merge trends

– Multitouch touchscreen – Stereoscopic screen

Easy interaction with virtual 3D objects!

  • Solve difficult tasks

– e.g. 3D docking task

slide-42
SLIDE 42

Skinput OmniTouch Toucheo

Video

slide-43
SLIDE 43

Skinput OmniTouch Toucheo

Video

Video: 06_toucheo.avi Video: 07_toucheo.avi

slide-44
SLIDE 44

Skinput OmniTouch Toucheo

Difficulties

3D and the Hands

  • Occlusions
  • Depth collision
slide-45
SLIDE 45

Skinput OmniTouch Toucheo

Difficulties

Eye Related

Contradiction

slide-46
SLIDE 46

Skinput OmniTouch Toucheo

Difficulties

Eye Related

Contradiction – 3D useful above/below plane

slide-47
SLIDE 47

Skinput OmniTouch Toucheo

Difficulties

Eye Related

Contradiction – 3D useful above/below plane – Touchscreen only in plane

slide-48
SLIDE 48

Skinput OmniTouch Toucheo

Difficulties

Usable Interface

  • Typical smartphone has 3+1 DOF,

Toucheo has 9+1

  • Intuitive interface?
slide-49
SLIDE 49

Skinput OmniTouch Toucheo

Difficulties

The 9+1 DOF

X Y Z

Translation: 3 DOF

slide-50
SLIDE 50

Skinput OmniTouch Toucheo

Difficulties

The 9+1 DOF

X Y Z

Rotation: 3 DOF

slide-51
SLIDE 51

Skinput OmniTouch Toucheo

Difficulties

The 9+1 DOF

X Y Z

Scaling: 3 DOF

slide-52
SLIDE 52

Skinput OmniTouch Toucheo

Difficulties

The 9+1 DOF

X Y Z

Scaling: 3+1 DOF

slide-53
SLIDE 53

Skinput OmniTouch Toucheo

Difficulties

Smartphone: 3+1 DOF

X Y Z

slide-54
SLIDE 54

Skinput OmniTouch Toucheo

How it works

The Setup

Screen Mirror Touchscreen Mirrored Screen

slide-55
SLIDE 55

Skinput OmniTouch Toucheo

How it works

The Setup

Screen Mirror Touchscreen Mirrored Screen

slide-56
SLIDE 56

Skinput OmniTouch Toucheo

How it works

The Setup

3D Screen Mirror Touchscreen Mirrored 3D Screen

slide-57
SLIDE 57

Skinput OmniTouch Toucheo

How it works

The Setup

slide-58
SLIDE 58

Skinput OmniTouch Toucheo

How it works

The Setup

  • Occlusion problems

eliminated

  • Depth collision

problems reduced

slide-59
SLIDE 59

Skinput OmniTouch Toucheo

How it works

3D to 2D - „Virtual Ray”

slide-60
SLIDE 60

Skinput OmniTouch Toucheo

How it works

The 9+1 DOF

X Y Z

Scaling: 3+1 DOF

slide-61
SLIDE 61

Skinput OmniTouch Toucheo

How it works

The 9+1 DOF

Video: 08_toucheo.avi

slide-62
SLIDE 62

Skinput OmniTouch Toucheo

How well it works

  • User study, 16 participants

– Bulky, quite big – 3D-docking task solved well – Feedback used to improve interface

slide-63
SLIDE 63

Skinput OmniTouch Toucheo

Personal Opinion

  • Clever setup to solve common problems
  • Interface: independent achievement
slide-64
SLIDE 64

Skinput OmniTouch Toucheo

Vision

  • Interface used in
  • ther applications
  • New generation

less bulky

slide-65
SLIDE 65

Toucheo OmniTouch / Skinput

Touchscreen Touch of non-planar objects Virtual 3D objects Real 3D objects Input from traditional touchscreen Input from camera / vibration sensors How to use touch events How to get touch events Novel GUI Standard GUI Display with stereoscopic screen Display with pico- projector Static / big Mobile / small

slide-66
SLIDE 66

Displays of the Future

  • Assumption

– Increase in screen size = increase in device size

  • Alternative

– Displays non-planar surfaces →

  • Market direction?
slide-67
SLIDE 67

Displays of the Future

Video: 09_future.avi Video: 10_future.avi Video: 11_future.avi Video: 12_future.avi Video: 13_future.avi Video: 14_future.avi

slide-68
SLIDE 68

Displays of the Future

?