Augmented Reality Combining Haptics and Vision Guangqi Ye, Jason J. - - PowerPoint PPT Presentation

▶
augmented reality combining haptics and vision
SMART_READER_LITE
LIVE PREVIEW

Augmented Reality Combining Haptics and Vision Guangqi Ye, Jason J. - - PowerPoint PPT Presentation

Sep 16, 2023 184 likes •437 views

Augmented Reality Combining Haptics and Vision Guangqi Ye, Jason J. Corso, Gregory D. Hager Allison M. Okamura Johns Hopkins University July, 2003 1 Vision + Haptics 2 Outline of the Talk Motivation System Framework

slide-1
SLIDE 1

1

Augmented Reality Combining Haptics and Vision

Guangqi Ye, Jason J. Corso, Gregory D. Hager Allison M. Okamura

Johns Hopkins University July, 2003

slide-2
SLIDE 2

2

Vision + Haptics

slide-3
SLIDE 3

3

Outline of the Talk

  • Motivation
  • System Framework
  • Implementation Details
  • Experimental Results
  • Short Videos and Conclusion
slide-4
SLIDE 4

4

Cons of Constant Contact

  • Prevents feeling of new tactile sensation
  • Limited workspace of haptic device

PHANToM 1.0A: 13 cm * 18 cm * 25 cm

  • Constant reminder of “a virtual world”
slide-5
SLIDE 5

5

Our Solution: Combining Vision

  • Project Goal:

integrating force feedback and tactile feedback

slide-6
SLIDE 6

6

Advantages of Our System

  • Combining force feedback and tactile

feedback

  • User-friendly
  • Immersiveness
slide-7
SLIDE 7

7

Modular Framework

  • Three modules

Vision, Haptic device, augmented reality

  • Vision: Tracking the user’s fingertip
  • Haptic device :

Meeting the user and Interaction rendering

  • Augmented Reality

Scene configuration Vision/Haptics 3-D registration

slide-8
SLIDE 8

8

Vision Subsystem

  • Hand Segmentation
  • Fingertip Detection
  • Fingertip Tracking
slide-9
SLIDE 9

9

Segmentation of Hand

  • Appearance Model: Hue Histogram
  • Classification Criterion: histogram matching
  • Skin color model

Single Gaussian model of hue distribution

  • Accuracy: 98%

∑ ∑

= =

=

n i i n i i i

Model Model Measure Model Measure HI

1 1

) , min( ) , (

slide-10
SLIDE 10

10

Segmentation Example

slide-11
SLIDE 11

11

Fingertip Detection

  • Geometrical Properties of Fingertip

True Fingertip False Fingertip

slide-12
SLIDE 12

12

Fingertip Detection Example

slide-13
SLIDE 13

13

Fingertip Tracking

  • Kalman Filter
  • Search for fingertip in a local area
  • 3-D coordinates calculation
slide-14
SLIDE 14

14

Augmented Reality Module

  • 3-D registraction
  • Calculate R and t

Optimal solution of absolute orientation

  • Precision: 0.5mm
slide-15
SLIDE 15

15

Augmented Reality Module

  • Scene Configuration
  • Virtual plane (n, d)
  • Button (p, n, w, h)
  • Passive objects
slide-16
SLIDE 16

16

Haptic Device Subsytem

  • Control Law
slide-17
SLIDE 17

17

Gravity Compensation

  • Torque of gravity of PHANToM parts
  • Force to compensate gravity
slide-18
SLIDE 18

18

Interaction Simulation

  • Overall force control
  • Object gain matrix
  • Adjust gain along the normal to the surface
slide-19
SLIDE 19

19

Model the Interaction

slide-20
SLIDE 20

20

Experimental Results

  • Implementation on average PC
  • Around 12 fps
  • Multiple objects
slide-21
SLIDE 21

21

Interaction Simulation Results

slide-22
SLIDE 22

22

A Short Movie

slide-23
SLIDE 23

23

Conclusion

  • Combine vision to resolve the problem of

constant contact

  • Modular framework incorporating force

feedback and tactile feedback

  • Key problems: visual tracking, 3-D

registration, interaction rendering

  • Further research: HMD, richer interaction
slide-24
SLIDE 24

24

  • Thanks
  • Comments/Questions ?
  • Acknowledgements:

The authors thank Jake Abbott for his assistance with PHANToM control. The work was supported in part by the Johns Hopkins University and National Science Foundation Grant #EEC- 9731478.