See-Through Head-Mounted Displays based on 3D Eye Localization Yuta - - PowerPoint PPT Presentation

▶

Mar 30, 2023 472 likes •730 views

1/23 Interaction-Free Calibration for Optical See-Through Head-Mounted Displays based on 3D Eye Localization Yuta Itoh and Gudrun Klinker TU Mnchen FAR: Fachgebiet Augmented Reality Technische Universitt Mnchen 2/23 Information

SLIDE 1

FAR: Fachgebiet Augmented Reality Technische Universität München

Interaction-Free Calibration for Optical See-Through Head-Mounted Displays based on 3D Eye Localization

Yuta Itoh and Gudrun Klinker TU München

1/23

SLIDE 2

FAR: Fachgebiet Augmented Reality Technische Universität München 2/23

This work is

–a part of EDUSAFE project –and funded by

Information

This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under PITN-GA-2012-316919- EDUSAFE.

SLIDE 3

FAR: Fachgebiet Augmented Reality Technische Universität München 3/23

Non Optical See-Through
Optical

ical Se See-Thr Through

ugh (OS

OST) T)

Head Mounted Displays (HMD)

SLIDE 4

FAR: Fachgebiet Augmented Reality Technische Universität München 4/23

Why I CAN’T Recommend OST-HMDs?

Latency
Depth cues
Field of View
Weight
…

and….

SLIDE 5

FAR: Fachgebiet Augmented Reality Technische Universität München 5/23

Spat atial ial registration

–Cal alib ibrati ration

Why I CAN’T Recommend OST-HMDs?

Latency
Depth cues
Field of View
Weight
…

SLIDE 6

FAR: Fachgebiet Augmented Reality Technische Universität München 6/23

HMD moves on one’s head
> frequent recalib

libation ation

Calibration of Eye&OST-HMD

SLIDE 7

FAR: Fachgebiet Augmented Reality Technische Universität München 7/23

HMD moves on one’s head
> frequent recalib

libation ation

Calibration of Eye&OST-HMD

SLIDE 8

FAR: Fachgebiet Augmented Reality Technische Universität München 8/23

Should be Simple & Accurate

Ideal Eye-HMD Calibration

SLIDE 9

FAR: Fachgebiet Augmented Reality Technische Universität München 9/23

Find 3D-2D Projection:

Camera Calibration Analogy

= K*[R t]

SLIDE 10

FAR: Fachgebiet Augmented Reality Technische Universität München 10/23

In the Eye of the Beholder…

Find 3D-2D Projection:

= K*[R t]

SLIDE 11

FAR: Fachgebiet Augmented Reality Technische Universität München 11/23

Manual alignment

– Intensive user interaction – User-dependent noise

Related Work

Predictive tracking for augmented reality RT Azuma - 1995 - ronaldazuma.com

SLIDE 12

FAR: Fachgebiet Augmented Reality Technische Universität München 12/23

SPAAM: Single Point Active Alignment Method

– Medium user interaction – User-dependent noise

Related Work

Single point active alignment method (SPAAM) for optical see-through HMD calibration for AR M Tuceryan, N Navab - Augmented Reality, 2000.(ISAR 2000). …, 2000 - ieeexplore.ieee.org Practical solutions for calibration of optical see-through devices Y Genc, M Tuceryan, N Navab - … of the 1st International Symposium on …, 2002 - dl.acm.org

N times

SLIDE 13

FAR: Fachgebiet Augmented Reality Technische Universität München 13/23

Utiliz

ilizes s 3D Ey D Eye Localiz lization tion

–Int nter erac action ion-fr free, ee, thus do not bother users –Mor

e ac accu curat ate e than a realistic SPAAM setup

Our Method

SLIDE 14

FAR: Fachgebiet Augmented Reality Technische Universität München

Off-axis Pinhole Camera

Rigidly connected

S 14/23

SLIDE 15

FAR: Fachgebiet Augmented Reality Technische Universität München

Projection by Full Display Parameters



15/23

SLIDE 16

FAR: Fachgebiet Augmented Reality Technische Universität München

Projection with Old Projection (Recycle)



16/23

SLIDE 17

FAR: Fachgebiet Augmented Reality Technische Universität München 17/23

1. Estimate a 2D

2D iris ellipse

– Iris detector + Fitting by RANSAC

2. Back project it to 3D circle

Robust real-time pupil tracking in highly off-axis images L Świrski, A Bulling, N Dodgson - … of the Symposium on Eye Tracking …, 2012 - dl.acm.org

3D Eye Position Estimation

Image-based Eye Pose and Reflection Analysis for Advanced Interaction Techniques andScene Understanding C Nitschke, A. Nakazawa, H. Takemura - CVIM,[ …, 2011 - ci.nii.ac.jp

SLIDE 18

FAR: Fachgebiet Augmented Reality Technische Universität München 18/23

Full Setup

–All display param.

Recycle Setup

–some display param. + Old projection  Exp xper erim iment ent

Formulation Summary

SLIDE 19

FAR: Fachgebiet Augmented Reality Technische Universität München 19/23

3D-2D pairs:
Eye positions:

“Ground-Truth” Acquisition

S

N times

SLIDE 20

FAR: Fachgebiet Augmented Reality Technische Universität München 20/23

Estimated

projection:

Reprojection Error:

Error Measurement

p ~,

SLIDE 21

FAR: Fachgebiet Augmented Reality Technische Universität München

Three Conditions

SPAAM Degraded SPAAM

PEst PEst

Proposed

PEst … … …

21/23

SLIDE 22

FAR: Fachgebiet Augmented Reality Technische Universität München

2D Projection Errors

22/23

SLIDE 23

FAR: Fachgebiet Augmented Reality Technische Universität München

3D Eye Position Estimates

≈10 cm

23/23

SPAAM is unsure about the depth th

SLIDE 24

FAR: Fachgebiet Augmented Reality Technische Universität München 24/23

Calibration of OST-HMDs using

3D D eye pos

siti

ition

–No user interaction required

No user-dependent error

–Two formulations: Recycle & Full

Same mathematically, yet not practically

–More accurate than a degraded calibration

w/ Recycle Setup

Summary

SLIDE 25

FAR: Fachgebiet Augmented Reality Technische Universität München

Thank you for your attention & Any questions?

25/23