Analysis of Viewer and View Ashish Tawari, Andreas Moegelmose, - - PowerPoint PPT Presentation

Ashish Tawari, Andreas Moegelmose, Sujitha Martin*, Thomas B. Moselund and Mohan M. Trivedi

• Oct. 14th, 2014

Qingdao, China

1

Attention Estimation by Simultaneous Analysis of Viewer and View

Introduction [Video]

• What is the

driver

• bserving?

2

Introduction

• 2012 Pedestrian Traffic Safety Facts:

– 4,743 pedestrians died in traffic crashes … a 6% increase from 2011 – 88% of pedestrian fatalities occur during normal weather conditions (clear/cloudy)

3

• NHTSA

Important Safety Reminders

“Look out for pedestrians everywhere, at all times. Very

• ften pedestrians are not walking where they should be.”
• NHTSA’s Safety Countermeasures Division

4

We propose an Advanced Driver Assistance Systems (ADAS) with simultaneous analysis of the VIEWER and VIEW… … to determine which pedestrians the driver has seen and has not seen.

Related Work

• Wearable cameras have commonly been used to capture first

person perspective.

• Z. Lu and K. Grauman, “Story-Driven Summarization for Egocentric Video,”

CVPR 2013.

• Face-looking cameras have been used for gaze estimation

before.

• A. Tawari, K. H. Chen and M. M. Trivedi, “Where is the Driver Looking:

Analysis of Head, Eye and Iris for Robust Gaze Zone Estimation,” ITSC 2014.

5

Our work is the first time combining the two modalities for driver attention analysis

The Approach: Hardware setup

• The view from first person perspective

– Google Glass – 1280 x 720 pixel resolution

• The viewer from a spatially distributed camera

setup

– GigE cameras – 960 x 1280 pixel resolution

6

The View: Salient Objects

• Pedestrians are detected using HOG/SVM trained
• n the Inria dataset
• Challenges in first person view

7

The View: Salient Objects

• To decrease false positive detection

– Determine the region of interest – Limit the size of the pedestrians

8

Raw pedestrian detection Filtered pedestrian detection

The Viewer

• Gaze-surrogate

– Coarse estimation of the gaze direction – Requires location of iris center and eye corners

9

The Viewer

• Gaze-surrogate

– Coarse estimation of the gaze direction – Requires location of iris center and eye corners

10

Attended Object Determination

• We combine the estimated gaze with salient zones

(pedestrians) from the wearable camera perspective to estimate exactly what the driver is looking at.

11

Experimental Evaluation

• The dataset:

– Multiple drivers – 40 minutes of data in total – Naturalistic on-road driving with a focus on intersections – Ground truth:

• 410 frames of 1413 annotated pedestrians in the first person view
• 300 frames of where the driver is looking in the first person view

12

Experimental Evaluation

Performance of the attention estimator compared with the center-bias as baseline

13

Estimator Mean gaze error (in pixels) Median gaze error (in pixels) Attended pedestrian accuracy (%) Manually annotated pedestrians Full system Center-bias based (baseline) 148.3 127.0 55.9 37.0 Proposed 54.1 32.2 79.4 46.0

Analysis of the View and the Viewer

14

Conclusion

• A new approach to analyze driver’s attention state

has been shown

• We fused the first-person video data with the face

video of the same person to infer attention

• Evaluated our work on naturalistic driving data
• The framework can easily accommodate any object
• f interest or even a low-level saliency model to

estimate the focus of attention.

15

Thank you!

• Please contact us for any questions and we will

respond to you within 24 hours.

Ashish Tawari: ashish.tawari@gmail.com Andreas Moegelmose: andreas@moegelmose.com Sujitha Martin: sujitha.martin@gmail.com Our website: http://cvrr.ucsd.edu

16