Balancing Privacy and Safety: Protecting Driver Identity in - - PowerPoint PPT Presentation

Balancing Privacy and Safety: Protecting Driver Identity in Naturalistic Driving Video Data

Sujitha Martin, Ashish Tawari and Mohan M. Trivedi Laboratory of Intelligent and Safe Automobiles September 19th, 2014

1

2

Question: Can you tell what the driver is doing?

3

Question: Can you identify the driver?

Outline

• Motivation
• Related Works
• De-Identification Filter

– Requirements and Challenges – Design: identity protection and gaze preservation

• Case Study

– Experiment Design – Performance: face recognition and gaze zone estimation

• Concluding Remarks

4

Motivation

• Why “Naturalistic” Driving Data?

5

Motivation

• The 100-Car Naturalistic Driving Study Data (NDS)

has collected data of

– 2 million miles, 42 000 hours – 82 crashes, 761 near crashes, 8295 critical incidents

• The SHRP-2 NDS has been collecting data of

– 3000 subjects, 1 million hours, 5 million trips, 33 million miles, 4 billion GPS points.

6

Motivation

• Public access to raw data of NDS, including data

from 100-Car and SHRP-2, is NOT available because of personal identifiable information

7

• Need ways to protect the

privacy of drivers

• But also preserve sufficient

details to infer driver behavior

Related Works

Privacy protection, by de-identifying people, is typically needed for one of two reasons…

8

Related Works

Privacy protection, by de-identifying people, is typically needed for one of two reasons…

1) Person is not intended to be in the image, or

9 Flores, A., and Belongie, S., “Removing pedestrians from google street view images,” In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, IEEE (2010).

Related Works

Privacy protection, by de-identifying people, is typically needed for one of two reasons…

1) Person is not intended to be in the image, or 2) Presence and action of person is intended but not their identity

10 Agrawal, P., and Narayanan, P., “Person de-identification in videos,” Circuits and Systems for Video Technology, IEEE Transactions on 21, 3 (2011), 299–31.

Related Works

Research Study Approach Preserve Context Sample Evaluation Cheng & Trivedi, 2005 Voxel reconstruction of the scene Action Looking inside the vehicle N/A Schiff, Meingast & Mulligan, 2009 Solid ellipsoidal overlays

• n faces

Scene and action Surveillance Hand labeled: false positives and false negatives Nodari et al., 2012 Replacing pedestrians with similar pedestrians from controlled dataset Scene and action Google street view Algorithmic detection, segmentation, matching and replacing results Our work Isolated segmentation of eyes with and without face mask Gaze Looking inside the vehicle User study of face recognition and gaze estimation 11

De-Identification Filter

13

De-Identification Filter

14

• How to protect identity and preserve gaze?

– Isolated segmentation of facial regions

De-Identification Filter

15

• How to protect identity and preserve gaze?

– Isolated segmentation of facial regions – Background distortion

Sujitha Martin, Ashish Tawari and Mohan M. Trivedi, “Towards Privacy Protecting Safety Systems for Naturalistic Driving Videos,” IEEE Transactions on Intelligent Transportation Systems, 2014.

Experimentation Evaluation

• Case Study of Face Recognition

– 10 participants – ~80 de-identified images – Given a de-identified image, participants choose one of the 12 candidates that best matches.

16

Experimental Evaluation

• Case Study of Gaze Zone Estimation

– Gaze zones of interest: Left, front, right, rear-view, inside

17

Experimental Evaluation

• Case Study of Gaze Zone Estimation

– Gaze zones of interest: Left, front, right, rear-view, inside – 10 participants – 150 de-identified images (2 drivers, 5 images per gaze zone, 3 types of de-identification)

18

Experimental Evaluation

19

De-Identification Method Recognition Rate

(chance =8.3%)

Gaze-zone Estimation Accuracy One-Eye 5% 65% Two-Eyes 8% 71% Mask with Two-Eyes 8%* 85% (a) One- Eye (b) Two- Eyes (c) Mask with Two- Eyes

Demo [Video]

20

Concluding Remarks

• Public access to raw data of NDS is NOT available because of

personal identifiable information.

• Need privacy protection while preserving sufficient details to

infer driver behavior.

• Case study on three de-identification scheme: One-eye, Two-

eyes and Mask with two-eyes.

• Achieved 85% accuracy in gaze-zone estimation with face

recognition below chance.

21

Thank You! Any Questions?

22