UMB-DB A Database of Partially Occluded 3D Faces Alessandro Colombo - - PowerPoint PPT Presentation

UMB-DB A Database of Partially Occluded 3D Faces

Alessandro Colombo Claudio Cusano Raimondo Schettini

Universit` a degli Studi di Milano-Bicocca

13 November 2011

UMB-DB

University of Milano-Bicocca DataBase

◮ Built to test algorithms and in challenging scenarios ◮ In particular, when faces are partially occluded

Why occlusions?

◮ Less constrained scenarios ◮ Facilitate cooperative users ◮ Improve reliability with respect to ‘holes’, self occlusions, noisy data

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 2 / 17

Databases of 3D faces

Database Subjects Acquisitions Expressions Poses Occlusions FRGCv.2 466 4007 6 1 BU3DFE 100 2500 6 1 ND2006 888 13450 5 1 York 350 5250 5 3? CASIA 123 1845 5 1 FRAV3D 106 1696 2 8 BJUT-3D 500 500 1 1 GavabDB 61 720 4 4 3DRMA 120 720 1 4 Texas 3D 118 1149 ? 1 Bosphorus 105 4666 34 13 381 UMB-DB 143 1473 4 1 578

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 3 / 17

Acquisitions

◮ 143 subjects

◮ 98 males, 45 females ◮ Most from 19 to 50 years old

◮ Minolta Vivid 900 laser scanner

◮ Slow mode, 25mm lens ◮ 640 × 480 samples ◮ Eyes closed for safety reasons ◮ 1.5–3m of distance from the device

◮ No further processing such as noise reduction or

holes filling

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 4 / 17

Acquisitions

◮ At least 9 acquisitions per subject

◮ three with neutral expression ◮ with smiling, angry, bored expressions ◮ occluded by scarf, hat, and hands

◮ 1473 acquisitions ◮ 578 occluded faces

◮ On average, occlusions cover 42% of the face ◮ With a maximum of 84% Claudio Cusano (cusano@disco.unimib.it) UMB-DB 5 / 17

Some examples

Annotations

Each acquisition includes Color image 3D model Up to 7 Mask of visible landmarks face and a set of labels indicating the facial expression, occluding object. . .

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 7 / 17

A system for the recognition of partially occluded 3D faces

◮ Non-occluded faces are

recognized as they are

◮ Small occlusions are restored

and the face is then recognized

◮ Faces with large occlusions

are rejected

3D Face Face detection Normalization Occlusion detection Rejection Restoration Recognition

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 8 / 17

Occlusion tolerant 3D face detection1

Selection of candidate features by curvature analysis

Input face Mean curvature Gaussian curv. HK classification Thresholded HK Candidate eyes Candidate noses

• 1A. Colombo, C. Cusano, R. Schettini, “Gappy PCA classification for
• cclusion tolerant 3D face detection,” J. of Math. Imaging and Vision,

35(3):193–207, 2009.

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 9 / 17

Occlusion tolerant 3D face detection

Pairs or triplets of feature points form candidate faces

◮ Filtered by orientation and size ◮ Registered on the mean face by a variation of ICP ◮ Points too from the mean face are discarded ◮ Classified by a ‘gappy’ PCA classifier

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 10 / 17

Detection results

56 false positives on the whole DB Acquisition type Number of faces Detected Faces % Neutral 441 437 99.1 Non-neutral 442 431 97.5 Occluded 578 553 95.7 Scarf 151 141 93.4 Glasses 75 71 94.7 Hair 33 30 90.9 Hand 165 150 90.9 Hat 183 179 97.8 Misc 28 26 92.85 Whole DB 1473 1421 96.5

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 11 / 17

Restoration of occlusions, and face recognition2

◮ Occlusions are detected on the basis of the

reconstruction error in an eigenspace

◮ Occluded regions are restored by a ‘Gappy’

PCA

◮ Restored faces are recognized

Occluded face Detected occlusions Restored face

• 2A. Colombo,C. Cusano, R. Schettini, “Three-dimensional occlusion

detection and restoration of partially occluded faces,” J. of Math. Imaging and Vision, 40(1):105–119, 2011.

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 12 / 17

Restoration examples

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 13 / 17

Recognition results

Normalization Test set EER (%) IR (%) Manual All cases 18.6 71.0 Automatic All cases 19.5 69.6 Automatic Neutral 1.9 98.0 Automatic Expressions 18.4 66.7 Automatic Occlusions 23.8 56.5

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 14 / 17

Results

DB Acquisitions Subjects EER (%) IR (%) UMB-DB 578 143 23.8 56.5 UND + occl. 477 158 10.9 96.1 Bosphorus 360 105 11.5 87.7

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 15 / 17

Results

0.01 0.1 1 0.01 0.1 1 FRR FAR scarf hand hat glasses misc

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 16 / 17

Conclusions

UMB-DB

◮ Large number of occlusions ◮ Great variability in terms of position, extent, occluding object ◮ Very challenging!

The database is publicly available http://www.ivl.disco.unimib.it/umbdb

Claudio Cusano (cusano@disco.unimib.it) UMB-DB 17 / 17