Deep Face Recognition Challenges and Tips for Real-life Deployment - PowerPoint PPT Presentation

Deep Face Recognition Challenges and Tips for Real-life Deployment research@hertasecurity.com

1 Deep Face Recognition 2 Public DBs 3 Public models 4 Managing imbalance 5 Embeddings 6 Conclusions

Deep Face Recognition  GPU-powered face recognition  Offices in Barcelona, Madrid, London, Los Angeles  Crowds, unconstrained Deep Face Recognit itio ion  Large training DBs, >100K images, >1K subjects (Public DBs)  Public models ( Inception , VGG , ResNet , SENet …), close to state -of-the-art  Typically, embedding layer (yielding facial descriptor) feeds one-hot encoding  Unconstrained (in-the-wild) environments HERTA www.hertasecurity.com

Public DBs • Mostly celebrities: subjects overlap 2.6K 10.6K VGG Face CWF 1.8K IJBB VGG Face 2 LFW 9.1K 5.7K HERTA www.hertasecurity.com

Public DBs • Highly imbalanced CW CWF LF LFW Demographic group Images / subject HERTA www.hertasecurity.com

Public models • Public models trained on public DBs (DIY) Fac FaceNet (2015) CWF / MS-1MC VGGFace (2015) VGG SphereFace (2017) Sp CWF VGGFace2 (2017) MS-1MC + VGG2 • Validate with demographically-balanced DB: Asian female: 1M 1M pairs Asian male: 1M 1M pairs Black female: 1M 1M pairs (50% same ID, Black male: 1M 1M pairs 50% different ID) White female:1M 1M pairs White male: 1M pairs 1M HERTA www.hertasecurity.com

Public models: examples of failures False positives False negatives HERTA www.hertasecurity.com

Public models: evaluation White male Black male Asian female Fac FaceNet (2015) VGGFace (2015) 1MC CWF VGG VGGFace2 (2017) SphereFace (2017) Sp 1MC VG2 CWF 1MC VG2 HERTA www.hertasecurity.com

Managing imbalance Undersampling SAMPLING (DATA-ORIENTED) Oversampling Cost-sensitive c learning TRAINING LOSS id (MODEL-ORIENTED) Multi-task gender “Features get better at learning understanding faces, improving ethnics performances of individual tasks” R Ranjan, VM Patel, R Chellappa . “ Hyperface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition .” TPAMI 2017 HERTA www.hertasecurity.com

Managing imbalance – Data augmentation • Data augmentation: makes imbalance mitigation much more effective Stochastic Oversampled DB Database DNN data augmentation I Masi et al. " Do we really need to collect millions of faces for effective face recognition? " ECCV 2016. HERTA www.hertasecurity.com

Managing imbalance – Proposal 𝑛𝑏𝑦 𝑌 Traditional imbalance: 𝑛𝑗𝑜(𝑌) 𝐸 9 𝐸 9 𝑌 Proposal: IDR 𝐸 1 𝐸 1 (𝑌) (robust to outliers) Ite Iterativ ive mult lti-la label l ove vers rsampli ling: 1. Find most imbalanced label L 2. Find most imbalanced category C within L 3. Draw random sample from C, replicate 𝐸 9 𝑌 𝑛𝑏𝑦 𝑌 𝐸 1 (𝑌) 𝑛𝑗𝑜(𝑌) #samples added #samples added HERTA www.hertasecurity.com

Managing imbalance – Sample training batch Before oversampling… …and after HERTA www.hertasecurity.com

Managing imbalance • Results with ResNet 20 (tiny network, for comparison only) • Better with almost 6X le less subjects, 2X le less images! 1.8 .8K subjects, 295K images 10.6K subjects, 494K images HERTA www.hertasecurity.com

Sparse embedding • Typically, in deep face recognition: embedding one-hot image CNN layer encoding • What about ReLU + embedding + one-hot encoding? (e.g. VGGFace) Why more dimensions, if 90% zero? Sparse 4096-d Dense 512-d Dict + Dense 256-d • Larger representation subspace, at expense of computational efficiency • But can gain it back! ̴200M comp/s HERTA www.hertasecurity.com

Conclusions • Public training / validation DBs: heavily bia iased at multiple levels • Without balancing, trained models will be biased, too! • Prefer “ better data ” over “ more data ” • Machine Learning vs Machine Teaching Designing algorithms to Choosing which examples passively train models to show a learner  Explainable ML Zhu, Xiaojin, et al. "An Overview of Machine Teaching." arXiv preprint arXiv:1801.05927 (2018). HERTA www.hertasecurity.com

Questions? research@hertasecurity.com