Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics ECCV 2018 Workshop, 14/09/2018
Introduction Convolutional Neural Networks are the state-of-the-art in image classification Networks can Not clear what Lack of Rely on “good” cheat by learning the network has comprehensively training data spurious learned annotated data variations mohsalvi@robots.ox.ac.uk 2
Contents • Removing a bias from a network • Removing multiple spurious variations from a network • LAOFIW – Labeled Ancestral Faces in the Wild mohsalvi@robots.ox.ac.uk 3
Spurious Variations vs Biases Spurious Variations “Side information” Biases mohsalvi@robots.ox.ac.uk 4
Gender Classification from Celebrity Faces mohsalvi@robots.ox.ac.uk 5
IMDB Faces Dataset [1] • Dataset consists of celebrity faces from I nternational M ovie D ata B ase • Contains Age, Gender, Identity Labels • Created two subsets of the dataset • Age • Gender • Cleaned labels [1] Rothe, Timofte, Van Gool. (2015) mohsalvi@robots.ox.ac.uk 6
Biased Datasets mohsalvi@robots.ox.ac.uk 7
Extremely Biased Datasets Training data Test data ? mohsalvi@robots.ox.ac.uk 8
Only young women, only old men Extremely Biased Datasets mohsalvi@robots.ox.ac.uk 9
Training on Extremely Biased Data Evaluated on a gender/age balanced test dataset Gender classification accuracy: 70% mohsalvi@robots.ox.ac.uk 10
Turning a Blind Eye • Primary task is the attribute of interest • Gender Classification • Secondary task denotes bias to be unlearned • Age Classification • Objective: learn feature representation that is informative for primary task, and uninformative for secondary task • Repurpose work in field of domain adaptation [2] [2] Tzeng, Hoffman, Darrell, and Saenko. (2015) mohsalvi@robots.ox.ac.uk 11
Methods (1/3) Based on VGG-M Network [2] Minimize: [2] Chatfield, Simonyan, Vedaldi, and Zisserman. (2014) mohsalvi@robots.ox.ac.uk 12
Methods (2/3) Minimize: Primary Loss mohsalvi@robots.ox.ac.uk 13
Methods (3/3) Act in opposition to each other Minimize: Secondary Secondary Classification Confusion Secondary Loss Alternate: Confusion Loss Primary Primary Classification Classification Secondary & Classification Secondary Secondary Confusion Confusion Cross-entropy between classifier output and uniform distribution mohsalvi@robots.ox.ac.uk 14
Results – Removing a bias Baseline Gender classification accuracy: 70% Age-Blind Gender classification accuracy: 86% mohsalvi@robots.ox.ac.uk 15
Removing multiple spurious variations (1/2) Problem 1: Multiple biases may be present in dataset Gender Age Pose Expression Ancestry mohsalvi@robots.ox.ac.uk 16
Removing multiple spurious variations (2/2) • Problem 2: no single dataset contains labels for all biases • Each labeled for a single purpose • Leverage information from multiple datasets Dataset 1: Dataset 2: Dataset 3: Dataset 4: Gender Age Ancestry Pose mohsalvi@robots.ox.ac.uk 17
LAOFIW Labeled Ancestral Origin Faces in the Wild 14,000 images in four classes: • Sub-Saharan Africa • Western Europe • East Asia Indian subcontinent • mohsalvi@robots.ox.ac.uk 18
Methods - Removing multiple spurious variations For M secondary tasks: mohsalvi@robots.ox.ac.uk 19
Removing multiple spurious variations experiments Gender* Age Ancestry Pose Experiment (1) Ancestry Age Gender Pose Experiment (2) * Not extremely biased mohsalvi@robots.ox.ac.uk 20
Results - Removing multiple spurious variations (1/2) Primary task: Gender Secondary tasks: Age, Ancestry, Pose mohsalvi@robots.ox.ac.uk 21
Results - Removing multiple spurious variations (2/2) Primary task: Ancestry Secondary tasks: Age, Gender, Pose mohsalvi@robots.ox.ac.uk 22
Conclusions • Can improve generalizability of models train on biased datasets • Can remove multiple spurious variations from feature representation of network • LAOFIW – ancestral origin dataset mohsalvi@robots.ox.ac.uk 23
Questions? mohsalvi@robots.ox.ac.uk 24
Recommend
More recommend
