Turning a Blind Eye: Explicit Removal of Biases and Variation from - - PowerPoint PPT Presentation

▶
turning a blind eye
SMART_READER_LITE
LIVE PREVIEW

Turning a Blind Eye: Explicit Removal of Biases and Variation from - - PowerPoint PPT Presentation

Jun 07, 2023 35 likes •278 views

Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings Mohsan Alvi, Andrew Zisserman, and Christoffer Nellker Bias Estimation in Face Analytics ECCV 2018 Workshop, 14/09/2018 Introduction

slide-1
SLIDE 1

Turning a Blind Eye:

Explicit Removal of Biases and Variation from Deep Neural Network Embeddings

Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics ECCV 2018 Workshop, 14/09/2018

slide-2
SLIDE 2

Introduction

mohsalvi@robots.ox.ac.uk 2

Convolutional Neural Networks are the state-of-the-art in image classification

Rely on “good” training data Not clear what the network has learned Networks can cheat by learning spurious variations Lack of comprehensively annotated data

slide-3
SLIDE 3

Contents

  • Removing a bias from a network
  • Removing multiple spurious variations from a network
  • LAOFIW – Labeled Ancestral Faces in the Wild

mohsalvi@robots.ox.ac.uk 3

slide-4
SLIDE 4

Spurious Variations vs Biases

mohsalvi@robots.ox.ac.uk 4

Spurious Variations Biases “Side information”

slide-5
SLIDE 5

Gender Classification from Celebrity Faces

mohsalvi@robots.ox.ac.uk 5

slide-6
SLIDE 6

IMDB Faces Dataset [1]

  • Dataset consists of celebrity faces from International Movie DataBase
  • Contains Age, Gender, Identity Labels
  • Created two subsets of the dataset
  • Age
  • Gender
  • Cleaned labels

mohsalvi@robots.ox.ac.uk 6

[1] Rothe, Timofte, Van Gool. (2015)

slide-7
SLIDE 7

Biased Datasets

mohsalvi@robots.ox.ac.uk 7

slide-8
SLIDE 8

Extremely Biased Datasets

mohsalvi@robots.ox.ac.uk 8

Training data

?

Test data

slide-9
SLIDE 9

Extremely Biased Datasets

Only young women, only old men

mohsalvi@robots.ox.ac.uk 9

slide-10
SLIDE 10

Training on Extremely Biased Data

Evaluated on a gender/age balanced test dataset

Gender classification accuracy: 70%

mohsalvi@robots.ox.ac.uk 10

slide-11
SLIDE 11

Turning a Blind Eye

  • Primary task is the attribute of interest
  • Gender Classification
  • Secondary task denotes bias to be unlearned
  • Age Classification
  • Objective:

learn feature representation that is informative for primary task, and uninformative for secondary task

  • Repurpose work in field of domain adaptation [2]

mohsalvi@robots.ox.ac.uk 11

[2] Tzeng, Hoffman, Darrell, and Saenko. (2015)

slide-12
SLIDE 12

Methods (1/3)

Based on VGG-M Network [2]

[2] Chatfield, Simonyan, Vedaldi, and Zisserman. (2014)

Minimize:

mohsalvi@robots.ox.ac.uk 12

slide-13
SLIDE 13

Methods (2/3)

Primary Loss

mohsalvi@robots.ox.ac.uk 13

Minimize:

slide-14
SLIDE 14

Methods (3/3)

Cross-entropy between classifier output and uniform distribution

mohsalvi@robots.ox.ac.uk 14

Minimize: Confusion Loss Secondary Loss

Secondary Classification Secondary Confusion

Act in opposition to each other Alternate:

Primary Classification & Secondary Confusion Secondary Classification Primary Classification Secondary Confusion

slide-15
SLIDE 15

Results – Removing a bias

Baseline Gender classification accuracy: 70% Age-Blind Gender classification accuracy: 86%

mohsalvi@robots.ox.ac.uk 15

slide-16
SLIDE 16

Removing multiple spurious variations (1/2)

Problem 1: Multiple biases may be present in dataset

mohsalvi@robots.ox.ac.uk 16

Age Gender Pose Expression Ancestry

slide-17
SLIDE 17

Removing multiple spurious variations (2/2)

  • Problem 2: no single dataset contains labels for all biases
  • Each labeled for a single purpose
  • Leverage information from multiple datasets

mohsalvi@robots.ox.ac.uk 17

Dataset 3: Ancestry Dataset 1: Gender Dataset 4: Pose Dataset 2: Age

slide-18
SLIDE 18

14,000 images in four classes:

  • Sub-Saharan Africa
  • Western Europe
  • East Asia
  • Indian subcontinent

Labeled Ancestral Origin Faces in the Wild LAOFIW

mohsalvi@robots.ox.ac.uk 18

slide-19
SLIDE 19

Methods - Removing multiple spurious variations

For M secondary tasks:

mohsalvi@robots.ox.ac.uk 19

slide-20
SLIDE 20

Removing multiple spurious variations experiments

mohsalvi@robots.ox.ac.uk 20

Ancestry Gender* Pose Age Gender Ancestry Pose Age

Experiment (1) Experiment (2) * Not extremely biased

slide-21
SLIDE 21

Results - Removing multiple spurious variations (1/2)

Primary task: Gender Secondary tasks: Age, Ancestry, Pose

mohsalvi@robots.ox.ac.uk 21

slide-22
SLIDE 22

Results - Removing multiple spurious variations (2/2)

Primary task: Ancestry Secondary tasks: Age, Gender, Pose

mohsalvi@robots.ox.ac.uk 22

slide-23
SLIDE 23

Conclusions

  • Can improve generalizability of models train on biased datasets
  • Can remove multiple spurious variations from feature representation
  • f network
  • LAOFIW – ancestral origin dataset

mohsalvi@robots.ox.ac.uk 23

slide-24
SLIDE 24

Questions?

mohsalvi@robots.ox.ac.uk 24