Quick Question A doctor is walking down the street with a boy. The - - PowerPoint PPT Presentation

▶
quick question
SMART_READER_LITE
LIVE PREVIEW

Quick Question A doctor is walking down the street with a boy. The - - PowerPoint PPT Presentation

Mar 18, 2024 180 likes •689 views

Quick Question A doctor is walking down the street with a boy. The boy is the doctors son, but the doctor is not the boys father. How is that possible? GENDER BIAS IN WORD EMBEDDINGS 1 Simple Answer The doctor is the boys

slide-1
SLIDE 1

GENDER BIAS IN WORD EMBEDDINGS

Quick Question

  • A doctor is walking down the street with a boy.

The boy is the doctor’s son, but the doctor is not the boy’s father. How is that possible?

1

slide-2
SLIDE 2

GENDER BIAS IN WORD EMBEDDINGS

Simple Answer

  • The doctor is the boy’s mother…

2

slide-3
SLIDE 3

Lipstick on a Pig:

Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them

HILA GONEN AND YOAV GOLDBERG BAR ILAN UNIVERSITY INRIA PARIS 12/3/19

ACCEPTED TO NAACL 2019

slide-4
SLIDE 4

GENDER BIAS IN WORD EMBEDDINGS

Outline

  • Background
  • Gender Bias in word embeddings
  • Current debiasing methods
  • Post-processing (Bolukbasi et al.)
  • During training (Zhao et al.)
  • Experiments that reveal the remaining bias
  • Conclusion

4

slide-5
SLIDE 5

Gender Bias in Applications

5

slide-6
SLIDE 6

GENDER BIAS IN WORD EMBEDDINGS

What do we mean by gender bias?

6

slide-7
SLIDE 7

GENDER BIAS IN WORD EMBEDDINGS

What do we mean by gender bias?

7

slide-8
SLIDE 8

GENDER BIAS IN WORD EMBEDDINGS

What do we mean by gender bias?

(Zhao et al.,NAACL, 2018)

8

slide-9
SLIDE 9

GENDER BIAS IN WORD EMBEDDINGS

What do we mean by gender bias?

(Hendricks et al., 2018)

9

slide-10
SLIDE 10

GENDER BIAS IN WORD EMBEDDINGS

Word Embeddings

  • TopK lists:

(Mikolov et al. 2013)

10

nurse

slide-11
SLIDE 11

GENDER BIAS IN WORD EMBEDDINGS

Word Embeddings

  • We will focus on gender bias in word embeddings

11

slide-12
SLIDE 12

Bias in word embeddings

12

slide-13
SLIDE 13

GENDER BIAS IN WORD EMBEDDINGS

Bias in Word Embeddings

13

slide-14
SLIDE 14

GENDER BIAS IN WORD EMBEDDINGS

Bias in Word Embeddings

14

  • Caliskan et al. replicate a spectrum of known biases from the literature using

word embeddings

  • Show that text corpora contain several types of biases:
  • morally neutral as toward insects or flowers
  • problematic as toward race or gender
  • veridical, reflecting the distribution of gender with respect to careers or first names
  • Introduce methods for identifying these biases
slide-15
SLIDE 15

GENDER BIAS IN WORD EMBEDDINGS

Bias in Word Embeddings

15

slide-16
SLIDE 16

GENDER BIAS IN WORD EMBEDDINGS

Bias in Word Embeddings

16

  • Permutation test:
  • X, Y: sets of target words (e.g. male names vs. female names)
  • A, B: sets of attribute words (e.g. career terms vs. family terms)
  • P-value is:
slide-17
SLIDE 17

GENDER BIAS IN WORD EMBEDDINGS

Bias in Word Embeddings

17

Concepts 1 Concepts 2 Attributes 1 Attributes 2 Flowers: buttercup, daisy, lily Insects: ant, caterpillar, flea Pleasant: freedom, health, love Unpleasant: abuse, crash, filth European American names: Brad, Brendan African American names: Darnell, Lakisha Pleasant: joy, love, peace Unpleasant: agony, terrible Male attributes: male, man, boy Female attributes: female, woman, girl Math words: math, algebra, geometry Arts Words: poetry, art, dance

slide-18
SLIDE 18

Definitions of Gender Bias in Word Embeddings

18

slide-19
SLIDE 19

GENDER BIAS IN WORD EMBEDDINGS

Definition of Gender Bias in Word Embeddings

19

(NIPS, 2016)

slide-20
SLIDE 20

GENDER BIAS IN WORD EMBEDDINGS

Definition of Gender Bias in Word Embeddings

20

  • We check how similar a word is to “he” and “she” (cosine similarity)
  • Note that we care about the difference between the two
  • This is the projection on the direction of “he – she”*

* This is the gender direction, can be computed using several pairs together (e.g. man-woman, brother-sister)

slide-21
SLIDE 21

GENDER BIAS IN WORD EMBEDDINGS

Definition of Gender Bias in Word Embeddings

21

Zhao et al.

  • bias(consultant) = -0.0023
  • bias(nurse) = -0.2471
  • bias(captain) = 0.1521
  • bias(table) = -0.0003
slide-22
SLIDE 22

Reducing of Gender Bias in Word Embeddings

22

slide-23
SLIDE 23

GENDER BIAS IN WORD EMBEDDINGS

Reduce Bias after Training

  • Bolukbasi et al. suggest to remove bias after training by removing

the projection of every neutral word on the gender direction

23

slide-24
SLIDE 24

GENDER BIAS IN WORD EMBEDDINGS

Reduce Bias after Training

  • 1. Define a gender direction:
  • 10 gender pair difference vectors:
  • woman, man | girl, boy | she, he | mother, father

daughter, son | gal, guy | female, male | her, his herself, himself | Mary, John

  • Compute and use their principal component

24

slide-25
SLIDE 25

GENDER BIAS IN WORD EMBEDDINGS

Reduce Bias after Training

  • 2. Define inherently neutral words:
  • Identify the set of gender specific words
  • The authors derive a list of 218 words from dictionary definitions:
  • mother, aunt, chairman, girlfriend, prince
  • The complementary set are the gender neutral words
  • The authors generalize the list to a broader vocabulary using SVM (~6500

words)

25

slide-26
SLIDE 26

GENDER BIAS IN WORD EMBEDDINGS

Reduce Bias after Training

  • 3. Zero the projection of all neutral words on the gender direction:
  • The bias of all neutral words is now zero by definition

26

Projection of on gender direction 𝑥

slide-27
SLIDE 27

GENDER BIAS IN WORD EMBEDDINGS

Reduce Bias after Training

  • 4. Equalize:
  • A family of equality sets (pairs):
  • For each pair, compute:
  • Equalize the words in the pair:

27

normalized, same bias for both words

slide-28
SLIDE 28

GENDER BIAS IN WORD EMBEDDINGS

Reduce Bias after Training

  • We will address these embeddings as HARD-DEBIASED

28

slide-29
SLIDE 29

GENDER BIAS IN WORD EMBEDDINGS

Reduce bias during training

(EMNLP, 2018)

29

slide-30
SLIDE 30

GENDER BIAS IN WORD EMBEDDINGS

Reduce bias during training

  • Zhao et al. suggest to reduce bias during training:
  • Train word embeddings using GloVe (Pennington et al., 2014)
  • Alter the loss to encourage the gender information to concentrate in the last

coordinate

  • To ignore gender information – simply remove the last coordinate

30

slide-31
SLIDE 31

GENDER BIAS IN WORD EMBEDDINGS

Reduce bias during training

  • Loss:

31

GloVe component (captures word proximity)

slide-32
SLIDE 32

GENDER BIAS IN WORD EMBEDDINGS

Reduce bias during training

  • Use two groups of male/female seed words, and encourage words from

different groups to differ in their last coordinate:

  • Seed words – according to WordNet
  • *The authors also experiment with another variant for this component

33

Gender coordinate(s)

slide-33
SLIDE 33

GENDER BIAS IN WORD EMBEDDINGS

Reduce bias during training

  • Encourage the representation of neutral-gender words (excluding

the last coordinate) to be orthogonal to the gender direction:

34

Vector w/o gender coordinate(s)

slide-34
SLIDE 34

GENDER BIAS IN WORD EMBEDDINGS

Reduce bias during training

  • Gender direction is estimated on the fly
  • Averaging differences between pairs of male-female words

35

slide-35
SLIDE 35

GENDER BIAS IN WORD EMBEDDINGS

Reduce bias during training

  • We will address these embeddings as GN-GLOVE

36

slide-36
SLIDE 36

GENDER BIAS IN WORD EMBEDDINGS

Some Results

37

  • Bolukbasi et al.:
  • Bias of all inherently-neutral words is zero by definition
  • Generated analogies are less stereotyped
slide-37
SLIDE 37

GENDER BIAS IN WORD EMBEDDINGS

Some Results

38

  • Zhao et al.:
  • Decrease bias in co-reference resolution
slide-38
SLIDE 38

GENDER BIAS IN WORD EMBEDDINGS

Problem solved?

39

  • Not so fast…
slide-39
SLIDE 39

GENDER BIAS IN WORD EMBEDDINGS

Clustering male- and female- biased words

40

  • We take the most biased words in the vocabulary according to the
  • riginal bias (500 male-biased and 500 female-biased)
  • We cluster them into two clusters using K-means
  • The clusters align with gender with accuracy of:
  • 92.5% compared to 99.99% (HARD-DEBIASED)
  • 85.6% compared to 100% (GN-GLOVE)
slide-40
SLIDE 40

GENDER BIAS IN WORD EMBEDDINGS

Clustering male- and female- biased words

41

HARD-DEBIASED GN-GLOVE

slide-41
SLIDE 41

GENDER BIAS IN WORD EMBEDDINGS

Bias-by-neighbors

  • Bias is still manifested by the word being close to socially-marked

feminine words

  • A new mechanism for measuring bias:
  • The percentage of male/female socially-biased words among the k

nearest neighbors of the target word

  • Pearson correlation with bias-by-projection:
  • 0.686 compared to 0.741 (HARD-DEBIASED)
  • 0.736 compared to 0.773 (GN-GLOVE)

42

slide-42
SLIDE 42

GENDER BIAS IN WORD EMBEDDINGS

Professions

  • We take a predefined list of professions
  • We show correlation between the bias-by-projection and bias-by-

neighbors, before and after debiasing

43

slide-43
SLIDE 43

GENDER BIAS IN WORD EMBEDDINGS

Professions

44

HARD-DEBIASED GN-GLOVE

slide-44
SLIDE 44

GENDER BIAS IN WORD EMBEDDINGS

Association with stereotyped words

45

  • We evaluate the association between female/male names and

female/male stereotyped words (experiments taken from Caliskan et al.)

  • All the associations have very small p-values

Female-associated Male-associated names Amy, Joan, Lisa John, Paul, Mike family vs. carrer home, parents, children executive, management, professional arts vs. math poetry, art, dance math, algebra, geometry arts vs. science dance, literature, novel science, technology, physics

slide-45
SLIDE 45

GENDER BIAS IN WORD EMBEDDINGS

Classifying to gender

46

  • Can a classifier learn to generalize from some gendered words to
  • thers based only on their representations?

5000 most biased words 1000 4000

test train SVM

slide-46
SLIDE 46

GENDER BIAS IN WORD EMBEDDINGS

Classifying to gender

47

  • Results:
slide-47
SLIDE 47

GENDER BIAS IN WORD EMBEDDINGS

Conclusion

48

  • Word embeddings exhibit gender bias
  • Debiasing is hard!
  • Social gender bias is picked up from the data by the models
  • A lot of the bias information is still recoverable

(even when the bias is low/zero according to the definition usually used)

  • The way we define the bias is important, and needs to be

reconsidered when trying to solve the problem

slide-48
SLIDE 48

GENDER BIAS IN WORD EMBEDDINGS

Questions?

49

slide-49
SLIDE 49

GENDER BIAS IN WORD EMBEDDINGS

Thank you!

50