Biases in NLP Models and What It Takes to Control them Kai-Wei - - PowerPoint PPT Presentation

▶
biases in nlp models and what it takes to control them
SMART_READER_LITE
LIVE PREVIEW

Biases in NLP Models and What It Takes to Control them Kai-Wei - - PowerPoint PPT Presentation

Feb 27, 2024 182 likes •920 views

Biases in NLP Models and What It Takes to Control them Kai-Wei Chang 1 A carton of ML (NLP) pipeline Prediction Evaluation (Structured) Inference Auxiliary Corpus/Models Representation (e.g, word embedding) Data Kai-Wei Chang

slide-1
SLIDE 1

Biases in NLP Models and What It Takes to Control them

Kai-Wei Chang

1

slide-2
SLIDE 2

A carton of ML (NLP) pipeline

Kai-Wei Chang (kw@kwchang.net) 2

Representation (Structured) Inference Prediction

Auxiliary Corpus/Models (e.g, word embedding)

Data Evaluation

slide-3
SLIDE 3

Motivate Example:

Coreference Resolution

Semantics Only w/ Syntactic Cues

  • Coreference resolution is biased1,2
  • Model fails for female when given same context

3

1Zhao et al. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. NAACL 2018. 2Rudinger et al. Gender Bias in Coreference Resolution. NAACL 2018

Kai-Wei Chang (kw@kwchang.net)

his ⇒ her

slide-4
SLIDE 4

Wino-bias data

v Stereotypical dataset v Anti-stereotypical dataset

Kai-Wei Chang (kw@kwchang.net) 4

slide-5
SLIDE 5

Gender bias in Coref System

48 53 58 63 68 73 78 E2E E2E (Debiased WE) E2E (Full model)

Steoetype Anti-Steoretype Avg

Kai-Wei Chang (kw@kwchang.net) 5

Neural Coref Model

slide-6
SLIDE 6

Gender bias in Coref System

48 53 58 63 68 73 78 E2E E2E (Debiased WE) E2E (Full model)

Steoetype Anti-Steoretype Avg

Kai-Wei Chang (kw@kwchang.net) 6

Neural Coref Model

slide-7
SLIDE 7

Gender bias in Coref System

48 53 58 63 68 73 78 E2E E2E (Debiased WE) E2E (Full model)

Steoetype Anti-Steoretype Avg

Kai-Wei Chang (kw@kwchang.net) 7

Neural Coref Model Mitigate WE Bias Mitigate Data Bias

slide-8
SLIDE 8

Misrepresentation and Bias

Kai-Wei Chang (kw@kwchang.net) 8

slide-9
SLIDE 9

Stereotypes

Which word is more likely to be used by a female ? (Preotiuc-Pietro et al. ‘16)

Giggle – Laugh

Credit: Yulia Tsvetkov

Kai-Wei Chang (kw@kwchang.net) 9

slide-10
SLIDE 10

Stereotypes

Which word is more likely to be used by a female ? (Preotiuc-Pietro et al. ‘16)

Giggle – Laugh

Kai-Wei Chang (kw@kwchang.net) 10

Credit: Yulia Tsvetkov

slide-11
SLIDE 11

Stereotypes

Which word is more likely to be used by a

  • lder person ?

(Preotiuc-Pietro et al. ‘16)

Impressive – Amazing

Kai-Wei Chang (kw@kwchang.net) 11

Credit: Yulia Tsvetkov

slide-12
SLIDE 12

Stereotypes

Which word is more likely to be used by a

  • lder person ?

(Preotiuc-Pietro et al. ‘16)

Impressive – Amazing

Kai-Wei Chang (kw@kwchang.net) 12

Credit: Yulia Tsvetkov

slide-13
SLIDE 13

Why do we intuitively recognize a default social group?

13

Credit: Yulia Tsvetkov

slide-14
SLIDE 14

Why do we intuitively recognize a default social group? Implicit Bias

14

Credit: Yulia Tsvetkov

slide-15
SLIDE 15

Data is riddled with Implicit Bias AI

BIASED

15

Modified from Yulia Tsvetkov’s slide

slide-16
SLIDE 16

Bias in Wikipedia

v Only small portion of editors are female

v Have less extensive articles about women v Have fewer topics important to women.

16

(Ruediger et al., 2010)

Kai-Wei Chang (kw@kwchang.net)

slide-17
SLIDE 17

Consequence: models are biased AI

BIASED

Credit: Yulia Tsvetkov

17

slide-18
SLIDE 18

Bias in Language Generation

  • Language generation is biased (GPT-2)

18

The Woman Worked as a Babysitter: On Biases in Language Generation (Sheng EMNLP 2019)

Kai-Wei Chang (kw@kwchang.net)

slide-19
SLIDE 19

Where’s Biases?

Kai-Wei Chang (kw@kwchang.net) 19

slide-20
SLIDE 20

A carton of ML (NLP) pipeline

Kai-Wei Chang (kw@kwchang.net) 20

Representation (Structured) Inference Prediction

Auxiliary Corpus/Models (e.g, word embedding)

Data Evaluation

slide-21
SLIDE 21

Representational Harm in NLP: Word Embeddings can be Sexist

Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings [Bolukbasi et al. NeurIPS16] 21 Google w2v embedding trained from the news

he: _______ she:_______ brother sister beer cocktail physician registered_nurse professor associate professor

Given gender direction (𝑤#$ − 𝑤&#$), find word pairs with parallel direction by cos(𝑤, − 𝑤-, 𝑤#$ − 𝑤&#$)

he she

Kai-Wei Chang (kw@kwchang.net)

slide-22
SLIDE 22

Implicit association test (IAT)

v Greenwald et al. 1998 v Detect the strength of a person's subconscious association between mental representations of

  • bjects (concepts)

Kai-Wei Chang (kw@kwchang.net) 22

https://implicit.harvard.edu

Boy Girl Math Reading

https://en.wikipedia.org/wiki/Implicit-association_test

slide-23
SLIDE 23

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 23

https://implicit.harvard.edu

Boy Girl

slide-24
SLIDE 24

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 24

https://implicit.harvard.edu

Boy Emily Girl

slide-25
SLIDE 25

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 25

https://implicit.harvard.edu

Boy Tom Girl

slide-26
SLIDE 26

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 26

https://implicit.harvard.edu

Math Reading

slide-27
SLIDE 27

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 27

https://implicit.harvard.edu

Math number Reading

slide-28
SLIDE 28

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 28

https://implicit.harvard.edu

Math Reading Boy Girl

slide-29
SLIDE 29

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 29

https://implicit.harvard.edu

Math Reading Boy Girl Algebra

slide-30
SLIDE 30

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 30

https://implicit.harvard.edu

Math Reading Boy Girl Julia

slide-31
SLIDE 31

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 31

https://implicit.harvard.edu

Reading Math Boy Girl

slide-32
SLIDE 32

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 32

https://implicit.harvard.edu

Reading Math Boy Girl Literature

slide-33
SLIDE 33

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 33

https://implicit.harvard.edu

Reading Math Boy Girl Dan

slide-34
SLIDE 34

Implicit association test (IAT)

Kai-Wei Chang (kw@kwchang.net) 34

https://implicit.harvard.edu

slide-35
SLIDE 35

Word Embedding Association Test (WEAT)

  • X: “mathematics”, “science”; Y: “arts”, “design”
  • A: “male”, “boy”; B: “female”, “girl”

35

Caliskan et al. Semantics derived automatically from language corpora contain human-like biases Science. 2017 “mathematics” “male”, “boy” “female”, “girl”

Kai-Wei Chang (kw@kwchang.net)

slide-36
SLIDE 36

Word Embedding Association Test (WEAT)

  • X: “mathematics”, “science”; Y: “arts”, “design”
  • A: “male”, “boy”; B: “female”, “girl”

36

Differential association of the two sets of words with the attributes Aggregate the target words

Kai-Wei Chang (kw@kwchang.net)

Caliskan et al. Semantics derived automatically from language corpora contain human-like biases Science. 2017

slide-37
SLIDE 37

Word Embedding Association Test (WEAT)

  • X: “mathematics”, “science”; Y: “arts”, “design”
  • A: “male”, “boy”; B: “female”, “girl”

37

Caliskan et al. Semantics derived automatically from language corpora contain human-like biases Science. 2017 The effect size of bias:

Kai-Wei Chang (kw@kwchang.net)

slide-38
SLIDE 38

Word Embedding Association Test

Caliskan et al. (2017)

IAT WEAT

Kai-Wei Chang (kw@kwchang.net) 38

slide-39
SLIDE 39

Word Embedding Association Test

Caliskan et al. (2017)

WEAT finds similar biases in Word Embeddings as IAT did for humans

Kai-Wei Chang (kw@kwchang.net) 39

slide-40
SLIDE 40

Kai-Wei Chang (kw@kwchang.net) 40

he she father mother king queen

slide-41
SLIDE 41

Kai-Wei Chang (kw@kwchang.net) 41

slide-42
SLIDE 42

Can we Extend the Analysis beyond Binary Gender?

Kai-Wei Chang (kw@kwchang.net) 42

slide-43
SLIDE 43

Beyond Gender & Race/Ethnicity Bias

Manzini et al. NAACL 2019 Biases in word embeddings trained on the Reddit data from US users.

Kai-Wei Chang (kw@kwchang.net) 43

slide-44
SLIDE 44

How about other Embedding?

Kai-Wei Chang (kw@kwchang.net) 44

slide-45
SLIDE 45

Bias Only in English?

v Language with grammatical gender

vMorphological agreement

Kai-Wei Chang (kw@kwchang.net) 45

(Zhou et al, EMNLP 2019)

slide-46
SLIDE 46

v Linear Discriminative Analysis (LDA)

v Identify grammatical gender direction

Kai-Wei Chang (kw@kwchang.net) 46

masculine words feminine words

slide-47
SLIDE 47

Kai-Wei Chang (kw@kwchang.net) 47

Male Female masculine feminine

slide-48
SLIDE 48

Kai-Wei Chang (kw@kwchang.net) 48

Male Female masculine feminine

slide-49
SLIDE 49

Kai-Wei Chang (kw@kwchang.net) 49

Male Female masculine feminine

slide-50
SLIDE 50

How about bilingual embedding?

[Zhou et al. EMNLP19]

Kai-Wei Chang (kw@kwchang.net) 50 Female doctor in Spanish male doctor in Spanish

slide-51
SLIDE 51

How about Contextualized Representation?

Gender Bias in Contextualized Word Embeddings

51 Kai-Wei Chang (kw@kwchang.net)

v First two components explain more variance than others

(Feminine) The driver stopped the car at the hospital because she was paid to do so (Masculine) The driver stopped the car at the hospital because he was paid to do so

gender direction: ELMo(driver) – ELMo(driver)

Zhao et al. NAACL 19

slide-52
SLIDE 52

The driver stopped the car at the hospital because she was paid to do so

Unequal Treatment of Gender

v Classifier

52

f :

<latexit sha1_base64="MiBzBz8O6rmAcrAGaRyZjZKL+Rs=">AB63icbVA9SwNBEJ2LXzF+RS1tFhPBKtwlhWIVtLGMYD4gOcLeZi9Zsrt37O4J4chfsLFQxNY/ZOe/cS+5QhMfDzem2FmXhBzpo3rfjuFjc2t7Z3ibmlv/+DwqHx80tFRoghtk4hHqhdgTmTtG2Y4bQXK4pFwGk3mN5lfveJKs0i+WhmMfUFHksWMoJNJlXDm+qwXHFr7gJonXg5qUCO1rD8NRhFJBFUGsKx1n3PjY2fYmUY4XReGiSaxphM8Zj2LZVYUO2ni1vn6MIqIxRGypY0aKH+nkix0HomAtspsJnoVS8T/P6iQmv/ZTJODFUkuWiMOHIRCh7HI2YosTwmSWYKGZvRWSCFSbGxlOyIXirL6+Tr3mNWr1h3qleZvHUYQzOIdL8OAKmnAPLWgDgQk8wyu8OcJ5cd6dj2VrwclnTuEPnM8fAOmNjA=</latexit>

ELMo(occupation) →

<latexit sha1_base64="r2/BIv92hsxnS9nS8QghuqsRs=">AB9HicbVA9TwJBEJ3DL8Qv1NLmIphYkTstCTaWGIiYAIXsrcsGFv9ydw5ALv8PGQmNs/TF2/hsXuELBl0zy8t5MZuaFseAGPe/bya2tb2xu5bcLO7t7+wfFw6OmUYmrEGVUPohJIYJLlkDOQr2EGtGolCwVji6mfmtMdOGK3mPk5gFERlI3ueUoJWCckfzwRCJ1uqp3C2WvIo3h7tK/IyUIEO9W/zq9BRNIiaRCmJM2/diDFKikVPBpoVOYlhM6IgMWNtSJmgnR+9NQ9s0rP7StS6I7V39PpCQyZhKFtjMiODTL3kz8z2sn2L8KUi7jBJmki0X9RLio3FkCbo9rRlFMLCFUc3urS4dE4o2p4INwV9+eZU0qxX/olK9q5Zq1kceTiBUzgHy6hBrdQhwZQeIRneIU3Z+y8O/Ox6I152Qzx/AHzucPgN+R6w=</latexit>

context gender

f

<latexit sha1_base64="nCmo2IjEmAQavNbnXg3xD8FOzI=">AB6nicbVA9TwJBEJ3DL8Qv1NJmI5hYkTstCTaWGIUJIEL2VvmYMPe3mV3z4Rc+Ak2Fhpj6y+y89+4wBUKvmSl/dmMjMvSATXxnW/ncLa+sbmVnG7tLO7t39QPjxq6zhVDFsFrHqBFSj4BJbhuBnUQhjQKBj8H4ZuY/PqHSPJYPZpKgH9Gh5CFn1FjpvhpW+WKW3PnIKvEy0kFcjT75a/eIGZphNIwQbXuem5i/Iwqw5nAamXakwoG9Mhdi2VNELtZ/NTp+TMKgMSxsqWNGSu/p7IaKT1JApsZ0TNSC97M/E/r5ua8MrPuExSg5ItFoWpICYms7/JgCtkRkwsoUxeythI6oMzadkg3BW35lbTrNe+iVr+rVxrXeRxFOIFTOAcPLqEBt9CEFjAYwjO8wpsjnBfn3flYtBacfOY/sD5/AGDRo1I</latexit>

ELMo embeddings gender prediction

Kai-Wei Chang (kw@kwchang.net)

slide-53
SLIDE 53

Unequal Treatment of Gender

v Classifier

Acc (%) 80 85 90 95 100 Male Context Female Context

  • ELMo propagates

gender information to

  • ther words
  • Male information is

14% more accurately propagated than female

53

f :

<latexit sha1_base64="MiBzBz8O6rmAcrAGaRyZjZKL+Rs=">AB63icbVA9SwNBEJ2LXzF+RS1tFhPBKtwlhWIVtLGMYD4gOcLeZi9Zsrt37O4J4chfsLFQxNY/ZOe/cS+5QhMfDzem2FmXhBzpo3rfjuFjc2t7Z3ibmlv/+DwqHx80tFRoghtk4hHqhdgTmTtG2Y4bQXK4pFwGk3mN5lfveJKs0i+WhmMfUFHksWMoJNJlXDm+qwXHFr7gJonXg5qUCO1rD8NRhFJBFUGsKx1n3PjY2fYmUY4XReGiSaxphM8Zj2LZVYUO2ni1vn6MIqIxRGypY0aKH+nkix0HomAtspsJnoVS8T/P6iQmv/ZTJODFUkuWiMOHIRCh7HI2YosTwmSWYKGZvRWSCFSbGxlOyIXirL6+Tr3mNWr1h3qleZvHUYQzOIdL8OAKmnAPLWgDgQk8wyu8OcJ5cd6dj2VrwclnTuEPnM8fAOmNjA=</latexit>

ELMo(occupation) →

<latexit sha1_base64="r2/BIv92hsxnS9nS8QghuqsRs=">AB9HicbVA9TwJBEJ3DL8Qv1NLmIphYkTstCTaWGIiYAIXsrcsGFv9ydw5ALv8PGQmNs/TF2/hsXuELBl0zy8t5MZuaFseAGPe/bya2tb2xu5bcLO7t7+wfFw6OmUYmrEGVUPohJIYJLlkDOQr2EGtGolCwVji6mfmtMdOGK3mPk5gFERlI3ueUoJWCckfzwRCJ1uqp3C2WvIo3h7tK/IyUIEO9W/zq9BRNIiaRCmJM2/diDFKikVPBpoVOYlhM6IgMWNtSJmgnR+9NQ9s0rP7StS6I7V39PpCQyZhKFtjMiODTL3kz8z2sn2L8KUi7jBJmki0X9RLio3FkCbo9rRlFMLCFUc3urS4dE4o2p4INwV9+eZU0qxX/olK9q5Zq1kceTiBUzgHy6hBrdQhwZQeIRneIU3Z+y8O/Ox6I152Qzx/AHzucPgN+R6w=</latexit>

context gender The writer taught himself to play violin .

Kai-Wei Chang (kw@kwchang.net)

slide-54
SLIDE 54

40 50 60 70 80 GloVe + ELMo

OntoNotes Pro. Anti.

Coreference with contextualized embedding vELMo boosts the performance v However, enlarge the bias (Δ)

Δ: 29.6 Δ: 26.6

54 Kai-Wei Chang (kw@kwchang.net)

slide-55
SLIDE 55

Does such Bias do “Harm” Certain People?

55 Kai-Wei Chang (kw@kwchang.net)

slide-56
SLIDE 56

Biases in NLP Classifiers/Taggers

v Gender Bias in Coreference resolution

v Zhao, Jieyu, et al. Gender bias in coreference resolution: Evaluation and debiasing

  • methods. NAACL (2018)

v Webster, Kellie, et al. Mind the GAP: A Balanced Corpus of Gendered Ambiguous

  • Pronouns. TACL (2018)

v Gender, Race, and Age Bias in Sentiment Analysis

v Svetlana and Mohammad. Examining gender and race bias in two hundred sentiment

analysis systems. arXiv (2018)

v Díaz, et al. Addressing age-related bias in sentiment analysis. CHI Conference on

Human Factors in Comp. Systems. (2018)

v LGBTQ identitiy terms bias in Toxicity classification

v Dixon, et al. Measuring and mitigating unintended bias in text classification. AIES.

(2018)

v Gender Bias in Occupation Classification

v De-Arteaga et al. Bias in Bios: A Case Study of Semantic Representation Bias in a

High-Stakes Setting. FAT* (2019)

v Gender bias in Machine Translation

v Prates, et al. Assessing gender bias in machine translation: a case study with

Google Translate. Neural Computing and Applications (2018)

Kai-Wei Chang (kw@kwchang.net) 56

slide-57
SLIDE 57

Kai-Wei Chang (kw@kwchang.net) 57

Towards Inclusive AI

slide-58
SLIDE 58

Examples of Harm from NLP Bias

Swinger et al. (2019)

Kai-Wei Chang (kw@kwchang.net) 58

Prevent Allocative Harm in Sensitive Applications

slide-59
SLIDE 59

Can we remove these biases?

59

Control

Kai-Wei Chang (kw@kwchang.net)

slide-60
SLIDE 60

This can be done by projecting gender direction out from gender neutral words using linear operations

[Bolukbasi; NeurIPS 16]

Kai-Wei Chang (kw@kwchang.net) 60

slide-61
SLIDE 61

Towards Debiasing

  • 1. Identify gender subspace: B
  • 2. Identify gender-definitional (S) and gender-neutral words (N)
  • 3. Apply transform matrix (T) to the embedding matrix (W)
  • a. Project away the gender subspace B from the gender-neutral

words N

  • b. But, ensure the transformation doesn’t change the embeddings

too much

Don’t modify embeddings too much Minimize gender component T - the desired debiasing transformation B - biased space W - embedding matrix N - embedding matrix of gender neutral words

Bolukbasi et al. (2016)

61

slide-62
SLIDE 62

Make Gender Information Transparent in Word Embedding

Learning Gender-Neutral Word Embeddings [Zhao et al; EMNLP18] Kai-Wei Chang (kw@kwchang.net) 62

1

  • 1

?

mother father doctor dimensions reserve for gender information 𝑥0 dimensions for other latent aspects 𝑥,

slide-63
SLIDE 63

Make Gender Information Transparent in Word Embedding

Learning Gender-Neutral Word Embeddings [Zhao et al; EMNLP18] Kai-Wei Chang (kw@kwchang.net) 63

𝑥, 𝑥0

slide-64
SLIDE 64

Make Gender Information Transparent in Word Embedding

Learning Gender-Neutral Word Embeddings [Zhao et al; EMNLP18] Kai-Wei Chang (kw@kwchang.net) 64

!"

𝑥, 𝑥0

slide-65
SLIDE 65

Gender bias in Coref System

48 53 58 63 68 73 78 E2E E2E (Debiased WE) E2E (Full model)

Steoetype Anti-Steoretype Avg

Kai-Wei Chang (kw@kwchang.net) 65

slide-66
SLIDE 66

Is Gender Information Actually Removed from Embedding?

Kai-Wei Chang (kw@kwchang.net) 66

slide-67
SLIDE 67

Completely removing bias is hard

  • Gonen, et al. Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases

in Word Embeddings But do not Remove Them. NAACL (2019). Kai-Wei Chang (kw@kwchang.net) 67

Number of male neighbors for each occupation x-axis: original bias

slide-68
SLIDE 68

Completely removing bias is hard

  • Gonen, et al. Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases

in Word Embeddings But do not Remove Them. NAACL (2019). Kai-Wei Chang (kw@kwchang.net) 68

Number of male neighbors for each occupation x-axis: original bias

slide-69
SLIDE 69

Should We Debias Word Embedding?

Kai-Wei Chang (kw@kwchang.net) 69

v Awareness is better than blindness (Caliskan et. al. 17)

Representation (Structured) Inference Prediction Auxiliary Corpus/Models (e.g, word embedding) Data

Data Augmentation Calibration

slide-70
SLIDE 70

Wino-bias data

v Stereotypical dataset v Anti-stereotypical dataset

Kai-Wei Chang (kw@kwchang.net) 70

slide-71
SLIDE 71

Data Augmentation-- Balance the data

vGender Swapping -- simulate sentence in opposite gender

Kai-Wei Chang (kw@kwchang.net) 71

John went to his house F2 went to her house

Named Entity are anonymized Gender words are swapped

Better than down/up sampling This idea has been used in computer vision as well

slide-72
SLIDE 72

Reduce Bias via Data Augmentation in Coreference Resolution

Kai-Wei Chang (kw@kwchang.net) 72

20 40 60 80 without augment with augment OntoNotes Pro. Anti.

slide-73
SLIDE 73

v Various Biases are embedded in NLP models v Controlling Biases is still an open problem

Kai-Wei Chang (kw@kwchang.net) 73 73

[ACL 2019]