FR-Train: A Mutual Information-based Fair and Robust Training Yuji - - PowerPoint PPT Presentation

Yuji Roh, Data Intelligence Lab, KAIST

FR-Train:

A Mutual Information-based Fair and Robust Training

Yuji Roh, Kangwook Lee, Steven E. Whang, Changho Suh

Trustworthy AI

“AI has significant potential to help solve challenging problems, including by advancing medicine, understanding language, and fueling scientific discovery. To realize that potential, it’s critical that AI is used and developed responsibly. ”

• AI, 2020

“Moving forward, “build for performance” will not suffice as an AI design paradigm. We must learn how to build, evaluate and monitor for trust.”

• Trusting AI, 2020

2

Trustworthy AI

Robustness Value Alignment Transparency & Accountability Fairness Explainability

3

Trustworthy AI

Robustness Value Alignment Transparency & Accountability Fairness Explainability

Data-related

4

Two approaches

5

⚬ Two-step approach: Sanitize data -> Fair training Downside: very difficult to “decouple" poisoning and bias

Poisoned Dataset Sanitization Fair Training

Two approaches

6

⚬ Two-step approach: Sanitize data -> Fair training Downside: very difficult to “decouple" poisoning and bias ⚬ Holistic approach: Fair & Robust training Performing the two operations along with model training results in much better performance

Poisoned Dataset Sanitization Fair Training Poisoned Dataset Fair and Robust Training

Two approaches

7

⚬ Two-step approach: Sanitize data -> Fair training Downside: very difficult to “decouple" poisoning and bias ⚬ Holistic approach: Fair & Robust training Performing the two operations along with model training results in much better performance

Poisoned Dataset Sanitization Fair Training Poisoned Dataset Fair and Robust Training

FR-Train

03

Motivation

01 02 04

FR-Train Experiments Takeaways

8

03

Motivation

01 02 04

FR-Train Experiments Takeaways

9

Trustworthy AI

Robustness Value Alignment Transparency & Accountability Fairness Explainability

Data-related

10

Fairness

11

⚬ A machine learning model learns bias and discriminations in the data ⚬ The fairness of a (binary) classifier can be defined in various ways: ⚬ The level of fairness can be measured as a ratio or difference

Demographic Parity

(⇔ Disparate Impact)

Equalized Odds

Feature Label Group attribute Predicted label

⚬ A machine learning model learns bias and discriminations in the data ⚬ The fairness of a (binary) classifier can be defined in various ways: ⚬ The level of fairness can be measured as a ratio or difference

Fairness

12

Demographic Parity

(⇔ Disparate Impact)

Equalized Odds

In this talk

Feature Label Group attribute Predicted label

Robustness

⚬ Datasets are easy to publish nowadays, but as a result easy to “poison" as well

• Poison = noisy, subjective, or even adversarial
• Attacker’s goal : Increase the test loss by poisoning data
• Defender’s goal : Train a classifier with small test loss

⚬ Already a serious issue in federated learning

13

Fairness + Robustness

14

What happens if we just apply a fairness-aware algorithm on a poisoned dataset?

• May result in a strictly suboptimal (accuracy, fairness) than vanilla training

Motivating example

A B A A B A B B A B

Clean

X 15

A, B : Sensitive groups : Positive label : Negative label Vanilla classifier (Acc, DI) = (1, 0.5)

A B A A B A B B A B

Motivating example

Clean

Vanilla classifier (Acc, DI) = (1, 0.5)

16 X

A, B : Sensitive groups : Positive label : Negative label

A B A A B A B B A B

Motivating example

Clean

Vanilla classifier (Acc, DI) = (1, 0.5) Fair classifier (Acc, DI) = (0.8, 1)

17 X

A, B : Sensitive groups : Positive label : Negative label

A B A A B A B B A B A B A A B A B B A B

Motivating example

Vanilla classifier (Acc, DI) = (1, 0.5) Fair classifier (Acc, DI) = (0.8, 1)

Clean Poisoned

X 18 X

A, B : Sensitive groups : Positive label : Negative label

A B A A B A B B A B A B A A B A B B A B

Motivating example

Vanilla classifier (Acc, DI) = (1, 0.5) Fair classifier (Acc, DI) = (0.8, 1) Vanilla classifier Accpoi = 0.9 (Accclean, DI) = (0.9, 0.67)

Clean Poisoned

Acc : ↓ DI : ↑

19 X X

A, B : Sensitive groups : Positive label : Negative label

A B A A B A B B A B A B A A B A B B A B

Motivating example

Vanilla classifier (Acc, DI) = (1, 0.5) Fair classifier (Acc, DI) = (0.8, 1) Vanilla classifier Accpoi = 0.9 (Accclean, DI) = (0.9, 0.67) Fair classifier Accpoi = 0.8 (Accclean, DI) = (0.6, 1)

Clean Poisoned

Acc : ↓ DI : 一

20 X X

A, B : Sensitive groups : Positive label : Negative label

Suboptimal

Fairness + Robustness

21

What happens if we just apply a fairness-aware algorithm on a poisoned dataset?

• May result in a strictly suboptimal (accuracy, fairness) than vanilla training

We need a holistic approach to fair and robust training. FR-Train!

03

Motivation

01 02 04

FR-Train Experiments Takeaways

22

FR-Train - Main contributions

23

⚬ FR-Train is a holistic framework for fair and robust training ⚬ Extends a state-of-the-art fairness-only method called Adversarial Debiasing

• Provides a novel mutual information (MI)-based interpretation of adversarial learning
• Adds a robust discriminator that uses a small clean validation set for data sanitization

⚬ We also propose crowdsourcing methods for constructing a clean validation set

FR-Train

“Classifier”

Giving loans

24

Permit

• r

Deny “Discriminator for Fairness”

Distinguish the gender

Male

• r

Female

FR-Train

“Classifier”

Giving loans

“Discriminator for Fairness”

Distinguish the gender

25

Permit

• r

Deny Male

• r

Female “Discriminator for Robustness”

Distinguish whether poisoned or clean Poisoned training set + Predicted label affected by poisoning

FR-Train

“Classifier”

Giving loans

“Discriminator for Fairness”

Distinguish the gender

26

Permit

• r

Deny Male

• r

Female + Clean true label “Discriminator for Robustness”

Distinguish whether poisoned or clean Clean validation set Poisoned training set + Predicted label affected by poisoning

Poisoned set

• r

Clean val. set

FR-Train

“Classifier”

Giving loans

“Discriminator for Fairness”

Distinguish the gender

27

Permit

• r

Deny Male

• r

Female + Clean true label “Discriminator for Robustness”

Distinguish whether poisoned or clean Clean validation set Poisoned training set + Predicted label affected by poisoning

Poisoned set

• r

Clean val. set

Constructed with crowdsourcing

Mutual information-based interpretation

Classifier Disc. Robustness Softmax

28

Theorem 1 - Fairness Theorem 2 - Robustness Disc. Fairness

Mutual information-based interpretation

Classifier Disc. Robustness Softmax

29

Theorem 1 - Fairness Theorem 2 - Robustness Disc. Fairness

Mutual information-based interpretation

Classifier Disc. Robustness Softmax

30

Theorem 1 - Fairness Theorem 2 - Robustness Disc. Fairness

Mutual information-based interpretation

Classifier Disc. Robustness Softmax

31

Theorem 1 - Fairness Theorem 2 - Robustness

See paper for proofs

Disc. Fairness

Mutual information-based interpretation

Classifier Disc. Fairness Disc. Robustness Softmax

32

Theorem 1 - Fairness Theorem 2 - Robustness

Reweight examples

03

Motivation

01 02 04

FR-Train Experiments Takeaways

33

Experimental setting

34

⚬ Synthetic data

• Poisoning (label flipping): 10% of training data
• Validation set: 10% of training data

⚬ Real data (results in paper)

• COMPAS: Predict recidivism in two years for criminals
• AdultCensus: Predict whether annual income > $50K or not
• Poisoning: 10% of training data
• Validation set: 5% of training data

Synthetic data results

35

Two-step approach : Data sanitization + Fair training Fair-only algorithms Data sanitization using clean val. set Logistic regression

Synthetic data results

36

Two-step approach : Data sanitization + Fair training Fair-only algorithms Data sanitization using clean val. set Logistic regression

Low accuracy

Synthetic data results

37

Two-step approach : Data sanitization + Fair training Fair-only algorithms Data sanitization using clean val. set Logistic regression

Low fairness

Synthetic data results

38

Two-step approach : Data sanitization + Fair training Fair-only algorithms Data sanitization using clean val. set Logistic regression

Also low accuracy

Synthetic data results

39

Two-step approach : Data sanitization + Fair training Fair-only algorithms Data sanitization using clean val. set Logistic regression

Holistic approach = high fairness & accuracy

03

Motivation

01 02 04

FR-Train Experiments Takeaways

40

Takeaways

⚬ Trustworthy AI needs both fair and robust training ⚬ However, addressing fairness and robustness separately is suboptimal ⚬ FR-Train is a holistic framework for trustworthy AI performing fair and robust training

• Mutual information-based interpretation of adversarial learning
• Novel architecture that enjoys the synergistic effect of fair and robust discriminators
• Requires a small clean validation set, which can be constructed using crowdsourcing

⚬ Lots of open problems

• Without clean validation set
• Other poisoning
• Algorithm stability

41

Takeaways

42

Thank you :)

⚬ Trustworthy AI needs both fair and robust training ⚬ However, addressing fairness and robustness separately is suboptimal ⚬ FR-Train is a holistic framework for trustworthy AI performing fair and robust training

• Mutual information-based interpretation of adversarial learning
• Novel architecture that enjoys the synergistic effect of fair and robust discriminators
• Requires a small clean validation set, which can be constructed using crowdsourcing

⚬ Lots of open problems

• Without clean validation set
• Other poisoning
• Algorithm stability