FairSquare: Probabilistic Verification of Program Fairness Aws - - PowerPoint PPT Presentation

▶
fairsquare
SMART_READER_LITE
LIVE PREVIEW

FairSquare: Probabilistic Verification of Program Fairness Aws - - PowerPoint PPT Presentation

Oct 02, 2023 255 likes •657 views

FairSquare: Probabilistic Verification of Program Fairness Aws Albarghouthi Loris DAntoni Samuel Drews University of Wisconsin-Madison Aditya V. Nori Microsoft Research Machine Data Model Learning Machine Data Model Learning small

slide-1
SLIDE 1

FairSquare:

Probabilistic Verification of Program Fairness

Aws Albarghouthi Loris D’Antoni Samuel Drews University of Wisconsin-Madison Aditya V. Nori Microsoft Research

slide-2
SLIDE 2

Data Model Machine Learning

slide-3
SLIDE 3

Data Model small error Machine Learning

slide-4
SLIDE 4

Data Model small error racist? sexist? Machine Learning

slide-5
SLIDE 5
slide-6
SLIDE 6

decision- making program

Group Fairness

slide-7
SLIDE 7

Group Fairness

sensitive feature (e.g. minority)

slide-8
SLIDE 8

Group Fairness

sensitive feature (e.g. minority)

slide-9
SLIDE 9

Group Fairness

population model

slide-10
SLIDE 10

Individual Fairness

similarity

slide-11
SLIDE 11

Q: What are good definitions of fairness?

slide-12
SLIDE 12

Q: What are good definitions of fairness? A: Someone else’s problem

slide-13
SLIDE 13

Contributions

  • Formalize fairness definitions as probabilistic

verification problems

  • Decision procedure for evaluating probabilistic

postconditions

slide-14
SLIDE 14

FairSquare

slide-15
SLIDE 15

Population Model Decision Program Fairness Definition

FairSquare

slide-16
SLIDE 16

Population Model Decision Program Unfairness proof Fairness proof Fairness Definition

FairSquare

slide-17
SLIDE 17
slide-18
SLIDE 18

population model decision-making program

slide-19
SLIDE 19

population model decision-making program

slide-20
SLIDE 20
slide-21
SLIDE 21
slide-22
SLIDE 22

“weighted volume”

colRank ethnicity yExp

represent assignments as a region

(LRA formula)

slide-23
SLIDE 23

popModel dec Pr

define popModel(): ethnicity ~ gauss(0,10) colRank ~ gauss(25,10) yExp ~ gauss(10,5) if ethnicity > 10: colRank ← colRank + 5 return colRank, yExp define dec(colRank, yExp): expRank ← yExp – colRank if colRank <= 5: hire ← true elif expRank > -5: hire ← true else: hire ← false return hire

slide-24
SLIDE 24

popModel dec Pr

define popModel(): ethnicity ~ gauss(0,10) colRank ~ gauss(25,10) yExp ~ gauss(10,5) if ethnicity > 10: colRank ← colRank + 5 return colRank, yExp define dec(colRank, yExp): expRank ← yExp – colRank if colRank <= 5: hire ← true elif expRank > -5: hire ← true else: hire ← false return hire

slide-25
SLIDE 25

popModel dec Pr

define popModel(): ethnicity ~ gauss(0,10) colRank ~ gauss(25,10) yExp ~ gauss(10,5) if ethnicity > 10: colRank ← colRank + 5 return colRank, yExp define dec(colRank, yExp): expRank ← yExp – colRank if colRank <= 5: hire ← true elif expRank > -5: hire ← true else: hire ← false return hire

slide-26
SLIDE 26
slide-27
SLIDE 27
slide-28
SLIDE 28
slide-29
SLIDE 29
slide-30
SLIDE 30
slide-31
SLIDE 31
slide-32
SLIDE 32
slide-33
SLIDE 33

Evaluating post

  • Obtain formula for each probability in post
  • Underapproximate the weighted volume of
  • Overapproximate by doing the same for

eventually, approximations are good enough

slide-34
SLIDE 34

“Live” demo

slide-35
SLIDE 35

Case Study

Income Dataset [1] Bayesian Networks Group Fairness for Women Decision Trees, Linear SVMs, RLU Neural Nets Population Model Fairness Definition Decision Program

[1] https://archive.ics.uci.edu/ml/datasets/adult

slide-36
SLIDE 36

[1] Gehr et al. CAV 2016 [2] Sankaranarayanan et al. PLDI 2013

Total FairSquare PSI [1] VolComp [2] 2 4 6 8 10 12 14

Fairness Verification Problems

NN SVM DT

slide-37
SLIDE 37

Population Model Decision Program Unfairness proof Fairness proof Fairness Definition

FairSquare

slide-38
SLIDE 38
slide-39
SLIDE 39