The European Commissions science and knowledge service Joint - - PowerPoint PPT Presentation

▶

Oct 30, 2022 155 likes •669 views

The European Commissions science and knowledge service Joint Research Centre Why machine learning may lead to unfairness Songl Tolan 1 , Marius Miron 1 , Emilia Gomez 1,2 , Carlos Castillo 2 1 European Commissions Joint Research Centre 2

SLIDE 1

The European Commission’s science and knowledge service

Joint Research Centre

SLIDE 2

Why machine learning may lead to unfairness

Songül Tolan1, Marius Miron1, Emilia Gomez1,2, Carlos Castillo2

1European Commission’s Joint Research Centre 2Universitat Pompeu Fabra

SLIDE 3

Machine learning for decision making

SLIDE 4

The criminal justice case

Trade-off: predictive performance vs fairness

SLIDE 5

Criminal recidivism

SLIDE 6

Criminal recidivism prediction

Human expert Decision / Sentence Prisoner

SLIDE 7

Criminal recidivism prediction

Human expert Decision / Sentence Outcome Prisoner

SLIDE 8

Criminal recidivism prediction

Human expert Decision / Sentence Outcome Prisoner

SLIDE 9

Criminal recidivism prediction

Machine learning model Prediction Outcome Features

SLIDE 10

Criminal recidivism prediction

Age at crime Sex Nationality Previous number of crimes Sentence Year of crime Probation Examples of static features:

SLIDE 11

Fairness

A decision is fair if it does not discriminate against people based on their membership to a protected group

SLIDE 12

Fairness

Age at crime Sex Nationality Previous number of crimes Sentence Year of crime Probation Example of protected features:

SLIDE 13

Measuring unfairness

Machine learning model Prediction Outcome Features Nationality Sex

SLIDE 14

Measuring unfairness

False positive False negative Prediction Outcome

SLIDE 15

False negative rate = Miss rate

Σ Σ

SLIDE 16

False positive rate = False alarm rate

Σ Σ

SLIDE 17

Group fairness - sex

Σ Σ

sex=Male sex=Male

SLIDE 18

False negative rate disparity

How likely it is for a member of a group to be wrongfully labeled as non-recidivist. FNRdisparity= FNRfemale FNRmale

SLIDE 19

Headache?

SLIDE 20

Too complicated?

The fairness in machine learning literature comprises at least 21 disparity metrics.

SLIDE 21

Juvenile recidivism

SLIDE 22

Structured Assessment of Violence Risk in Youth (SAVRY)

Risk assessment tools

high degree of involvement from human experts
open and interpretable (in comparison with COMPAS)
24 risk factors scored low, medium or high

SLIDE 23

SAVRY

Early violence Self-harm history Home violence Poor school achievement Stress and poor coping Substance abuse Criminal parent/caregiver Examples of SAVRY features:

SLIDE 24

Criminal recidivism prediction

Expert Final expert evaluation Outcome SAVRY features

Σ

SAVRY sum

SLIDE 25

Static ML

Machine learning model Prediction Outcome Features

SLIDE 26

SAVRY ML

Machine learning model Prediction Outcome SAVRY features

SLIDE 27

Static + SAVRY ML

Machine learning model Prediction Outcome Features

SLIDE 28

Juvenile offenders in Catalonia1

Dataset

855 people
crimes between 2002 -2010, release in 2010
age at crime between 12 and 17 years old
status followed up on 2013 and 2015

1. Open data: http://cejfe.gencat.cat/en/recerca/opendata/jjuvenil/reincidencia-justicia-menors/index.html

SLIDE 29

Training a set of ML methods

Experimental setup

logistic regression (logit), multi-layer perceptron (mlp),

support vector machines (lsvm), k-nearest neighbors (knn), random forest (rf), naive bayes (nb)

k-fold cross validation with k=10 (10% test, 10% validation,

80% training)

we run 50 different experiments with different initial conditions
we compute feature importance with LIME1

1. LIME https://github.com/marcotcr/lime

SLIDE 30

Predictive performance - AUC ROC

SLIDE 31

Results, predictive performance AUC

SAVRY Sum has 0.64 AUC Expert has 0.66 AUC

SLIDE 32

Results: disparity, sex

False alarm rates Miss rates

SLIDE 33

Results: disparity, sex

False alarm rates Miss rates

SLIDE 34

Results: disparity, sex

False alarm rates Miss rates

SLIDE 35

Results: disparity, sex

False alarm rates Miss rates

SLIDE 36

Results: disparity, sex

False alarm rates Miss rates

SLIDE 37

Results: disparity, sex

False alarm rates Miss rates

SLIDE 38

Results: disparity, nationality

False alarm rates Miss rates

SLIDE 39

Results: disparity, nationality

False alarm rates Miss rates

SLIDE 40

Results: disparity, nationality

False alarm rates Miss rates

SLIDE 41

Results: disparity, nationality

False alarm rates Miss rates

SLIDE 42

Results: disparity, nationality

False alarm rates Miss rates

SLIDE 43

Results: feature importance for logit

SLIDE 44

Results: feature importance for mlp

SLIDE 45

Results: difference in base rates (prevalence)

SLIDE 46

Results: difference in base rates

SLIDE 47

Results: difference in base rates

SLIDE 48

Conclusions

ML models have better predictive performance
ML models tend to discriminate more
static features outweigh SAVRY features as importance
preliminary study: the cause may be in the data (base rates)

SLIDE 49

We propose a methodology and a ML framework1

Contributions

to easily train ML models on tabular data (csv files)
to evaluate these models in terms of predictive

performance and fairness

to connect to interpretability frameworks
to reproduce with ease results and research

1. Open framework: https://gitlab.com/HUMAINT/humaint-fatml

SLIDE 50

Thank you!

Any questions?

You can find me at @nkundiushuti & marius.miron@ec.europa.eu & mariusmiron.com