(De)Constructing Bias on Skin Lesion Datasets A. Bissoto, M. - - PowerPoint PPT Presentation

▶
de constructing bias on skin lesion datasets
SMART_READER_LITE
LIVE PREVIEW

(De)Constructing Bias on Skin Lesion Datasets A. Bissoto, M. - - PowerPoint PPT Presentation

Jul 14, 2023 35 likes •302 views

(De)Constructing Bias on Skin Lesion Datasets A. Bissoto, M. Fornaciali, E. Valle, S. Avila RECOD Lab., IC, University of Campinas (UNICAMP) RECOD Lab., DCA, FEEC, University of Campinas (UNICAMP) ISIC Workshop @ CVPR 2019 RECOD

slide-1
SLIDE 1

(De)Constructing Bias on Skin Lesion Datasets

  • A. Bissoto¹, M. Fornaciali², E. Valle², S. Avila¹

¹RECOD Lab., IC, University of Campinas (UNICAMP) ²RECOD Lab., DCA, FEEC, University of Campinas (UNICAMP) ISIC Workshop @ CVPR 2019

slide-2
SLIDE 2

2

RECOD Titans

melanoma research 5 years 2014–2019

slide-3
SLIDE 3

h t t p : / / w w w . t

  • d

a y i f

  • u

n d

  • u

t . c

  • m

/ i n d e x . p h p / 2 1 3 / 1 2 / a n t i

  • t

a n k

  • d
  • g

s

  • w
  • r

l d

  • w

a r

  • i

i / 3

slide-4
SLIDE 4

Bias

Reproduced from: “Unbiased Look at Dataset Bias”, Torralba et al. (2011) 4

slide-5
SLIDE 5

Reproduced from: “An Overview of Melanoma Detection in Dermoscopy Images Using Image Processing and Machine Learning”, Mishra et al. (2016)

Confounders on Skin Lesion Datasets

5

Vignetting (dark borders) Staining Color markers Rulers

slide-6
SLIDE 6

Inflate Performance Spurious Correlations Destruction Experiments

Bias

Play Down Performance Legitimate (Overlooked?) Correlations Construction Experiments

6

slide-7
SLIDE 7

➔ Educational ➔ Rich Metadata ➔ Clinical and dermoscopic images for every case ➔ Clinical data (location, diameter, elevation) ➔ Metadata for dermoscopic features. ➔ Large ➔ Diverse ➔ Different sources, different devices ➔ Segmentation masks for lesion (large subset) ➔ Segmentation masks for dermoscopic features (small subset).

Datasets

Atlas of Dermoscopy ISIC Archive

7

slide-8
SLIDE 8

Destruction Experiments

8

slide-9
SLIDE 9

Traditional

9

slide-10
SLIDE 10

Traditional Only Skin

10

slide-11
SLIDE 11

Traditional Only Skin Bbox

11

slide-12
SLIDE 12

Traditional Only skin Bbox Bbox70

slide-13
SLIDE 13

13

Destruction Experiments

slide-14
SLIDE 14

14

Destruction Experiments

slide-15
SLIDE 15

Destruction Experiments

Performance of machine learning with all cogent information removed on ISIC Archive: 71% AUC

15

slide-16
SLIDE 16

Destruction Experiments

Performance of machine learning with all cogent information removed on ISIC Archive: 71% AUC

¹“The Melanoma Classification Benchmark”, Brinker et al. (2019) 16

Performance of 157 dermatologists¹ on ISIC Archive: 67% AUC

slide-17
SLIDE 17

Construction Experiments

17

slide-18
SLIDE 18

Traditional b) Grayscale Attributes c) RGB Attributes d) Traditional + Grayscale Attributes

18

slide-19
SLIDE 19

Traditional Grayscale Attributes c) RGB Attributes d) Traditional + Grayscale Attributes

19

slide-20
SLIDE 20

Traditional Grayscale Attributes RGB Attributes d) Traditional + Grayscale Attributes

20

slide-21
SLIDE 21

Traditional Grayscale Attributes RGB Attributes Traditional + Grayscale Attributes

21

slide-22
SLIDE 22

Construction Experiments

22

slide-23
SLIDE 23

Machine learning results results are probably optimistic Feeding the model with relevant dermoscopic attributes is worse than feeding it with “only skin” or “bbox” sets Solving the bias problem is critical for deploying automated skin lesion analysis to the real world

Conclusions

23

slide-24
SLIDE 24

24

Team

24

slide-25
SLIDE 25

Acknowledgments

REC D

reasoning for complex data

25

slide-26
SLIDE 26

Thanks!

ISIC Workshop @ CVPR 2019