Dermoscopy Image? Catarina Barata and Carlos Santiago Computer and - - PowerPoint PPT Presentation

▶

Sep 16, 2023 760 likes •1.06k views

How Important Is Each Dermoscopy Image? Catarina Barata and Carlos Santiago Computer and Robot Vision Lab Motivation Dermoscopy Datasets 100000 10000 1000 100 2013 2014 2015 2016 2017 2018 2019 Computer and Robot Vision Lab 2

SLIDE 1

Computer and Robot Vision Lab

How Important Is Each Dermoscopy Image?

Catarina Barata and Carlos Santiago

SLIDE 2

Computer and Robot Vision Lab

Motivation

100 1000 10000 100000 2013 2014 2015 2016 2017 2018 2019

Dermoscopy Datasets

SLIDE 3

Computer and Robot Vision Lab

Motivation

De Deep Ne Networks works Li Like ke Da Data ta

SLIDE 4

Computer and Robot Vision Lab

Motivation

Class Distribution

50 100 150 200 2013 2014 2015 2016 2017 2018 2019

Dermoscopy Datasets

MEL NV

SLIDE 5

Computer and Robot Vision Lab

Motivation

200 400 600 800 1000 2013 2014 2015 2016 2017 2018 2019

Dermoscopy Datasets

MEL NV

Class Distribution

SLIDE 6

Computer and Robot Vision Lab

Motivation

500 1000 1500 2000 2500 2013 2014 2015 2016 2017 2018 2019

Dermoscopy Datasets

BKL MEL NV

Class Distribution

SLIDE 7

Computer and Robot Vision Lab

Motivation

2000 4000 6000 8000 10000 2013 2014 2015 2016 2017 2018 2019

Dermoscopy Datasets

VASC DF BCC AKIEC BKL MEL NV

Class Distribution

SLIDE 8

Computer and Robot Vision Lab

Motivation

5000 10000 15000 20000 2013 2014 2015 2016 2017 2018 2019

Dermoscopy Datasets

SCC VASC DF BCC AKIEC BKL MEL NV

Class Distribution

SLIDE 9

Computer and Robot Vision Lab

Motivation

Why is this a problem?

Network bias
Poor Generalization

Class # Samples Deep Net Recall (%) NV 6741 95 MEL 1119 66 BKL 1101 77 AKIEC 331 45 BCC 517 88 DF 116 43 VASC 143 68

SLIDE 10

Computer and Robot Vision Lab

Me Li Likes kes Bala alanced nced Data ta More…

SLIDE 11

Computer and Robot Vision Lab

Challenges

Deal with class imbalance
Not all classes are equally hard
Are all samples equally important?

Class # Samples Deep Net Recall (%) NV 6741 95 MEL 1119 66 BKL 1101 77 AKIEC 331 45 BCC 517 88 DF 116 43 VASC 143 68

SLIDE 12

Computer and Robot Vision Lab

Goal

How to make the most of the available data?

Data Augmentation
Importance Sampling
Sample Weighting

SLIDE 13

Computer and Robot Vision Lab

Goal

How to make the most of the available data?

Data Augmentation
Importance Sampling
Sample Weighting

SLIDE 14

Computer and Robot Vision Lab

Sample Weighting

… …

DNN

Feedforward and Compute Sample Loss ℓ𝑘

Loss

Backpropagate and Update Model Parameters

ℒ = 1 𝑁 ෍

𝑘 𝑁

ℓ𝑘 Batch Samples

SLIDE 15

Computer and Robot Vision Lab

Sample Weighting

… …

DNN

Feedforward and Compute Sample Loss ℓ𝑘

Loss

Backpropagate and Update Model Parameters

ℒ = 1 𝑁 ෍

𝑘 𝑁

ℓ𝑘 Batch Samples

Cross Entropy Loss (CEL) Focal Loss (CEL)

SLIDE 16

Computer and Robot Vision Lab

Sample Weighting

… …

DNN

Feedforward and Compute Sample Loss ℓ𝑘

Batch Samples Loss

Backpropagate and Update Model Parameters

ℒ = 1 𝑁 ෍

𝑘 𝑁

𝒙𝒌 ℓ𝑘 Sample Weights

𝒙𝟐 𝒙𝟑 𝒙𝟒 𝒙𝟓 𝒙𝟔 𝒙𝟕 𝒙𝟖 𝒙𝟗 𝒙𝟘 𝒙𝟐𝟏 𝒙𝟐𝟐 𝒙𝟐𝟑 𝒙𝟐𝟒 𝒙𝟐𝟓 𝒙𝟐𝟔 𝒙𝟐𝟕

SLIDE 17

Computer and Robot Vision Lab

Weighting Strategies

Class-Balanced Losses

– Class-balanced (CB)[1]: 𝑥

𝑘 = 𝑂

𝑂𝑧𝑘 – Effective Number of Samples (ES)[2]: 𝑥

𝑘 = 1−𝛾 1−𝛾

𝑂𝑧𝑘, 𝛾 =

𝑂−1 𝑂

[1] Provost, Machine Learning From Imbalanced Datasets 101, AAAI 2000 [2] Cui et al., Class-balanced Loss Based on Effective Number of Samples, CVPR 2019

SLIDE 18

Computer and Robot Vision Lab

Weighting Strategies

Class-Balanced Losses

– Class-balanced (CB): 𝑥

𝑘 = 𝑂

𝑂𝑧𝑘 – Effective Number of Samples (ES): 𝑥

𝑘 = 1−𝛾 1−𝛾

𝑂𝑧𝑘, 𝛾 =

𝑂−1 𝑂

0.08 0.08 0.51 1.08 0.50 0.50 0.08 0.08 0.51 0.51 1.08 1.69 0.08 0.50 3.90 4.81 0.11 0.11 0.52 1.08 0.51 0.51 0.11 0.11 0.52 0.52 1.08 1.67 0.11 0.51 3.82 4.71

SLIDE 19

Computer and Robot Vision Lab

Weighting Strategies

Curriculum Learning

arg min

𝒙

1 𝑁 ෍

𝑘 𝑁

𝑥

𝑘 ℓ𝑘 + 𝐻 𝒙; 𝜇

SLIDE 20

Computer and Robot Vision Lab

Weighting Strategies

Curriculum Learning

– Self-paced Learning (SPL)[1]: 𝐻 𝒙; 𝜇 = −𝜇 𝒙 1 – Online Hard Example Mining (OHEM)[2]: 𝐻 𝒙; 𝜇 = +𝜇 𝒙 1

[1] Kumar et al., Self-Paced Learning for Latent Variable Models, NeurIPS 2010 [2] Shrivastava et al., Training Region-based Object Detectors with Online Hard Example Mining, CVPR 2016

SLIDE 21

Computer and Robot Vision Lab

Weighting Strategies

Curriculum Learning

– Self-paced Learning (SPL): 𝐻 𝒙; 𝜇 = −𝜇 𝒙 1 – Online Hard Example Mining (OHEM): 𝐻 𝒙; 𝜇 = +𝜇 𝒙 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

ℓ𝑘< 𝜇 ℓ𝑘> 𝜇

SLIDE 22

Computer and Robot Vision Lab

Experimental Setup

DNN Architectures

– Flat Classifier (VGG-16) – Hierarchical Classifier[1]

Dataset

– ISIC 2018

Performance Metrics

– Recall – Precision – F1-Score

[1] Barata et al., Explainable Skin Lesion Diagnosis Using Taxonomies, Pattern Recognition 2020 [2] Woo et al., CBAM: Convolutional Block Attention Module, ECCV 2018

– Accuracy – Balanced Accuracy + CBAM[2]

SLIDE 23

Computer and Robot Vision Lab

Results

SLIDE 24

Computer and Robot Vision Lab

Results

SLIDE 25

Computer and Robot Vision Lab

Results

SLIDE 26

Computer and Robot Vision Lab

Results

SLIDE 27

Computer and Robot Vision Lab

Results

SLIDE 28

Computer and Robot Vision Lab

Conclusions

Weighting strategies significantly affect the performance of a DNN
Some weighting schemes may induce bias
Features learned by DNNs change according to the learning strategy
OHEM achieves the best overall performance

SLIDE 29

Computer and Robot Vision Lab

Thank You!

ana.c.fidalgo.barata@tecnico.ulisboa.pt