Dermatologist-level classification of skin cancer with deep neural - - PowerPoint PPT Presentation

▶
dermatologist level classification of skin cancer with
SMART_READER_LITE
LIVE PREVIEW

Dermatologist-level classification of skin cancer with deep neural - - PowerPoint PPT Presentation

Jun 21, 2023 461 likes •949 views

Dermatologist-level classification of skin cancer with deep neural networks Enhancing the Expert Andre Esteva PI: Sebastian Thrun Stanford University 1 How can technology assist a human? 2 3 4 5 How can AI assist a dermatologist? 6

slide-1
SLIDE 1

Dermatologist-level classification of skin cancer with deep neural networks

Enhancing the Expert

Andre Esteva PI: Sebastian Thrun Stanford University

1

slide-2
SLIDE 2

How can technology assist a human?

2

slide-3
SLIDE 3

3

slide-4
SLIDE 4

4

slide-5
SLIDE 5

5

slide-6
SLIDE 6

How can AI assist a dermatologist?

6

slide-7
SLIDE 7

Skin Cancer

7

slide-8
SLIDE 8
  • 5.4M cases of non-melanoma skin cancer each year in US

Skin Cancer

8

slide-9
SLIDE 9
  • 5.4M cases of non-melanoma skin cancer each year in US
  • 20% of Americans will get skin cancer

Skin Cancer

9

slide-10
SLIDE 10
  • 5.4M cases of non-melanoma skin cancer each year in US
  • 20% of Americans will get skin cancer
  • Actinic Keratosis (pre-cancer) affects 58 million Americans

Skin Cancer

10

slide-11
SLIDE 11
  • 5.4M cases of non-melanoma skin cancer each year in US
  • 20% of Americans will get skin cancer
  • Actinic Keratosis (pre-cancer) affects 58 million Americans
  • 76,000 melanomas each year - 10,000 deaths

Skin Cancer

11

slide-12
SLIDE 12
  • 5.4M cases of non-melanoma skin cancer each year in US
  • 20% of Americans will get skin cancer
  • Actinic Keratosis (pre-cancer) affects 58 million Americans
  • 76,000 melanomas each year - 10,000 deaths
  • $8.1B in US annual costs for skin cancer

Skin Cancer

12

slide-13
SLIDE 13
  • 5.4M cases of non-melanoma skin cancer each year in US
  • 20% of Americans will get skin cancer
  • Actinic Keratosis (pre-cancer) affects 58 million Americans
  • 76,000 melanomas each year - 10,000 deaths
  • $8.1B in US annual costs for skin cancer

Skin Cancer

13

Years

100 75 50 25

Survival Probability

1 5 10 15

Stage I Stage II Stage III Stage IV

slide-14
SLIDE 14
  • 5.4M cases of non-melanoma skin cancer each year in US
  • 20% of Americans will get skin cancer
  • Actinic Keratosis (pre-cancer) affects 58 million Americans
  • 76,000 melanomas each year - 10,000 deaths
  • $8.1B in US annual costs for skin cancer

Skin Cancer

Years

100 75 50 25

Survival Probability

1 5 10 15

Stage I Stage II Stage III Stage IV

14

1 2 3 4

slide-15
SLIDE 15

15

slide-16
SLIDE 16

16

slide-17
SLIDE 17

Early detection is critical

17

slide-18
SLIDE 18

6.3 billion smartphones

18

slide-19
SLIDE 19

Skin Cancer Classification

19

slide-20
SLIDE 20

~130,000 images of skin 2000 diseases

Skin Cancer Classification

20

slide-21
SLIDE 21

~130,000 images of skin 2000 diseases

Skin Cancer Classification

21

slide-22
SLIDE 22

Epidermal Lesions Melanocytic Lesions Melanocytic Lesions (Dermoscopy) Benign Malignant

Skin Cancer Classification

22

slide-23
SLIDE 23

Deep Convolutional Neural Network (Inception-v3)

Skin Cancer Classification

23

slide-24
SLIDE 24

Acral-lent. melanoma Amelanotic melanoma Lentigo melanoma ... Blue nevus Halo nevus Mongolian spot …

Training Classes (757) Deep Convolutional Neural Network (Inception-v3) Skin Lesion Image

Skin Cancer Classification

Partitioning Algorithm 24

slide-25
SLIDE 25

Acral-lent. melanoma Amelanotic melanoma Lentigo melanoma ... Blue nevus Halo nevus Mongolian spot …

Training Classes (757) Deep Convolutional Neural Network (Inception-v3) Inference Classes (varies by task)

92% Malignant 8% Benign

Skin Lesion Image

Skin Cancer Classification

Partitioning Algorithm 25

slide-26
SLIDE 26

Skin Cancer Classification

P = 0.1 P = 0.05 P = 0.05 P = 0.1 P = 0.02 P = 0.03 P = 0.05

Training Classes Inference Classes

P = 0.4

26

slide-27
SLIDE 27

Dermatologist-level performance

27

slide-28
SLIDE 28

Validation set

Skin Cancer Classification

28

slide-29
SLIDE 29

Validation set

Classifier Three-way accuracy Dermatologist 1 65.6% Dermatologist 2 66.0% CNN 69.5% CNN - PA 72.0% Disease classes: three-way classification

  • 0. Benign single lesions
  • 1. Malignant single lesions
  • 2. Non-neoplastic lesions

Skin Cancer Classification

29

slide-30
SLIDE 30

Validation set

Classifier Three-way accuracy Dermatologist 1 65.6% Dermatologist 2 66.0% CNN 69.5% CNN - PA 72.0% Classifier Nine-way accuracy Dermatologist 1 53.3% Dermatologist 2 55.0% CNN 48.9% CNN - PA 55.3% Disease classes: nine-way classification

  • 0. Cutaneous lymphoma and lymphoid infiltrates
  • 1. Benign dermal tumors, cysts, sinuses
  • 2. Malignant dermal tumor
  • 3. Benign epidermal tumors, hamartomas, milia, and

growths

  • 4. Malignant and premalignant epidermal tumors
  • 5. Genodermatoses and supernumerary growths
  • 6. Inflammatory conditions
  • 7. Benign melanocytic lesions
  • 8. Malignant Melanoma

Disease classes: three-way classification

  • 0. Benign single lesions
  • 1. Malignant single lesions
  • 2. Non-neoplastic lesions

Skin Cancer Classification

30

slide-31
SLIDE 31

Skin Cancer Classification

Test set

31

slide-32
SLIDE 32

Skin Cancer Classification

Test set: Dermatologist Comparison (376 images)

32

slide-33
SLIDE 33

Sensitivity

Carcinoma: 135 images Specificity Sensitivity

Skin Cancer Classification

Algorithm: AUC = 0.96 Dermatologists (25) Average Dermatologist

Test set: Dermatologist Comparison (376 images)

33

slide-34
SLIDE 34

Sensitivity Sensitivity Sensitivity

Carcinoma: 135 images Melanoma: 130 images Melanoma: 111 dermoscopy images Specificity Specificity Specificity Sensitivity Sensitivity Sensitivity

Skin Cancer Classification

Algorithm: AUC = 0.96 Dermatologists (25) Average Dermatologist Algorithm: AUC = 0.94 Dermatologists (22) Average Dermatologist Algorithm: AUC = 0.91 Dermatologists (21) Average Dermatologist

Test set: Dermatologist Comparison (376 images)

34

slide-35
SLIDE 35

Sensitivity Sensitivity Sensitivity

Carcinoma: 707 images Melanoma: 225 images Melanoma: 1010 dermoscopy images Specificity Specificity Specificity Sensitivity Sensitivity Sensitivity

Skin Cancer Classification

Test set: Total (1942 images)

Algorithm: AUC = 0.96 Algorithm: AUC = 0.96 Algorithm: AUC = 0.94

35

slide-36
SLIDE 36

How does the algorithm work?

36

slide-37
SLIDE 37

37

T-SNE Visualization

Van der Maaten & Hinton, 2008

slide-38
SLIDE 38

Epidermal Benign Epidermal Malignant Melanocytic Benign Melanocytic Malignant

38

Basal Cell Carcinomas Squamous Cell Carcinomas Melanomas Nevi Seborrheic Keratoses

T-SNE Visualization

slide-39
SLIDE 39

Epidermal Benign Epidermal Malignant Melanocytic Benign Melanocytic Malignant Basal Cell Carcinomas Squamous Cell Carcinomas Melanomas Nevi Seborrheic Keratoses

39

T-SNE Visualization

slide-40
SLIDE 40

40

What is the network fixating on?

slide-41
SLIDE 41

Malignant Melanocytic Lesion

41

What is the network fixating on?

Simonyan, Zisserman, 2014

slide-42
SLIDE 42

Malignant Melanocytic Lesion Malignant Epidermal Lesion Malignant Dermal Lesion Benign Melanocytic Lesion Benign Epidermal Lesion Benign Dermal Lesion Inflammatory Condition Genodermatosis Cutaneous Lymphoma 42

What is the network fixating on?

slide-43
SLIDE 43

43

What does the network misclassify?

slide-44
SLIDE 44

CNN Dermatologist 1 Dermatologist 2 True Label Predicted Label Predicted Label Predicted Label 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 1

44

What does the network misclassify?

slide-45
SLIDE 45

Dermatologist-level Classification

  • f Skin Cancer with Deep Neural

Networks

Andre Esteva*, Brett Kuprel*, Rob Novoa, Justin Ko, Susan Swetter, Helen Blau, Sebastian Thrun Nature, 2017 (Equal contribution authors*)

45

slide-46
SLIDE 46

How can AI assist a dermatologist?

46

slide-47
SLIDE 47

Community

47

slide-48
SLIDE 48

Questions?

esteva@cs.stanford.edu @andreesteva cs.stanford.edu/people/esteva

48