Detecting skin cancer with an iPhone Tory Jarmain CEO & - - PowerPoint PPT Presentation

▶
detecting skin cancer with an iphone
SMART_READER_LITE
LIVE PREVIEW

Detecting skin cancer with an iPhone Tory Jarmain CEO & - - PowerPoint PPT Presentation

Dec 09, 2023 161 likes •817 views

Detecting skin cancer with an iPhone Tory Jarmain CEO & Co-Founder Skin disorders are prevalent in 43% of outpatient visits Source: Why do patients visit their doctors? Assessing the most prevalent conditions in a defined US

slide-1
SLIDE 1

Detecting skin cancer with an iPhone


 Tory Jarmain

CEO & Co-Founder

slide-2
SLIDE 2

Skin disorders are prevalent in 43% of outpatient visits

Source: “Why do patients visit their doctors? Assessing the most prevalent conditions in a defined US population.“ Stauver, et al.

slide-3
SLIDE 3

PREVALENCE DISEASE GROUP 0-18 19-29 30-49 50-64 65+ All Skin disorders 33% 38% 41% 50% 66% 43% Arthritis and joint disorders 14% 25% 35% 50% 63% 34% Back problems 6% 20% 29% 34% 44% 24% Lipid metabolism disorders 0% 3% 19% 49% 70% 22% Upper respiratory disease 24% 19% 22% 22% 23% 22%

Source: “Why do patients visit their doctors? Assessing the most prevalent conditions in a defined US population.“ Stauver, et al.

slide-4
SLIDE 4

Source: American Academy of Dermatology

1 in 5 
 Americans will develop skin cancer in their lifetime

slide-5
SLIDE 5

Source: American Academy of Dermatology

1 in 3 
 cancer diagnoses is skin cancer

slide-6
SLIDE 6

Source: “Deep Networks for Early Stage Skin Disease and Skin Cancer Classification.” Esteva, et al.

Dermatologists have 52% accuracy classifying skin lesions with non-dermoscopic imagery

slide-7
SLIDE 7

MELANOMA BENIGN

slide-8
SLIDE 8

Source: American Academy of Dermatology

slide-9
SLIDE 9

The average wait time to see a dermatologist is 1 month in the U.S. and 3-6 months in much of the world. In that time, skin disorders can become life threatening.

Source: Merritt Hawkins

slide-10
SLIDE 10

Use deep learning to triage 
 skin disorder cases OUR IDEA

slide-11
SLIDE 11

Snap a photo, detect a skin disorder and see visually similar cases SOLUTION

slide-12
SLIDE 12

View results Learn more Snap a photo

D

slide-13
SLIDE 13
slide-14
SLIDE 14

MELANOMA DETECTION

slide-15
SLIDE 15

Assist physicians in binary classification of skin lesions SCOPE

slide-16
SLIDE 16

Achieve or improve upon state-of-the-art results for skin lesion segmentation and classification. Measure the impact of segmentation on the accuracy of the classifier.

GOALS

slide-17
SLIDE 17

Segmenting skin lesions improves the accuracy and sensitivity

  • f a deep learning

classification model

HYPOTHESIS

slide-18
SLIDE 18

Dermoscopic images may contain artifacts, be low contrast, and contain multiple lesions CHALLENGES

slide-19
SLIDE 19
slide-20
SLIDE 20
slide-21
SLIDE 21
slide-22
SLIDE 22
slide-23
SLIDE 23

CONVOLUTIONAL NEURAL NETWORKS

slide-24
SLIDE 24

CONVOLUTION LAYER

Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

slide-25
SLIDE 25

CONVOLUTION LAYER

Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

slide-26
SLIDE 26

CONVOLUTION LAYER

Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

slide-27
SLIDE 27

ACTIVATION LAYER

Source: “Skin Lesion Detection From Dermoscopic Images Using Convolutional Neural Networks”

slide-28
SLIDE 28

POOLING LAYER

Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

slide-29
SLIDE 29

FULLY CONNECTED LAYER

Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

slide-30
SLIDE 30

MAIN SCHEME

Source: LeCun, et al.

slide-31
SLIDE 31

MAIN SCHEME

Source: LeCun, et al.

slide-32
SLIDE 32

MAIN SCHEME

Source: LeCun, et al.

slide-33
SLIDE 33

MAIN SCHEME

Source: LeCun, et al.

slide-34
SLIDE 34

Class Benign Malignant Total Training subset 727 173 900 Test subset 304 75 379

slide-35
SLIDE 35

DATA AUGMENTATION

Original image

Random transformations

slide-36
SLIDE 36

METHOD SCHEME

slide-37
SLIDE 37

Source: “Convolutional Networks for Biomedical Image Segmentation.” Olaf Ronneberger, et al. Original skin lesion Binary mask

SEGMENTATION

slide-38
SLIDE 38

CLASSIFICATION WITH VGG-16

  • Five convolutional blocks
  • 3 x 3 receptive field
  • ReLU as Activation Function
  • Max-Pooling
  • Classifier block
  • 3 fully-connected layers at

the top of the network

slide-39
SLIDE 39

Pretrain a ConvNet on a very large dataset (e.g. ImageNet, which contains 1.2 million images with 1000 categories), and then use the ConvNet either as an initialization or a fixed feature extractor for the task of interest.

TRANSFER LEARNING

slide-40
SLIDE 40

Train this Freeze these

slide-41
SLIDE 41

EVALUATION

slide-42
SLIDE 42

SEGMENTATION EVALUATION

Rank Participant Jaccard 
 Index Dice 
 Coefficient Accuracy Sensitivity Specificity 1 Adrià Romero Lopez 0.918 0.869 0.918 0.930 0.954 1 Urko Sanchez 0.843 0.910 0.953 0.910 0.965 2 Lequan Yu 0.829 0.897 0.949 0.911 0.957 3 Mahmudur Rahman 0.822 0.895 0.952 0.880 0.969 Ground truth Mask obtained 1 MIDDLE Group

1

slide-43
SLIDE 43
slide-44
SLIDE 44
slide-45
SLIDE 45
slide-46
SLIDE 46

CLASSIFICATION EVALUATION

slide-47
SLIDE 47

Model Accuracy Loss Sensitivity Precision Unaltered lesion classifier 0.847 0.472 0.824 0.952 Perfectly segmented 
 lesion classifier 0.840 0.496 0.865 0.962 Automatically segmented lesion classifier 0.817 0.514 0.892 0.968

CLASSIFICATION EVALUATION

slide-48
SLIDE 48

Model Accuracy Loss Sensitivity Precision Unaltered lesion classifier 0.847 0.472 0.824 0.952 Perfectly segmented 
 lesion classifier 0.840 0.496 0.865 0.962 Automatically segmented lesion classifier 0.817 0.514 0.892 0.968

CLASSIFICATION EVALUATION

With segmentation

  • Accuracy decreases
  • Loss increases
slide-49
SLIDE 49

Model Accuracy Loss Sensitivity Precision Unaltered lesion classifier 0.847 0.472 0.824 0.952 Perfectly segmented 
 lesion classifier 0.840 0.496 0.865 0.962 Automatically segmented lesion classifier 0.817 0.514 0.892 0.968

CLASSIFICATION EVALUATION

With segmentation

  • Sensitivity increases
  • Precision increases
slide-50
SLIDE 50

SENSITIVITY

The most important metric in medical settings. 
 By missing a true melanoma case (False Negative) the model would fail in early

  • diagnosis. It is better to raise a False Positive

than to create a False Negative.

slide-51
SLIDE 51

SENSITIVITY

number of true positives number of true positives + number of false negatives Sensitivity =

slide-52
SLIDE 52

Model Accuracy Loss Sensitivity Precision Unaltered lesion classifier 0.847 0.472 0.824 0.952 Perfectly segmented 
 lesion classifier 0.840 0.496 0.865 0.962 Automatically segmented lesion classifier 0.817 0.514 0.892 0.968

CLASSIFICATION EVALUATION

The automatically segmented classifier performs best

slide-53
SLIDE 53

CONFUSION MATRICES

Unaltered Classifier

Perfectly Segmented Classifier

Automatically Segmented Classifier

False Negatives descending

slide-54
SLIDE 54

23-WAY CLASSIFICATION

slide-55
SLIDE 55

23,000 images 600 diseases

Sources: “A Deep Learning Approach to Universal Skin Disease Classification.” Liao, Haofu. “Deep Networks for Early Stage Skin Disease and Skin Cancer Classification.“ Esteva, et al.

  • Acne and rosacea
  • Malignant lesions
  • Atopic dermatitis
  • Bullous disease
  • Bacterial infections
  • Eczema
  • Exanthems & drug eruptions
  • Hair diseases
  • STDs
  • Pigmentation disorders
  • Connective tissue diseases
  • Melanoma, nevi & moles
  • Nail diseases
  • Contact dermatitis
  • Psoriasis & lichen planus
  • Infestations & bites
  • Benign tumors
  • Systemic disease
  • Fungal infections
  • Urticaria
  • Vascular tumors
  • Vasculitis
  • Viral infections
slide-56
SLIDE 56

Deep Residual Networks

slide-57
SLIDE 57

Source: “Deep Residual Networks: Deep Learning Gets Way Deeper.” He, Kaiming.

slide-58
SLIDE 58

Source: “Deep Residual Networks: Deep Learning Gets Way Deeper.” He, Kaiming.

slide-59
SLIDE 59

Source: “Deep Residual Networks: Deep Learning Gets Way Deeper.” He, Kaiming.

slide-60
SLIDE 60

Source: “Deep Residual Learning for Image Recognition.” Kaiming, et al.

RESIDUAL LEARNING

slide-61
SLIDE 61

23-WAY CLASSIFICATION RESULTS

slide-62
SLIDE 62

Accuracy Best in paper Triage Top-1 73.1% 76.1% Top-5 91.0% 92.4% Accuracy Best in paper Triage Top-1 60.0% 64.8% Top-5 80.3% 80.5%

“A Deep Learning Approach to Universal Skin Disease Classification” Liao, Haofu. “Deep Networks for Early Stage Skin Disease and Skin Cancer Classification” Esteva, et al.

slide-63
SLIDE 63

FUTURE WORK

slide-64
SLIDE 64

THANK YOU

slide-65
SLIDE 65

tory@triage.com