SLIDE 1
Direct Uncertainty Prediction for Medical Second Opinions Maithra - - PowerPoint PPT Presentation
Direct Uncertainty Prediction for Medical Second Opinions Maithra - - PowerPoint PPT Presentation
Direct Uncertainty Prediction for Medical Second Opinions Maithra Raghu , Katy Blumer, Rory Sayres, Ziad Obermeyer, Sendhil Mullainathan, Jon Kleinberg Poster #246 Poster #246: Direct Uncertainty Prediction for Medical Second Opinions Human
SLIDE 2
SLIDE 3
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Human Expert Disagreements
SLIDE 4
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Doctor Disagreements
SLIDE 5
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Doctor Disagreements
Diagnostic Concordance Amongst Pathologists Interpreting Breast Biopsy Specimens, UW School of Medicine, JAMA, 2015
SLIDE 6
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Doctor Disagreements
Diagnostic Concordance Amongst Pathologists Interpreting Breast Biopsy Specimens, UW School of Medicine, JAMA, 2015
- Agreement between individual
pathologist grade and a panel consensus score on ~240 breast biopsies, 6900 individual case diagnoses
- 25% disagreement between
pathologists and consensus
SLIDE 7
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Doctor Disagreements
Grade 3: Moderate Diabetic Retinopathy Grade 2: Mild Diabetic Retinopathy
Ophthalmology: Diagnosis from Fundus Photographs
SLIDE 8
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
The Source of Disagreements
Grade 3: Moderate Diabetic Retinopathy Grade 2: Mild Diabetic Retinopathy
Random Mistakes? Ophthalmology: Diagnosis from Fundus Photographs
SLIDE 9
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
The Source of Disagreements
Diagnosis Type Diagnosis Type Fraction of votes Patient 1 Patient 2
SLIDE 10
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
ML for Doctor Disagreement Prediction
Given input (image) x, predict the amount of disagreement. Flag patients for medical second opinions.
SLIDE 11
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
ML for Doctor Disagreement Prediction
Given input (image) x, predict the amount of disagreement. Flag patients for medical second opinions. Training data: xi, with multiple labels y(i)
1,...,y(i) k (different
doctors) I.e. (xi, pi), pi grade distribution, target U(pi) (e.g. U() = entropy)
SLIDE 12
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
ML for Doctor Disagreement Prediction
Given input (image) x, predict the amount of disagreement. Flag patients for medical second opinions. Training data: xi, with multiple labels y(i)
1,...,y(i) k (different
doctors) I.e. (xi, pi), pi grade distribution, target U(pi) (e.g. U() = entropy) 1) Uncertainty Via Classification (UVC): (i) train classifier on empirical distribution of labels (xi, pi) (ii) postprocess with U() 2) Direct Uncertainty Prediction (DUP): directly predict scalar uncertainty score (xi, U(pi))
SLIDE 13
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Direct Uncertainty Prediction
SLIDE 14
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Direct Uncertainty Prediction
Hidden information:
61 (age) F (gender) medical history
SLIDE 15
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Direct Uncertainty Prediction
Theorem: DUP gives an unbiased estimate of true uncertainty
SLIDE 16
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Empirical Results: Synthetic Examples
Mixture of Gaussians SVHN and CIFAR-10: Image Blurring Application
SLIDE 17
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Large Scale Medical Application
Diabetic Retinopathy (DR) 5 class scale: 1 None 2 Mild 3 Moderate 4 Severe 5 Proliferative
Referable
SLIDE 18
Poster #246: Direct Uncertainty Prediction for Medical Second Opinions
Large Scale Medical Application
SLIDE 19