SLIDE 1
ICIP’18 Tutorial on Interpretable Deep Learning 2
From ML Successes to Applications ICIP18 Tutorial on Interpretable - - PowerPoint PPT Presentation
From ML Successes to Applications ICIP18 Tutorial on Interpretable - - PowerPoint PPT Presentation
From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box Models Huge volumes of data Solve task Computing power Deep Neural Network Information (implicit) ICIP18 Tutorial on Interpretable Deep
SLIDE 2
SLIDE 3
ICIP’18 Tutorial on Interpretable Deep Learning
3
Computing power Deep Neural Network Information (implicit) Solve task Huge volumes of data
Black Box Models
SLIDE 4
ICIP’18 Tutorial on Interpretable Deep Learning
4
Black Box Models
SLIDE 5
Why interpretability ?
SLIDE 6
ICIP’18 Tutorial on Interpretable Deep Learning
6
Why Interpretability ?
verify system understand weaknesses legal aspects learn new things from data We ¡need ¡interpretability ¡in ¡order ¡to:
SLIDE 7
ICIP’18 Tutorial on Interpretable Deep Learning 7
Why Interpretability ?
Wrong decisions can be costly and dangerous 1) Verify that classifier works as expected
“Autonomous car crashes, because it wrongly recognizes …” “AI medical diagnosis system misclassifies patient’s disease …”
SLIDE 8
ICIP’18 Tutorial on Interpretable Deep Learning
2) Understand weaknesses & improve classifier
8
Why Interpretability ?
Generalization error Generalization error + human experience
SLIDE 9
ICIP’18 Tutorial on Interpretable Deep Learning
“It's not a human move. I've never seen a human play this move.” (Fan Hui) 3) Learn new things from the learning machine Old promise: “Learn about the human brain.”
9
Why Interpretability ?
SLIDE 10
ICIP’18 Tutorial on Interpretable Deep Learning 10
4) Interpretability in the sciences Learn about the physical / biological / chemical mechanisms. (e.g. find genes linked to cancer, identify binding sites …)
Why Interpretability ?
SLIDE 11
ICIP’18 Tutorial on Interpretable Deep Learning
European Union’s new General Data Protection Regulation
11
5) Compliance to legislation “right to explanation” “With interpretability we can ensure that ML models work in compliance to proposed legislation.” Retain human decision in order to assign responsibility.
Why Interpretability ?
SLIDE 12
ICIP’18 Tutorial on Interpretable Deep Learning 12
Example: Autonomous Driving
SLIDE 13
ICIP’18 Tutorial on Interpretable Deep Learning 13
Example: Medical Diagnosis
SLIDE 14
ICIP’18 Tutorial on Interpretable Deep Learning 14
Example: Quantum Chemistry
SLIDE 15
ICIP’18 Tutorial on Interpretable Deep Learning 15
From Input to Abstractions
SLIDE 16
ICIP’18 Tutorial on Interpretable Deep Learning 16
Learning Hierarchical Representations
SLIDE 17
ICIP’18 Tutorial on Interpretable Deep Learning 17
Learning Hierarchical Representations
- Multiple neurons with similar structure, but with different weight parameters.
- Compose them into a deep layered architecture.
SLIDE 18
ICIP’18 Tutorial on Interpretable Deep Learning
18
Dimensions of Interpretability
SLIDE 19
ICIP’18 Tutorial on Interpretable Deep Learning
19
Dimensions of Interpretability
SLIDE 20
ICIP’18 Tutorial on Interpretable Deep Learning 20
Dimensions of Interpretability
Decision Analysis "why a given image is classified as a scooter" Model Analysis "what does something predicted as a scooter typically look like."
SLIDE 21
Model analysis
SLIDE 22
ICIP’18 Tutorial on Interpretable Deep Learning 22
- find prototypical example of a category
- find pattern maximizing activity of a neuron
Interpreting the Model
goose cheeseburger car simple regularizer (Simonyan et al. 2013) complex regularizer (Nguyen et al. 2016)
Activation Maximization
SLIDE 23
ICIP’18 Tutorial on Interpretable Deep Learning 23
Interpreting the Model
SLIDE 24
ICIP’18 Tutorial on Interpretable Deep Learning 24
Limitations of Global Interpretations
SLIDE 25
ICIP’18 Tutorial on Interpretable Deep Learning 25
Making Deep Neural Nets Transparent
SLIDE 26
Decision analysis
SLIDE 27
ICIP’18 Tutorial on Interpretable Deep Learning
Black Box
27
Decision Analysis: LRP
Layer-wise Relevance Propagation (LRP) (Bach et al., PLOS ONE, 2015)
SLIDE 28
ICIP’18 Tutorial on Interpretable Deep Learning
28
Decision Analysis: LRP
Classification cat rooster dog
SLIDE 29
ICIP’18 Tutorial on Interpretable Deep Learning
29
What makes this image a “rooster image” ? Idea: Redistribute the evidence for class rooster back to image space.
Decision Analysis: LRP
SLIDE 30
ICIP’18 Tutorial on Interpretable Deep Learning
30
Theoretical ¡interpretation Deep ¡Taylor ¡Decomposition (Montavon ¡et ¡al., ¡2017)
Decision Analysis: LRP
SLIDE 31
ICIP’18 Tutorial on Interpretable Deep Learning
31
Explanation cat rooster dog
Decision Analysis: LRP
SLIDE 32
ICIP’18 Tutorial on Interpretable Deep Learning
32
Heatmap ¡of ¡prediction ¡“3” Heatmap ¡of ¡prediction ¡“9”
Decision Analysis: LRP
SLIDE 33
ICIP’18 Tutorial on Interpretable Deep Learning 33