Tutorial on Interpreting and Explaining Deep Models in Computer - - PowerPoint PPT Presentation

▶
tutorial on interpreting and explaining deep models in
SMART_READER_LITE
LIVE PREVIEW

Tutorial on Interpreting and Explaining Deep Models in Computer - - PowerPoint PPT Presentation

Apr 29, 2023 181 likes •417 views

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability

slide-1
SLIDE 1

Tutorial on Interpreting and Explaining Deep Models in Computer Vision

Wojciech Samek (Fraunhofer HHI) Grégoire Montavon (TU Berlin) Klaus-Robert Müller (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability GM 10:00 - 10:30 Coffee Break ALL 10:30 - 11:15 Applications of Interpretability WS 11:15 - 12:00 Further Applications and Wrap-Up KRM

slide-2
SLIDE 2

Why interpretability?

slide-3
SLIDE 3

Why interpretability?

slide-4
SLIDE 4

Why interpretability?

slide-5
SLIDE 5

Why interpretability?

slide-6
SLIDE 6

Why interpretability? Insights!

slide-7
SLIDE 7

Why interpretability?

slide-8
SLIDE 8

Overview and Intuition for different Techniques: sensitivity, deconvolution, LRP and friends.

slide-9
SLIDE 9

Understanding Deep Nets: Two Views

Understanding what mechanism the network uses to solve a problem or implement a function. Understanding how the network relates the input to the output variables.

slide-10
SLIDE 10
slide-11
SLIDE 11

Approach 1: Class Prototypes

Image from Symonian’13

“How does a goose typically look like according to the neural network?”

goose non-goose

Class prototypes

slide-12
SLIDE 12

Approach 2: Individual Explanations

Images from Lapuschkin’16

“Why is a given image classified as a sheep?”

sheep non-sheep

slide-13
SLIDE 13
  • 3. Sensitivity analysis

Sensitivity analysis: The relevance of input feature i is given by the squared partial derivative: evidence for “car”

DNN

input

slide-14
SLIDE 14

Understanding Sensitivity Analysis

Problem: sensitivity analysis does not highlight cars Sensitivity analysis explains a variation of the function, not the function value itself. Observation: Sensitivity analysis:

slide-15
SLIDE 15

Sensitivity Analysis Problem: Shattered Gradients

[Montufar’14, Balduzzi’17]

Input gradient (on which sensitivity analysis is based), becomes increasingly highly varying and unreliable with neural network depth.

slide-16
SLIDE 16

Shattered Gradients II

[Montufar’14, Balduzzi’17]

Example in [0,1]:

Input gradient (on which sensitivity analysis is based), becomes increasingly highly varying and unreliable with neural network depth.

slide-17
SLIDE 17

LPR is not sensitive to gradient shattering

slide-18
SLIDE 18

Explaining Neural Network Predictions

Layer-wise relevance Propagation (LRP, Bach et al 15) first method to explain nonlinear classifiers

  • based on generic theory (related to Taylor decomposition – deep taylor decomposition M et al 16)
  • applicable to any NN with monotonous activation, BoW models, Fisher Vectors, SVMs etc.

Explanation: “Which pixels contribute how much to the classification” (Bach et al 2015) (what makes this image to be classified as a car) Sensitivity / Saliency: “Which pixels lead to increase/decrease of prediction score when changed” (what makes this image to be classified more/less as a car) (Baehrens et al 10, Simonyan et al 14)

  • Cf. Deconvolution: “Matching input pattern for the classified object in the image” (Zeiler & Fergus 2014)

(relation to f(x) not specified)

Each method solves a different problem!!!

slide-19
SLIDE 19

Classification cat ladybug dog large activation

Explaining Neural Network Predictions

slide-20
SLIDE 20

Explanation cat ladybug dog

=

Initialization

Explaining Neural Network Predictions

slide-21
SLIDE 21

Explanation cat ladybug dog Theoretical interpretation Deep Taylor Decomposition ?

Explaining Neural Network Predictions

depends on the activations and the weights: LRP naive z-rule

slide-22
SLIDE 22

Explanation cat ladybug dog Relevance Conservation Property

Explaining Neural Network Predictions

large relevance

slide-23
SLIDE 23

Gradients

LRP (Bach&et&al.,&2015) Deep/Taylor/Decomposition (Montavon&et&al.,&2017&(arXiv&2015)) LRP/for/LSTM (Arras&et&al.,&2017) Probabilistic/Diff (Zintgraf&et&al.,&2016) Sensitivity (Baehrens&et&al.&2010) Sensitivity (Simonyan&et&al.&2014) Deconvolution (Zeiler&&&Fergus&2014) Meaningful/Perturbations (Fong&&&Vedaldi 2017) DeepLIFT (Shrikumar&et&al.,&2016)

Decomposition

Sensitivity (Morch&et&al.,&1995) Gradient/vs./Decomposition (Montavon&et&al.,&2018)

Optimization

Guided/Backprop (Springenberg&et&al.&2015) Integrated/Gradient/ (Sundararajan&et&al.,&2017) Gradient/times/input/ (Shrikumar&et&al.,&2016) PatternLRP (Kindermans&et&al.,&2017) LIME (Ribeiro&et&al.,&2016)

Deconvolution Understanding/the/Model

Network/Dissection (Zhou&et&al.&2017) Inverting/CNNs (Mahendran&&&Vedaldi,&2015) Deep/Visualization (Yosinski&et&al.,&2015) Feature/visualization (Erhan&et&al.&2009) Synthesis/of/preferred/inputs (Nguyen&et&al.&2016) Inverting/CNNs (Dosovitskiy&&&Brox,&2015) GradKCAM (Selvaraju&et&al.,&2016) Excitation/Backprop (Zhang&et&al.,&2016) RNN/cell/state/analysis (Karpathy&et&al.,&2015)

Historical remarks on Explaining Predictors

TCAV (Kim&et&al.&2018)