Tutorials Interpretable Deep Learning: Towards Understanding & - - PowerPoint PPT Presentation
Tutorials Interpretable Deep Learning: Towards Understanding & Explaining DNNs P a r t 2 : M e t h o d s o f E x p l a n a t i o n W o j c i e c h S a m e k , G r g o i r e M o n t a v
1 / 3 6
W
c i e c h S a m e k , G r é g
r e M
t a v
, K l a u s
e r t M ü l l e r
2 / 3 6 interpreting predicted classes
explaining individual decisions
3 / 3 6
Q: Where in the image the neural networks sees evidence for a car?
car non-car
4 / 3 6
5 / 3 6
Q: In which proportion has each car contributed to the prediction?
car non-car
6 / 3 6
G
l : D e t e r m i n e t h e s h a r e
t h e
t p u t t h a t s h
l d b e a t t r i b u t e d t
a c h i n p u t v a r i a b l e . D e c
p
i t i
p r
e r t y :
i n p u t
D N N
7 / 3 6
G
l : D e t e r m i n e t h e s h a r e
t h e
t p u t t h a t s h
l d b e a t t r i b u t e d t
a c h i n p u t v a r i a b l e . D e c
i n g a p r e d i c t i
i s g e n e r a l l y d i f fi c u l t .
8 / 3 6
c
p u t e s f
e a c h p i x e l : e x p l a n a t i
f
“ c a r ” ( h e a t m a p ) : e v i d e n c e f
“ c a r ” D N N
i n p u t
9 / 3 6
Question: If sensitivity analysis computes a decomposition of something: Then, what does it decompose?
1 / 3 6
S e n s i t i v i t y a n a l y s i s e x p l a i n s a v a r i a t i
t h e f u n c t i
, n
t h e f u n c t i
v a l u e i t s e l f .
e x p l a n a t i
f
“ c a r ” i n p u t v a r i a t i
= m a k e s
e t h i n g a p p e a r l e s s / m
e a c a r .
1 1 / 3 6
2 . F i r s t
d e r e x p a n s i
a t r
p
n t : O b s e r v a t i
: e x p l a n a t i
d e p e n d s
t h e r
p
n t . 1 . T a k e a l i n e a r m
e l : 3 . I d e n t i f y i n g l i n e a r t e r m s :
a d e c
i t i
r
p
n t
1 2 / 3 6
O b t a i n e d r e l e v a n c e s c
e s H
t
h
e t h e r
p
n t ?
l
e n e s s t
h e a c t u a l d a t a p
n t
e m b e r s h i p t
h e i n p u t d
a i n ( e . g . p i x e l s p a c e )
e m b e r s h i p t
h e d a t a m a n i f
d . r
p
n t
1 3 / 3 6
s e c
d
d e r t e r m s a r e h a r d t
n t e r p r e t a n d c a n b e v e r y l a r g e
S i mp l e T a y l
d e c
i t i
i s n
s u i t a b l e f
h i g h l y n
i n e a r mo d e l s .
N
l i n e a r m
e l
1 4 / 3 6
I n t e g r a t e d G r a d i e n t s
[ S u n d a r a r a j a n ’ 1 7 ]
:
u l l y d e c
p
a b l e
e q u i r e c
p u t i n g a n i n t e g r a l ( e x p e n s i v e )
h i c h i n t e g r a t i
p a t h ?
[ S u n d a r a r a j a n ’ 1 7 ] A x i
a t i c A t t r i b u t i
f
D e e p N e t w
k s . I C M L 2 1 7 : 3 3 1 9
3 2 8
1 5 / 3 6
S p e c i a l c a s e w h e n t h e
i g i n i s a r
p
n t a n d t h e g r a d i e n t a l
g t h e i n t e g r a t i
p a t h i s c
s t a n t :
g r a d i e n t x i n p u t
1 6 / 3 6
1 7 / 3 6
V i e w t h e d e c i s i
a s a g r a p h c
u t a t i
i n s t e a d
a f u n c t i
e v a l u a t i
, a n d p r
a g a t e t h e d e c i s i
b a c k w a r d s u n t i l t h e i n p u t i s r e a c h e d .
1 8 / 3 6
[ B a c h ’ 1 5 ]
1 9 / 3 6
2 / 3 6
[ B a c h ’ 1 5 ]
C a r e f u l l y e n g i n e e r e d p r
a g a t i
r u l e :
neuron contribution available for redistribution normalization term pooling received messages
2 1 / 3 6
neuron contribution available for redistribution normalization term pooling received messages neuron activation available for redistribution normalization term weighted sum
2 2 / 3 6
neuron activation available for redistribution normalization term weighted sum
E l e m e n t
i s e
e r a t i
s V e c t
e r a t i
s
2 3 / 3 6
C
e t h a t r e u s e s f
w a r d a n d g r a d i e n t c
p u t a t i
s :
neuron activation available for redistribution normalization term weighted sum
S e e a l s
t t p : / / w w w . h e a t m a p p i n g .
g / t u t
i a l
2 4 / 3 6
G P U
a s e d i m p l e m e n t a t i
L R P : C h e c k
t i N N v e s t i g a t e [ A l b e r ’ 1 8 ] h t t p s : / / g i t h u b . c
/ a l b e r m a x / i n n v e s t i g a t e
2 5 / 3 6
2 6 / 3 6
Q u e s t i
: S u p p
e t h a t w e h a v e p r
a g a t e d t h e r e l e v a n c e u n t i l a g i v e n l a y e r . H
s h
l d i t b e p r
a g a t e d
e l a y e r f u r t h e r ? I d e a : B y p e r f
m i n g a T a y l
e x p a n s i
t h e r e l e v a n c e .
[ M
t a v
’ 1 7 ]
2 7 / 3 6
O b s e r v a t i
: R e l e v a n c e a t e a c h l a y e r i s a p r
u c t
t h e a c t i v a t i
a n d a n a p p r
i m a t e l y l
a l l y c
s t a n t t e r m .
neuron activation available for redistribution normalization term weighted sum
R e mi n d e r :
2 8 / 3 6
R e l e v a n c e n e u r
: T a y l
e x p a n s i
: R e d i s t r i b u t i
:
2 9 / 3 6
(same as LRP-
α
1
β
)
( D e e p T a y l
g e n e r i c ) ✔
1 . n e a r e s t r
2 . r e s c a l e d e x c i t a t i
s C h
c e
r
p
n t
3 / 3 6
( D e e p T a y l
g e n e r i c ) P i x e l s d
i n : C h
c e
r
p
n t R e s u l t i n g p r
a g a t i
r u l e
3 1 / 3 6
( D e e p T a y l
g e n e r i c ) W
d e mb e d d i n g :
a d a p t e d f r
T e n s
fl
t u t
i a l
C h
c e
r
p
n t R e s u l t i n g p r
a g a t i
r u l e
king man queen woman
3 2 / 3 6
[ M
t a v
’ 1 7 ]
3 3 / 3 6
[ K a u f f m a n n ’ 1 8 ]
O u t l i e r s c
e
O n e
l a s s S V M r e w r i t t e n a s a m i n
i n g
e r d i s t a n c e s : D e e p T a y l
d e c
p
i t i
:
3 4 / 3 6
t l i e r d i g i t s p i x e l
i s e e x p l a n a t i
w h y t h e y a r e
t l i e r s d a t a s e t
3 5 / 3 6
i n p u t e x p l a n a t i
f
t l i e r n e s s
3 6 / 3 6
Explaining deep neural networks is non-trivial. Simple gradient-based methods either do not ask the right question, or are difficult to scale to deep models. Propagation-based approaches (e.g. LRP) seem to work better on complex DNN models. (This will be validated in Part 3). Deep Taylor Decomposition provides a theoretical framework for understanding and deriving LRP-type explanation procedures.