Methods for interpreting and understanding deep neural networks - - PowerPoint PPT Presentation

„Methods for interpreting and understanding deep neural networks“

Presented by Philipp Wimmer

Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller 2017

M

• t

i v a t i

• n
• U

n d e r s t a n d i n g a n d v a l i d a t i n g d e e p n e u r a l n e t w

• r

k s i s h a r d

– M

a n y p a r a m e t e r s

– H

i g h l y n

• n

l i n e a r

– I

n t e r p r a t a b i l i t y w a s n ‘ t a g

• a

l

• f

D N N s

• A

b i l i t y t

• v

a l i d a t e i s n e c c e s s a r y f

• r

u n d e r s t a n d i n g a n d r e a l w

• r

l d a p p l i c a b i l i t y

• E

x a m p l e : D

• n

‘ t k n

• w

i f h i g h p r e d i c t i

• n

a c c u r a c y i s d u e t

• a

n

• m

a l y i n t r a i n i n g d a t a

I n t e r p r e t a t i

• n

A n i n t e r p r e t a t i

• n

i s t h e m a p p i n g

• f

a n a b s t r a c t c

• n

c e p t ( e . g . a p r e d i c t e d c l a s s ) i n t

• a

d

• m

a i n t h a t t h e h u m a n c a n m a k e s e n s e

• f

. G

• a

l : P r

• d

u c i n g a p r

• t
• t

y p e

E x p l a n a t i

• n

A n e x p l a n a t i

• n

i s t h e c

• l

l e c t i

• n
• f

f e a t u r e s

• f

t h e i n t e r p r e t a b l e d

• m

a i n , t h a t h a v e c

• n

t r i b u t e d f

• r

a g i v e n e x a m p l e t

• p

r

• d

u c e a d e c i s i

• n

( e . g . c l a s s i fi c a t i

• n
• r

r e g r e s s i

• n

) . G

• a

l : P r

• d

u c i n g a h e a t ma p

P a r t A : I n t e r p r e t i n g

• A

c t i v a t i

• n

M a x i m a z a t i

• n

( A M )

• A

M w i t h a n e x p e r t

• A

M i n c

• d

e s p a c e ( u s i n g G e n e r a t i v e A d v e r s e r i a l N e t w

• r

k s )

A c t i v a t i

• n

M a x i m i z a t i

• n
• P

r

• d

u c i n g a p r

• t
• t

y p e v i a m a x i m i z i n g

• C

l a s s p r

• b

a b i l i t e s m

• d

e l e d b y t h e D N N a r e f u n c t i

• n

s w i t h a g r a d i e n t

• U

s e g r a d i e n t d e s c e n t t

• m

a x i m i z e ( j u s t l i k e t r a i n i n g a D N N i n r e v e r s e

C l a s s p r

• b

a b i l i t i e s R e g u l a r i z e r

A r c h i t e c t u r e

• f

A M

• S

i m p l e t

• c
• m

p u t e

• R

e g u l a r i z e r p r e f e r e s i n p u t s c l

• s

e t

• t

h e

• r

i g i n ( m e a n

• f

d a t a )

• U

n n a t u r a l l

• k

i n g p r

• t
• y

p e

I m p r

• v

i n g A M w i t h e x p e r t

• R

e p l a c e r e g u l a r i z e r w i t h a m

• r

e s

• p

h i s t i c a t e d a p p r

• a

c h

• E

x p e r t i s t h e d a t a d e n s i t y

• F
• r

e x a m p l e

• b

t a i n e d b y t r a i n i n g a n G a u s s i a n R B M

• O

f t e n m

• r

e c

• m

p l e x d e n s i t y m

• d

e l s a r e n e e d e d

C l a s s p r

• b

a b i l i t i e s M

• d

e l

• f

t h e d a t a

(a) maximation of class probability function (b) favoring natural images – often suffjcient. (c) desired (d)

• ptimization of the

expert itself, hides failure modes

P e r f

• r

m i n g A M i n c

• d

e S p a c e

• O

f t e n l e a r n i n g t h e e x p e r t t

• a

h i g h a c c u r a c y i s h a r d

• E

x p e r t

• f

t e n v e r y c

• m

p l e x s u c h t h a t m a x i m i z i n g i s d i ffi c u l t

• A

s

• l

u t i

• n

i s t

• n
• t

e x p l i c i t e l y l e a r n p ( x )

• I

n s t e a d s a m p l e f r

• m

a n c

• d

e s p a c e w i t h k n

• w

n d i s t r i b u t i

• n

w h i c h w a s

• b

t a i n e d b y t r a i n i n g a n G A N

• T

h e n a p p l y d e c

• d

i n g f u n c t i

• n

t

• g

e t a p r

• t
• y

p e

D e c

• d

e d p

• i

n t i n C

• d

e s p a c e

P e r f

• r

m i n g A M i n c

• d

e s p a c e

• D

i s t r i b u t i

• n

i n c

• d

e s p a c e i s b y c

• n

s t r u c t i

• n

G a u s s i a n

• R

e g u l a r i z e r f a v

• r

s p

• i

n t s w i t h a h i g h p r

• b

a b i l i t y

R e s u l t s

S i m p l e A M A M w i t h e x p e r t A M i n c

• d

e s p a c e

P a r t B : E x p l a i n i n g

• S

e n s i t i v i t y A n a l y s i s

• (

L a y e r w i s e ) R e l e v a n c e P r

• p

a g a t i

• n

S e n s i t i v i t y A n a l y s i s

S e n s i t i v i t y A n a l y s i s

• G

r a d i e n t i s e a s i l y c a l c u l a t e d v i a b a c k p r

• p

a g a t i

• n
• T

h e m e a s u r e d r e l e v a n c e s c

• r

e i s n

• t

w h a t i s w a n t e d

• M

e a s u r e s n

• t

t h e r e l e v a n c e , b u t t h e l

• c

a l s l

• p

e

• f

i t

R e l e v a n c e P r

• p

a g a t i

• n
• M

a k e u s e

• f

t h e g r a p h s t r u c t u r e

• f

D N N s

• P

r

• p

a g a t e t h e r e l e v a n c e s c

• r

e b a c k w a r d s t h r

• u

g h t h e n e t w

• r

k i s s i m i l a r t

• t

h e b a c k p r

• p

a g a t i

• n
• f

t h e e r r

• r

d u r i n g t h e t r a i n i n g p h a s e

• R

e l e v a n c e h a s t

• b

e c

• n

s e r v e d ( s i m i l a r t

• c

u r r e n t i n a n e l e c t r i c c i r c u i t )

• L
• c

a l c

• n

s e r v a t i

• n

a t e a c h n e u r

• n
• F

i l t e r i n g : A b l e t

• b

l

• c

k t h e fl

• w

t h r

• u

g h c e r t a i n n e u r

• n

s

P r

• p

a g a t i

• n

R u l e

R e l e v a n c e C

• u

n t e r

• R

e l e v a n c e

H y p e r p a r a m e t e r s

• f

L R P

• R

a t i

• f

α a n d β d e t e r m i n e s t h e i n fl u e n c e

• f

c

• u

n t e r v a r i a n c e

C

• n

c l u s i

• n
• T

w

• m

e t h

• d

s f

• r

i n c r e a s i n g p

• s

t

• h
• c

i n t e r p r e t a b i l i t y

• N
• n

e e d t

• c

h a n g e e x i s t i n g a l g

• r

i t h m s

• E

n a b l e s b e t t e r u n d e r s t a n d i n g a n d v a l i d a t i

• n
• S

h

• u

l d b e i n t h e t

• l

b

• x
• f

e v e r y

• n

e u s i n g D N N s