weighted Methods Importance 20 : - Sleep Planning Inference - - PowerPoint PPT Presentation

SLIDE 1 Lecture 20 : Importance

• weighted

Methods Wake

• Sleep

, Planning as Inference Scribes :

Xiong yi

Zhang

Jered McInerney

SLIDE 2 Variational Auto encoders :

Summary

Idea I : Use neural netwonus to parameterize a deep generative model pg ( x , 2- ) Idea 3+4 : Use auto encoding variational inference to train an inference model go, ( Z Ix ) that approximates the posterior pg CZIX ) Idea 2 : Use samples from go , ( 7 K ) to approximate the gradient To log pok ) Idea 5 : Use me parameterization to compute

Deff

SLIDE 3 Auto encoding Variational Methods : General View Variational Bound ;

Any

method that produces ' a valid importance weight defines a lower bound

£1943

• ftp.#ggzixsllogwDsEtp..llogpdxYw=Pg%E7I

, Gradient Estimation : Use samples 2- s

• qczlx

)

To

L ( O , G) =

ftp.#qopaxil0ologpocx.ZD

) (Always the some )

Tq L ( O

, 4) =

ftp.#qopxxif9ologqq1zixsflegwlx,zstb

)) (REINFORCE

)

Tg

L I 0,10) = Epa, # pce , [ Og log w ( x , 2- aces) )) ( Repwaweteiud )

SLIDE 4 Auto encoding Variational Methods : General View Importance

• weighted

auto encoders & Auto encoding SMC : Replace W with unbiased estimator EI [ IT = pocx )

£194

• .

ftp.#qzixsleogwDsEf..llogpdxyw=pofx.zy9ql7IX

) n

↳

Wake

• sleep

methods ; Replace

lower

band with upper bound

Ulp

) > ,

log

poles ^

1-

Planning as inference ! Replace target pot x. 7) with a density exp [ RIZ ) T pgcz )

SLIDE 5 Importance

• weighted

Auto encodes Normal Auto encoder : Use reparauretevized Marte Carlo

£194 )

• ftp.t#pceileogwo.ak.xYywo.oi=g:YI:Y?T

, = is

II.

leg

wo.aceb.h.xbsxbnlln.it/sxYiii)qbih~pCE

) Importance

• weighted

Antoon coder : Replace weight w K with an average weight I =

I

E Wh U I £ ( 99 ) =

Epa ,

# go , , like

flog

I )

Eof

log E)

slog

Egli )

• leypfx

, = ' g

leg ↳ £

. Wo ,¢lEb' h ,xb )) TLDR : Move Sum

• ver

k inside log

SLIDE 6 Importance

• weighted

Auto encodes Importance

• weighted

Auto encoder : Replace weight w K with an average weight I =

¥

E Wh U I L ( 0,0) =

Epa ,

# go , cziikix , [log I ) = is E. leg ( II. PH3z.net#qf2-oCEb.h)/Xb )) Note : Any unbiased estimation E [ I ] = po Cx ) defines a lower bound

• n

Epa , [ leg Pok ) ] L =

Ep

Egl by

I ) s

Epl log EGEE ) )

= Ep flog pok ))

SLIDE 7 Importance

• weighted

Auto encodes Gradient Computation ; Gradients

• f

the IWAE bound are self

• normalized

importance sampling estimates L ( 90 ) =

Epa ,

# punk , flog 'EE

woo

, CE !× ) ) REINFORCE TRICK T we Wiley w I For oflolol) =

Epix,

# pceiikgf.gg#If0o.$wgfEk ) ) =

Epix .

# peeing

I

f

wqY.l.IT?7.,0o.oibgw..pld.xY

9 Self

• normalized

weight I gradient

• f

log weight

SLIDE 8

Minimizing

• n

Upper Bound Idea ; Make go , Cz it ) similar to poczlx )

by

minimizing inclusive KL rather than exclusive KL

• UH

)=

Epa , # p.com/

leg

]

Note : Mt " 9 ) instead

• f

Klfqllp ) = ftp.x.llogpocxyt#pcxilEpcaix,lley

!

])

= Ep

I log

Poet ) ] t

Epc ,→ I

KL ( pottksllqotfks)) 7, Ep

I log

Po CA ) Making UH ) smaller also reduces the KL divergence

SLIDE 9

Minimizing

an Upper Bound Idea ; Make go , Cz it ) similar to poczlx )

by

minimizing inclusive KL rather than exclusive KL UH =

Epa , # p.com/bgPgIYIIT

, ]

I

No dependence

• n

parameters 4

• To

,U4

) = # peal # pg*× , I

Deploy

94171×3 )

T

Need samples

from

potfk

) ( can use any Monte Carlo method )

SLIDE 10

Minimizing

• n

Upper Bound Idea ; Make go , Cz it ) similar to poczlx )

by

minimizing inclusive KL rather than exclusive KL

• To

,U4

) = # peal # pg*× , I

Deploy

94171×3 )

• Use

self

• normalized

IS =

ftp.xsf#g.czxs/P:YIT

, Oo ,

bgqattixs ))

I

§

?

, , Woio Czbih , ×b , EE . Wo,¢Czb4×b ,

4 leg 991744×6 )

[ Sometimes referred to as " reweighed " wake

• sleep

SLIDE 11

Reweighed

Wake

• sleep

Wake phase : Sample Xb n pdat ' C x ) and approximate b 2- b' h ~ pact 1×1 using

self

• normalized

importance sampling with proposal zb.hn

qgzlxb )

b , h To Epa , I log poem ) =

LEE IT

me to

pocxb.tk

)

• To

,

Epcxilkllpottix

) " 9ohm )) =

• ¥EE§be

To, go , lzbihlxbl Sleep phase : Sample Xb , 7 b ~ pocx , 7)

from

the generative model and compute

gradient

( often

shipped )

• 9

ftp.cnn/logPg::ITIl/=fE4bggo,czbixbs

b