Project Wednesday Apr 25 : Apr Project Reports : Friday 27 - PowerPoint PPT Presentation

Lecture Review 22 : Homework Due Friday 4 : ( 18 Apr ) Next Wednesday Exam : Presentations Project Wednesday Apr 25 : Apr Project Reports : Friday 27 Will Notes today release Lecture : ) submitted if (

Exam Topic List https://course.ccs.neu.edu/cs7140sp18/exam-topics.html

Review Minimizing KL Divergences : f ) Density Target 2 : Y ) 1×21 MY Pc 1×2 ' × × '×' ' Approximation : ¢ ) I ( ,×z q × , µ ( ¢£ ) ( under " ¢ = x ; , approximates variance ) 1<2 ( ) ) 11 ,×z1 qcx ) ,×z;¢ pcx y arg min , , ¢ KL ( PC :O) ) EPILBP H qk ,×zly ) min × arg ,xz , , ( ¢ oven - approximates variance )

Review Minimizing KL Divergences : f ) Density Target 2 : Y ) 1×21 MY Pc 1×2 × 1 × '×' ' Approximation : 4 ) ( ,×z 1 q × , ( under ;¢z , ) ) qkz g. ( × ;¢ = , approximates variance ) K2( ) ) , ,×z1y 11 qcx ) ,×z ;¢ pcx argmin , ¢ KL ( PC :O) ) EPILBP H qk ,×zly ) min × arg ,xz , , ( ¢ oven - approximates variance )

Review Vaniaticnal 9) Max Likelihood Inference vs : LC 10,41 plxsd KL(q( Expectation maximization ;g ) p(×,z 't ] an ;¢,[ log L ( 01,01 # = , qcz ;¢ log ;O ) ) 11 pczlx pc 7 ;¢ ) . = - log ) PC ;D ) When ;O = x ;¢ ) PAK ql 7 = step ) true after M ( - Variation Inference ' ' ] PK 't [ log # = ql 7,0 ;¢ ) ;¢ ) 7,0 9C In A plot ,Olx :D ) ) KL( 917,0 leg ;¢ ) optimize ) Do not = -

Review Vaniaticnal Max Likelihood Inference vs : ( Hand ) Example Question : • • • • • Stretch EM bound the • • • bound The VBEM • and K function of c ) as a VBEM EM L 56 789 56 34 789 k 34 k 2 1 1 2

Review Vaniaticnal Max Likelihood Inference vs : ( Hand ) Example Question : • • • • • Stretch EM bound the • • • bound the VBEM • and K function of c ) as a VBEM EM L +t#t t 56 789 56 34 789 K 34 k 2 1 1 2

Aside Maximum Likelihood KL minimization ' as . Data ( Xn PDATA x ) ~ N ; n n = , . . . , |d7 Model :O plx :O ) ) pcx 7 : = , KL divergence ; ) ) ( pot 't KL x ) ( 11 ;o pcx - pPHA(× , ] , [ log + log E :o) pcx = - poatak . # I , §Y log Ix ] H = pcxn ) ;o

Review Minimizing KL Divergences : KL( PDATAHP ) ( p ) pllq ) KL KL .( 11 a Variation belief EM al Loopy propagation - - Stochastic Expectation propagation VI - - Black VI box - - Expectation maximization - VAES - ( likelihood ) max ( encoder ) ( ) ✓ AES decoder .

Review Conjugate Priors : 1 yttcx ) ) ] hcxl Likelihood acy pcxiy exp ) : = - hc g) eipfytd . acrpiz ) ] pcyli Conjugate ) au prior = : . , ] Joint pl x. y ) exp[ ,c x : E) exp{ acts ) ] heap at yi = - 15 ) Posterior pcylx ) pcy = ) ] ac 5) AN [ Likelihood Marginal ) exp pcx = -

VBEM Using Functional Derivatives Updates Functional Derivatives Differentiate : an function to integral with respect a |dx ( ( x ) ) L[ bk ) bcx , ) acx ) ccxi acx > + c = , , " " Idea Differentiate functions integrand wnt : 81 oh i ok bcx , and fak = = 1 = Ek Jbcx ) , ,

VBEM Eqa aca pcx leg ,[ Using Functional Derivatives Updates ,q(m[ pcxiz pczlo Joint = log log + log pl O ) log plx 7,0 ) ) ,O ) + , Objective ) 7,0 PCX , LGHI ,a( d) [ log ) : qco , , 7,0 ) ] got , ] log E # = - , qa an qco ) ] [ log Derivatives Functional # - ,• , a , 7,0 ) ] Eta ,•)[ log leg §§µ PK qczs i o = - - =

VBEM plx Using Functional Derivatives Updates 9101 Etqcz Lcqataca ) Eamon . ,[ log PgYI'}Yh- ) : , [ log ) ] pcxitt . ,[ loggia ] Eg , , Eqn = - ,q,o qco ) ] a) [ log Derivatives Functional # - qc ,0 ) ] ,o , [ log # log §§µ qh plat . ) 1 - - o = = a , [ leg , 7,0 ) ) log 8¥ . - 1 = .

VBEM Using Functional Derivatives Updates Derivatives Functional ) ] ,o , [ log # log §§µ qh 7,0 pcx . ) o - 1 = - = , a 7) [ leg , 7,9 ) ) log El 910 ) plx c qc 1 = ofgffm = - - Updates ) ] ] exp [ Etqco ) [ log 7,0 gas x pc x. ] ] I log exp [ # , 7,9 ) qlo ) × . , pix qn