Ruiyang Elizabeth : , , Released Homework 4 yesterday : Apr - PowerPoint PPT Presentation

Product Lecture Algorithm Sum 17 : - Yaoshen Scribes Ruiyang Elizabeth : , , Released Homework 4 yesterday : Apr 13 Fri Due

Today Variables Marginal Discrete Exact over : Goal Marginal form Compute of the : P(×i=×ii×j=×i ) ' ×K=×K ) , .PK '=× ×§µ= = ' - , ;,µ , All Assumption variables Xw discrete : are Enables Calculation of posterior : 1 Xj × ;) , Xj=×j ) p ( X p( X =x ; ;=×i = = ; ÷ ;) lj=x

Markov Example Chain : : Rearrange Terms Idea Sum in b d Pla ) c = , , , pcblcspccld pcd ) b) ) al pc B D C [ [ [ ABCD a ,b Naive pca ) d) pc = c. : , b= D= CI . ) yd( C) - B plalb ) ( § pcblo ) ( § [ ) ) ) pccidipld = b= , , = § pcaib ) ( § pcblc ) B ) ) I ydk pcalbikcb ) = b= - , , ( b ) ya

Markov Example Chain : =b¥ ,p( aiblplbk , § , a£ ABCD ) )p( cld pc as . B B = § c) yd yccb [ ) yicb ) pcaib ybla ) pcalb ) ) = b= = , i AB KLB Jdc b = { C [ ) ) pcbi 4 pcd ( c 3 ( d) i p = c D= =L , BC Cost CD : Question What Computational the Complexity is : ? AB of computing ) BCTCD pca +

Sum Graphs Factor product on . - branching ) Network ( Bayesian non ) plc bl blc b d ) Id d) pla pc PC ) 1 p( c a = , , , - branching ) Graph ( Factor non fz ( b. f. f fald ) ( ,d ) I b) b d ( c) Pla ) a. c = 3 , , ,

Sum Graphs Factor product on . - branching ) Graph ( Factor non f. f< fzlb fs b. a ) d (d) b) ( d) Pla ( ) c. c = a , , , , , Variable Messages Variable µa÷ to : ,c ) § fuld ) fzl f. fzlb b. c ) d) pea b) 1 a. c. = , b) § , . f. fzlb b ) a ) ,c ) ( pca ( µd→< = a. , , ← 1 b ) b →

Sum Graphs Factor product on . - branching ) Graph ( Factor non b d Pla ) c = , , , f. fzl b. b) a ) ( a , f< f. (d) d) ( c. , Graph ( singly ) Factor ( ) b. Tree connected a. a - Factors with b multiple d ) edges PC c a = , , , \ Vons f. ( fz ( b. with b) d) c. a. multiple q f3( fald fsld ) c) ) ,e edges

Sum Graphs Factor product on . Graph ( singly ) Factor connected - fd b. [ d) b. (b) ( a ) µc µh c. f. Q = → → 'd 1 d ) ° µd→ a- f. µc→ ( 0 µf ( c ) = q Q f. } → c gpa µk→alal= fz µi3→ ( do ) a ) = as Ap d ) µfs→d( I d) µd→ fzldl µfa→d( = µe→ facet fald fsldl { ,e ) dla ) Mts . : → ( d ) µd→f . t µ ( d) µfs→d( d d) fy → Factor Variable Messages : as . - ~ fsla ) { b) { µfzl fuld f. f3( b. d) I a. b ) c) ,e ) pca = c. , i ( b ) M fz→b

Sum Graphs Factor product on . Advantage Factor Graphs Can compute marginal of : any { field d) µd→fa( ) < ( e) ,e p = t s s , d) µfz→dldlµfr→dl Md→fdd)= I 9 9 = [ fzlb ^ µfz→d 1 a ) d) ( to ) My , 9 a → ( C ) µc ' fz -

Sum Graphs Factor product on . General :{ d } Factors b. helfzl torn c. : M ¢f( Xf ) X ) ( p = f a } ( factors fz ,fu,fs ) ) held ( x ) which ? he in × occurs Xf ( that f variables nelf ) depends hecfl I ) on [ Factor Variable M Old Xe ) → fly ) ( × ) µ µf→× → : = gene 5 } x } Xfix (f) { \{ ( Sum ) M Variable Factor I × ) µ×→f = → µ9→x K ) : { f } ' ( × ) he \ 9 E ( Product )

Variables Belief Compute Marginal Propagation All for : < , rd ss 4 f General Marginal Form a : L < s M pkl a µt→×l× ) s s < s fehecx ) Algorithm All Messages Compute : Pich variable 1 x any . Compute incoming messages z , Compute outgoing 3 messages

Variable Variable Variable Messages Factor ←s vs as - branching ) Graph ( Factor non < < < < < < < I b ( b ) fz( . b.cl c ) µ 1 µ< = f. → < → ( b ) → b fz

^ gzn gun Algorithm / Example Backward Forward HMMS ) : - Model Generative Graph Factor =k~Dik( fg fz f fz ytlk Discrete ) , ( 17 hi Mk S ~ s < s g < < , . . . , , . .AkK ) ^ htlht Ah g. , g , . . , , 1h+=k Vt1ht=h ) pl ✓ + ~ Marginal , Compute Goal : 1 ht IY ) ) Vt p = Bt( he ) i , , . . - , .+,→hdhHµg+→htlht ) µft→hilhtlµf & -5 atchl

Algorithm / Example Backward Forward HMMS ) : - Model Generative Graph Factor fg fz f fz Disc ( ) , hi 17k 17 Is g g ~ s g g > . . , . , , ... ,Ank ) Disdain ^ htlht ^ ^ =L ^ ~ 92 93 9 94 , . , , pN+lh+=h ) Vtlht=h ~ plvtlht = h ) ' Forward ) ( outgoing Pass messages p µg+→h+lht=h\µf+→h+lht=h ) tlhl • = = µn+→f↳,lht=H { ft ( h ,l ) µh+ Mgeshtlhteh ) heel ) . ,→f+( = = Ahl , I b ) ) de pcht.hlht.pl

Algorithm / Example Backward Forward HMMS ) : - Model Generative Graph Factor fg fz f fz ) Disc ( , hi 17k ni ~ < < 2 < < ( s ... , , ... ,Ank ) Disdain htlht =L ^ ^ ^ ^ ~ 92 93 9 94 , . , , ^ ^ n ^ pN+lh+=h ) Vtlht=h ~ Backward Pass ( Incoming Messages ) ) µh++,→f+ll ) ( h ) fell { → h.lk B Mf ) ,h = = , t 4 )µft+,→ht+ " { fell .h ) " ) µgt+,sh++ = , , Ahl Bttill p(✓t+ , lhtnol ) )

/ Algorithm Example Forward Backward HMMS ) : - Forward Pass 1h ,=k ( ,=kap( d. pch ( k ) t.tl K ) v. = = h ) { K ( t Vtlht i ) Aeu 1<2 . ,ll dt h ) ) l Pc at > = =L Kt ) 1<2 ) 0( Backward OCK Pass ( t . 1) < < + ( ) K ptlhl t=T 1 = k ltctl § pi✓t+,lht*=l)P++,ll 1<2 Ahe ) 1h ) B = + it Marginal s " 0+14 . ,→h+l↳µgµhd↳µfphd ytlh Ptlh ) µf+ ) a =