19 : Baham Sabbir Scribes : , Belief Propagation : Problem - PowerPoint PPT Presentation

The Algorithm Lecture Junction Tree 19 : Baham Sabbir Scribes : ,

Belief Propagation : Problem The With Loops R far Graph Directed b ; a , b. d pca ) c clb d ) p ( b ( Pla ) dla l a) p( ) = p , A C p fz fu a. Fi Graph Factor ; 3h b s a. s a a ,b d pc ) c , n a ,b ) f }( fu( b. fz( f. (a) d) c. d) ↳a• = a. L I • C Almost Bayesian loops have all networks =

Loopy Belief Propagation Step Messages Initialize 1 : He h ] he{qB µ[ f. x. f) EE ][ 1 = × a. fl fz fu f. ,f][ h ] µ[ 1 = × b a. a Step Update 2 : messages a. a. f for SCHEDULE : € a c for helf ) e : × Update µf→× and µ×→t Problem not guaranteed Convergence :

Junction Trees Today : loops Transform With Idea graphical models : loops without into trees clique Markov Network Junction Tree Advantage : Can compute marginal exact , =

Reparameterisatim pet plays ) pcaib ) = Tya PC b) Markov Example Chain ) : twice b pcalb ) pcblc pca,b_7 d ) PC ) a = c , , , marginalsgyy pcd ) pccld ) Express Repurametnisation joint as : and b c occur of product \ . Plb , c) , # plc ,d ) b d ) p( a c = , , , peb ) it by plb ) divide ) pcc

Graphs Clique A Definition fully connected clique is a : - of model Subset variables graphical in a potentials Clique ' )¢(X2 a . -17 ¢( b. ( a. b. c) p( 4171 ¢ 7 1 d) a. b. d) c. c. = = - X 'RX2 ) b. ¢( ¢( c) 7 In Potential separator 72 ' X 'nX2 2/2 :{ b. e. a } a ,b ,c} 7 X '={

Clique Graphs Absorption Normalizing : Goal to Transform clique that graph ensure : marginal potentials form of take densities the ) ¢( V ) ¢( =p(s7 pcuuw ) W = : s ) ¢( = dcvsdcw ) = marginal § ( S ) cliques for ( analogous BP ) to ¢( W ) ' of ( V ) (5) ¢ pcw ) ) pcu = =

Clique Graphs Absorption Normalizing : Uuw ) V ) ¢( ¢( w 4µW %) p( ) = s ) ¢ ( pcvuw 10¥51 Marginal for V [ PIV ) ¢( D) £u¢(W)WiV ) = = ¢( s ) Absorption from V 5 to through W toys ¢*lv ) := 4*15 ) 2 011 D) ¢ ( w ) = WW ( s ) ¢ ¢N)0(W)_t ¢*ys ) ¢w ) = = toys ) ¢( s ) s ) ¢( )

Clique Graphs Absorption Normalizing : 0*4 # * ) V ) ¢( ¢( W w ) p( V. = csl Absorption from V through to W S ¢**Cs g*lw [ ) ) ¢*w ¢ ( w ) ) ⇐ ÷ 01W ¢* ( s ) . Preserves density * * ¢* ' " ' 0401101W w ' o¥*yYy¢iwida¥÷y = = s ) ¢*( * # ( s ) ¢ , j

Clique Graphs Absorption Normalizing : V ) ¢ ( ) ¢( w ) w p( V. = s ) ¢ ( §(v)§( w ) = ¢(s7 U ) 44¥ = ¢*lV ) of ( v ) ¢*ls ) W ) ¢**cI=p( ¢ ( plu ) w{sQ( = = = , =g*lw)=¢( # §lw ) # fgs ( s ) V ) w ) w ) of ⇐ * ( s ) ¢ ¢*% ) § ( s ) pcs ) = =

Clique Graphs Absorption Normalizing : Absorb General Case along directions edge both each in : Schedule followed Compute Incoming messages : by outgoing messages

( Singly ) Junction Trees Connected ¢( ) ¢ ,Xu ) ¢ ( xs ,×a\ ( 1 X Xu ,Xu × xz pcxi xz = } , , , , ) ( § ) ) ( §¢( Xu ) ) ¢ 1×3,41 ¢( pcxi Xu ) ¢ C Xz , Xu × × ,xa = } , , , ) ( § locks ,×u ) ) ( { ) ¢( ¢ ( × ,xa1 pcxz xz ) xu xu = , , , ( § ) ( { , ,×a ) ) qk ¢( pcxs ,×n xul × = , , Clique Graph Markov Networh

( Singly ) Junction Trees Connected ¢( ) ¢ ,Xu ) ¢ ( xs ,×a\ ( 1 X Xu ,Xu × xz pcxi xz = } , , , , ) ) ( { , ,xu ) ( { ) ) p( |p( in Qcx ( x ¢ pcxi xu ) ¢cx ,xn = . , I. , ,×u ) ) ( §¢c× ,×a ) ) ( folk ) ¢( × ) xz ,xa xu = . , . , ) ) ( § ,×n ) ) ( { 011×3.41 dcx ¢c xaxa 4,41 ' = , , ) ( , ) , ,X< Xu ) X } th PCX PCXZ p , ¢( ¢ ,Xu ) 01 ( xs ,xu\ ( 1 ,Xu x xz = , 2 ,xn ) ) ) ( ( { ) ) ( { ) ) ( { dcx ¢k},×n dcx ,xa . , PCXL , )2 ) PCX Xa = X Xz > , , , , ,

( Singly ) Junction Trees Connected ¢ ( ) ,×⇐ ¢ , Xu ) ¢ ( xs ( 1 X Xu Xu × xz pcxi xz = } , , , , , -10k€ of ( X :) Potentials Clique xul ) ×µ pcxe p(×3 pc × ) xa = , , , , ' ) ' 4^15 Potentials Separator pcxn ) www.w.ru#iEikEnYKYYkn.+iatru)

( Singly ) Junction Trees Connected Junction Tree Graph Markov Network Clique x General Property When variable in x occurs : a all then separators loop remove a may x on we , from arbitrarily separator chosen an

Property Junction Intersection Trees Running : A Definition tree tree junction Is : a for V of W all if each pair nodes and , , from the U the path to W nodes contain on Vn intersection W Consistency For clique of V nodes W pain and : any tree the marginal for intersection the junction in a . Un I consistency satisfies condition W the = I [ # ( ) 011 W ) ) ( 0 I ¢ = = W\U Dlw i ,

connected ) ( Singly Constructing Junction Trees - by Moralize Step adding edges 1 a graph : between parents

connected ) ( Singly Constructing Junction Trees - Define Step 2 clique graph : a = -

connected ) ( Singly Constructing Junction Trees - Break loops Step Junction Tree 3 obtain to : a X X Any Definition tree maximal weight spanning : tree The weight tree junction of is is a a . curdinalittes separators defined the of as sum over ,

connected ) ( Singly Constructing Junction Trees - Define Step 4 potentials : f g) b. d. p( e. a. c. , Pcb clb a) a ) ) ( pcflc p ( plgle p = , plhle . b. ( c) a. pcdsplelcidd ( e) d. C , ) ) ) - - - ¢( e. h ) ¢lc f) e. g) ¢( , c ) ¢ ¢ (e) ( =| 1 =

connected ) ( Singly Constructing Junction Trees - Propagate 5 Step according to : messages an absorption schedule ECX 'l=p( F) dlxi ) → y § \u £ ' ' , .es ;) ¢csis= ¢ 's → µ *

( General Constructing Case ) Junction Trees Problem Marginalization edges : introduce can , b) ¢( ¢( a ,b 4lb ) d) d pc ) c. a c. = ,c , ¢ ( d a ) , { a ,b ) ) d PC b. = c pca a , , , ¢( b. b) c ) 01 a. = { ¢Cc,d)¢ a ) ( d. * c) a.

( General Constructing Case ) Junction Trees Solution by Triangulate diagonals graph adding : loops all to 4 size

Greedy Variable Elimination fewest Strategy Eliminate variable adds that edges : at step each

. Trees Junction Computational Complexity : Network Markov Junction Tree . Exponential maximal in clique size Exponential clique Complexity in she : Can elimination depend order on - Not necessarily optimal complexity -