19 : Baham Sabbir Scribes : , Belief Propagation : Problem - - PowerPoint PPT Presentation

▶
19
SMART_READER_LITE
LIVE PREVIEW

19 : Baham Sabbir Scribes : , Belief Propagation : Problem - - PowerPoint PPT Presentation

Apr 04, 2024 219 likes •487 views

The Algorithm Lecture Junction Tree 19 : Baham Sabbir Scribes : , Belief Propagation : Problem The With Loops R far Graph Directed b ; a , b. d pca ) c clb d ) p ( b ( Pla ) dla l a) p( ) = p , A C p fz fu a. Fi

slide-1
SLIDE 1 Lecture

19

: The Junction Tree Algorithm Scribes : Baham , Sabbir
slide-2
SLIDE 2 Belief Propagation : The Problem With Loops a b Directed Graph ; pca , b. c d ) = Pla) p ( b l a) p( dla ) p ( clb , d ) A C
  • a. Fi
pfz Factor Graph ; a s a. s b pc a ,b , c d ) n Rfar ↳a•

fu

=
  • f. (a)
fz( a ,b ) f }( a. d) fu( b.
  • c. d)

3h

L I
  • C
Almost all Bayesian networks have loops =
slide-3
SLIDE 3 Loopy Belief Propagation Step 1 : Initialize Messages µ[ f. × ][ h ] = 1 He
  • x. f)
EE he{qB
  • a. fl
µ[ × ,f][ h ] = 1

fz

a a. b Step 2 : Update messages

f.

a. a.

fu

for f € SCHEDULE : a c for × e helf ) : Update µ×→t and µf→× Problem : Convergence not guaranteed
slide-4
SLIDE 4 Today : Junction Trees Idea : Transform graphical models With loops into clique trees without loops Markov Network Junction Tree Advantage : Can compute exact = marginal ,
slide-5
SLIDE 5 Reparameterisatim pcaib ) = plays ) PC b) Example : Markov Chain ) PC a , b , c , d ) = pcalb ) pcblc ) pccld ) pcd ) Repurametnisation : Express joint as b and c
  • ccur

twice

product
  • f

marginalsgyy

\

. p( a , b , c , d ) =

pca,b_7

Plb , c) plc ,d ) peb )

pet

Tya

, # it divide by plb ) pcc )
slide-6
SLIDE 6 Clique Graphs Definition : A clique is a fully
  • connected
Subset
  • f
variables in a graphical model Clique potentials a . 1 ¢ ( a. b. c) ¢( b. c. d) 4171 ' )¢(X2 7

p(

  • a. b.
c. d) =
  • =
  • 17
¢( b. c) 7 ¢( X 'RX2 ) In separator Potential X '={ a ,b ,c} 2/2 :{ b.
  • e. a }
7 ' X 'nX2 72
slide-7
SLIDE 7 Absorption : Normalizing Clique Graphs Goal : Transform clique graph to ensure that potentials take the form
  • f
marginal densities pcuuw ) = ¢( V ) ¢( W ) : ¢( s ) = = dcvsdcw ) marginal for cliques § ( S ) ( analogous to BP )
  • f ( V )
= pcu ) ¢( W ) = pcw ) ¢ ' (5)

=p(s7

slide-8
SLIDE 8 Absorption : Normalizing Clique Graphs p(

Uuw

) = ¢( V ) ¢( w ) ¢ ( s ) Marginal for V PIV ) = [

pcvuw

) = ¢( D)

£u¢(W)WiV

¢( s ) Absorption from W to V through 5 ¢*lv ) := 011 D) toys 4*15 ) = 2 ¢ ( w ) ¢ ( s ) WW ¢N)0(W)_t = ¢w )

10¥51

%)

=

4µW

) toys ) ¢( s )

¢*ys

) ¢( s )
slide-9
SLIDE 9 Absorption : Normalizing Clique Graphs * p( V. w ) = ¢( V ) ¢( W )

#

csl Absorption from V to W through S g*lw ) ÷ ¢ ( w )

0*4

¢**Cs ) ⇐ [ ¢*w ) . ¢* ( s ) 01W Preserves density * " ¢* * ' w ' =
  • ¥*yYy¢iwida¥÷y
, = 0401101W ' ¢ # * ( s ) ¢*( s ) j
slide-10
SLIDE 10 Absorption : Normalizing Clique Graphs p( V. w ) = ¢( V ) ¢ ( w ) ¢ ( s ) = §(v)§( w ) ¢(s7
  • f ( v )
= ¢*lV ) = ¢ ( U ) 44¥ , = plu ) ¢*ls ) = w{sQ( W ) §lw )

=g*lw)=¢(

w )

¢**cI=p(

w )
  • f

#

( s ) ⇐ fgs

#

V ) ¢ * ( s ) § ( s ) = ¢*% ) = pcs )
slide-11
SLIDE 11 Absorption : Normalizing Clique Graphs General Case : Absorb in both directions along each edge Schedule : Compute Incoming messages followed by
  • utgoing
messages
slide-12
SLIDE 12 Junction Trees ( Singly Connected ) pcxi , xz , X } , Xu ) = ¢ ( × , ,Xu 1 ¢( xz ,Xu ) ¢ ( xs ,×a\ pcxi , Xu ) = ¢ C × , ,xa ) ( § ¢( Xz , Xu ) ) ( §¢( × } , Xu ) ) pcxz , xu ) = ¢( xz , xu ) ( { ¢ ( × , ,xa1 ) ( § locks ,×u ) ) pcxs , xul = ¢1×3,41 ( § qk , ,×n ) ( { ¢( × , ,×a ) ) Markov Networh Clique Graph
slide-13
SLIDE 13 Junction Trees ( Singly Connected ) pcxi , xz , X } , Xu ) = ¢ ( × , ,Xu 1 ¢( xz ,Xu ) ¢ ( xs ,×a\ pcxi , xu ) = ¢cx ,,xu ) ( { Qcx . ,xn ) ) ( { ¢ ( x , in ) )

|p(

xz , xu ) = ¢( × . ,xa ) ( folk . ,×u ) ) ( §¢c× , ,×a ) )

p(

I.

' 4,41 = 011×3.41 ( § dcx , ,×n ) ) ( { ¢c xaxa ) ) PCX , ,X< , ) PCXZ , Xu ) p ( X } th , ) = ¢ ( x , ,Xu 1 ¢( xz ,Xu ) 01 ( xs ,xu\ 2

(( {

dcx . ,xa ) ) ( { ¢k},×n ) ) ( { dcx , ,xn ) )) = PCX , , Xz , X > , Xa , ) PCXL, )2
slide-14
SLIDE 14 Junction Trees ( Singly Connected ) pcxi , xz , X } , Xu ) = ¢ ( × , , Xu 1 ¢ ( xz , Xu ) ¢ ( xs

,×⇐

Clique Potentials
  • f ( X :)
= pc × , , ×µ ) pcxe , xul p(×3 , xa ) Separator Potentials 4^15 ' ' )
  • 10k€
pcxn )

www.w.ru#iEikEnYKYYkn.+iatru)

slide-15
SLIDE 15 Junction Trees ( Singly Connected ) Markov Network Clique Graph Junction Tree x General Property : When a variable x
  • ccurs
in all separators
  • n
a loop , then we may remove x from an arbitrarily chosen separator
slide-16
SLIDE 16 Junction Trees : Running Intersection Property Definition : A clique tree Is a junction tree if , for each pair
  • f
nodes V and W , all nodes
  • n
the path from U to W contain the intersection Vn W Consistency : For any pain
  • f
nodes V and W in a junction tree . the marginal for the intersection I = Un W satisfies the consistency condition # ( I ) = I ¢ ( 0 ) = [ 011 W ) Dlw W\U i ,
slide-17
SLIDE 17 Constructing Junction Trees ( Singly
  • connected )
Step 1 : Moralize a graph by adding edges between parents
slide-18
SLIDE 18 Constructing Junction Trees ( Singly
  • connected )
Step 2 : Define a clique graph

=

slide-19
SLIDE 19 Constructing Junction Trees ( Singly
  • connected )
Step 3 : Break loops to
  • btain
a Junction Tree X X Definition : Any maximal weight spanning tree is a junction tree . The weight
  • f
a tree is defined as the sum
  • ver
curdinalittes
  • f
separators ,
slide-20
SLIDE 20 Constructing Junction Trees ( Singly
  • connected )
Step 4 : Define potentials p( a. b. c. d. e. f , g) = p (

a)

Pcb

) p (

clb

, a )

.

( a. b. c)

pcdsplelcidd

( C , d. e)

pcflc

)

plgle

)

plhle

)
  • ¢lc
, f) ¢(
  • e. g)
¢( e. h ) ¢ ( c ) = 1 ¢ (e) =|
slide-21
SLIDE 21 Constructing Junction Trees ( Singly
  • connected )
Step 5 : Propagate messages according to an absorption schedule

y

\u

dlxi ) → ECX'l=p( F)

µ

§

£

¢ 's ' ' → ¢csis= ,.es ;)

*

slide-22
SLIDE 22 Constructing Junction Trees ( General Case ) Problem : Marginalization can introduce edges pc a ,b , c. d) = ¢( a , b) 4lb ,c ) ¢( c. d ) ¢ ( d , a ) PC a ,b , c ) = { pca , b. a , d ) = 01 a. b) ¢( b. c )

{

¢Cc,d)¢ ( d. a )

*

a. c)
slide-23
SLIDE 23 Constructing Junction Trees ( General Case ) Solution : Triangulate graph by adding diagonals to all size 4 loops
slide-24
SLIDE 24 Greedy Variable Elimination Strategy : Eliminate variable that adds fewest edges at each step
slide-25
SLIDE 25 . Junction Trees : Computational Complexity Markov Network Junction Tree . Exponential in maximal clique size Complexity : Exponential in clique she
  • Can
depend
  • n
elimination
  • rder
  • Not
necessarily
  • ptimal
complexity