Re Reas ason onin ing g Un Unde der Un Uncerta tain inty - PowerPoint PPT Presentation

Re Reas ason onin ing g Un Unde der Un Uncerta tain inty ty: Varia Va iabl ble eli limin inat atio ion Com omputer Science c cpsc sc322, Lecture 3 30 (Te Text xtboo ook k Chpt 6.4) June, 20 20, 2 2017 CPSC 322, Lecture 30 Slide 1

Lectu ture re Ov Overv rvie iew • Recap p Intr tro o Varia iabl ble Eli limin inati tion on • Variable Elimination • Simplifications • Example • Independence • Where are we? CPSC 322, Lecture 30 Slide 2

Bnet t Infe ference: Ge General • Suppose the variables of the belief network are X 1 ,…, X n . • Z is the query variable • Y 1 =v 1 , …, Y j =v j are the observed variables (with their values) • Z 1 , …, Z k are the remaining variables   P ( Z | Y v , , Y v )  • What we want to compute: 1 1 j j   P ( Z , Y v , , Y v )  • We can actually compute: 1 1 j j     P ( Z , Y v , , Y v ) P ( Z , Y v , , Y v )      1 1 j j  1 1 j j P ( Z | Y v , , Y v )   1 1 j j     P ( Y v , , Y v ) P ( Z , Y v , , Y v )   1 1 j j 1 1 j j Z CPSC 322, Lecture 29 Slide 3

Infe ference wit ith Fa Facto tors We can compute P(Z, Y 1 =v 1 , …,Y j =v j ) by • expressing the joint as a factor, f (Z, Y 1 …,Y j , Z 1 …,Z j ) • ass ssign gning Y 1 =v 1 , …, Y j =v j • and su summing o g out the variables Z 1 , …,Z k      P ( Z , Y v , , Y v ) f ( Z , Y ,.., Y , Z ,.., Z )     1 1 j j 1 j 1 k Y v , , Y v  1 1 j j Z Z k 1 CPSC 322, Lecture 29 Slide 4

Varia iabl ble Eli limin inati tion on Intr tro o (1 (1)      P ( Z , Y v , , Y v ) f ( Z , Y ,.., Y , Z ,.., Z )     1 1 j j 1 j 1 k Y v , , Y v  1 1 j j Z Z k 1 • Using the chain r rule and the definition on of of a Bn Bnet, we n  can write P(X 1 , …, X n ) as P ( X i pX | ) i  i 1 • We can express the joint factor as a product of factors n  f ( X i pX , ) f(Z, Y 1 …, Y j , Z 1 …, Z j ) i  i 1 n      P ( Z , Y v , , Y v ) f ( X , pX )   1 1 j j i i    Z Z i 1 Y v , , Y v  k 1 1 1 j j CPSC 322, Lecture 29 Slide 5

Varia iabl ble Eli limin inati tion on Intr tro o (2 (2) Inference in belief networks thus reduces to computing “the sums of products….” n      P ( Z , Y v , , Y v ) f ( X , pX )   1 1 j j i i  Z Z i 1   Y v , , Y v  k 1 1 1 j j 1. Construct a factor for each conditional probability. 2. In each factor assign the observed variables to their observed values. 3. Multiply the factors 4. For each of the other variables Z i ∈ {Z 1 , …, Z k } , sum out Z i CPSC 322, Lecture 29 Slide 6

Lectu ture re Ov Overv rvie iew • Recap p Intr tro o Varia iabl ble Eli limin inati tion on • Variable Elimination • Simplifications • Example • Independence • Where are we? CPSC 322, Lecture 30 Slide 7

Ho How to to si simpl plif ify th the Co Compu puta tati tion on? • Assume we have turned the CPTs into factors and performed the assignments  f ( X , pX ) f ( varsX ) i i i  f ( C , D , G ) ?  t G n n     f ( X , pX )  f ( varsX i )    i i Y v , , Y v  1 1 j j  Z Z i 1  Z Z i 1 k 1 k 1 Let’s focus on the basic case, for instance…     f ( C , D ) f ( A , B , D ) f ( E , A ) f ( D ) A CPSC 322, Lecture 30 Slide 8

Ho How to to si simpl plif ify: ba basi sic case se n  Let’s focus on the basic case. f ( varsX i )  Z i 1 1     f ( C , D ) f ( A , B , D ) f ( E , A ) f ( D ) A • How can we compute efficiently? Factor out those terms that don't involve Z 1 !             f ( varsX ) f ( varsX )     i i       i | Z varsXi Z i | Z varsXi 1 1 1 CPSC 322, Lecture 30 Slide 9

Ge General l case se: Su Summin ing g out t varia iable les s effi fficie ientl tly                 f f ( f f ) f f         1 h 1 i i 1 h   Z Z Z Z Z k 1 k 2 1       ... f f f  1 i Z Z k 2 Now to sum out a variable Z 2 from a product f 1 × … ×f i × f’ of factors, again partition the factors into two sets • F: those that • F: those that CPSC 322, Lecture 30 Slide 10

Analogy with “Computing sums of products” Th This s si simplification on is s si similar to o what you ou can d do o in b basi sic alge gebra with multiplication on and addition on • It takes 14 multiplications or additions to evaluate the expression a b + a c + a d + a e h + a f h + a g h . • This expression be evaluated more efficiently…. CPSC 322, Lecture 30 Slide 1 1

Varia iabl ble eli limin inati tion on or orde derin ing Is there only one way to simplify? P(G,D=t) =  A,B,C, f(A,G) f(B,A) f(C,G) f(B,C) P(G,D=t) =  A f(A,G)  B f(B,A)  C f(C,G) f(B,C) P(G,D=t) =  A f(A,G)  C f(C,G)  B f(B,C) f(B,A) CPSC 322, Lecture 30 Slide 12

Var aria iable le eli limi minat atio ion al algo gori rith thm: m: Su Summ mmar ary P(Z (Z, , Y 1 …, Y j , , Z 1 …, Z j ) To To c compu pute P(Z (Z| Y Y 1 =v =v 1 , … ,Y j =v j ) ) : 1. Construct a factor for each conditional probability. 2. Set the observed variables to their observed values. 3. Given an elimination ordering, simplify/decompose sum of products B. Y 2 4. Perform products and sum out Z i A. Y 1 =v =v 1 5. Multiply the remaining factors (all in ? ) C. Z 2 D. Z 6. Normalize: divide the resulting factor f(Z) by  Z f(Z) . CPSC 322, Lecture 10 Slide 13

Var aria iable le eli limi minat atio ion al algo gori rith thm: m: Su Summ mmar ary P(Z (Z, , Y 1 …, Y j , , Z 1 …, Z j ) To To c compu pute P(Z (Z| Y Y 1 =v =v 1 , … ,Y j =v j ) ) : 1. Construct a factor for each conditional probability. 2. Set the observed variables to their observed values. 3. Given an elimination ordering, simplify/decompose sum of products 4. Perform products and sum out Z i 5. Multiply the remaining factors (all in ? ) Z 6. Normalize: divide the resulting factor f(Z) by  Z f(Z) . CPSC 322, Lecture 10 Slide 14

Lectu ture re Ov Overv rvie iew • Re Recap p Intr tro o Varia iabl ble Eli limin inati tion on • Variable Elimination • Simplifications • Example • Independence • Where are we? CPSC 322, Lecture 30 Slide 15

Var aria iable le eli limi minat atio ion exa xamp mple le Compute P(G | H=h 1 ) . P(G,H) =  A,B,C,D,E,F,I P(A,B,C,D,E,F,G,H,I) • CPSC 322, Lecture 30 Slide 16

Var aria iable le eli limi minat atio ion exa xamp mple le Compute P(G | H=h 1 ) . P(G,H) =  A,B,C,D,E,F,I P(A,B,C,D,E,F,G,H,I) • Chain Rule + Conditional Independence: P(G,H) =  A,B,C,D,E,F,I P(A)P(B|A)P(C)P(D|B,C)P(E|C)P(F|D)P(G|F,E)P(H|G)P(I|G) CPSC 322, Lecture 30 Slide 17

Var aria iable le eli limi minat atio ion exa xamp mple le (s (ste tep1) Compute P(G | H=h 1 ) . P(G,H) =  A,B,C,D,E,F,I P(A)P(B|A)P(C)P(D|B,C)P(E|C)P(F|D)P(G|F,E)P(H|G)P(I|G) • Factorized Representation: P(G,H) =  A,B,C,D,E,F,I f 0 (A) f 1 (B,A) f 2 (C) f 3 (D,B,C) f 4 (E,C) f 5 (F, D) f 6 (G,F,E) f 7 (H,G) f 8 (I,G) • f 0 (A) • f 1 (B,A) • f 2 (C) • f 3 (D,B,C) • f 4 (E,C) • f 5 (F, D) • f 6 (G,F,E) • f 7 (H,G) CPSC 322, Lecture 30 Slide 18 • f 8 (I,G)

Var aria iable le eli limi minat atio ion exa xamp mple le (s (ste tep 2) Compute P(G | H=h 1 ) . Previous state: P(G,H) =  A,B,C,D,E,F,I f 0 (A) f 1 (B,A) f 2 (C) f 3 (D,B,C) f 4 (E,C) f 5 (F, D) f 6 (G,F,E) f 7 (H,G) f 8 (I,G) Observe H : P(G,H=h 1 ) =  A,B,C,D,E,F,I f 0 (A) f 1 (B,A) f 2 (C) f 3 (D,B,C) f 4 (E,C) f 5 (F, D) f 6 (G,F,E) f 9 (G) f 8 (I,G) • f 0 (A) • f 9 (G) • f 1 (B,A) • f 2 (C) • f 3 (D,B,C) • f 4 (E,C) • f 5 (F, D) • f 6 (G,F,E) • f 7 (H,G) • f 8 (I,G) CPSC 322, Lecture 30 Slide 19

Var aria iable le eli limi minat atio ion exa xamp mple le (s (ste teps s 3-4) 4) Compute P(G | H=h 1 ) . Previous state: P(G,H) =  A,B,C,D,E,F,I f 0 (A) f 1 (B,A) f 2 (C) f 3 (D,B,C) f 4 (E,C) f 5 (F, D) f 6 (G,F,E) f 9 (G) f 8 (I,G) Elimination ordering A, C, E, I, B, D, F : P(G,H=h 1 ) = f 9 (G)  F  D f 5 (F, D)  B  I f 8 (I,G)  E f 6 (G,F,E)  C f 2 (C) f 3 (D,B,C) f 4 (E,C)  A f 0 (A) f 1 (B,A) • f 9 (G) • f 0 (A) • f 1 (B,A) • f 2 (C) • f 3 (D,B,C) • f 4 (E,C) • f 5 (F, D) • f 6 (G,F,E) • f 7 (H,G) • f 8 (I,G) CPSC 322, Lecture 30 Slide 20