Graphical Models Graphical Models Conditional Independence 1 - - PowerPoint PPT Presentation

▶

Jul 16, 2023 477 likes •557 views

Conditional Independence d-Separation Belief Propogation Conditional Independence d-Separation Belief Propogation Graphical Models Graphical Models Conditional Independence 1 Steven J Zeil d-Separation 2 Old Dominion Univ. Fall 2010

SLIDE 1

Conditional Independence d-Separation Belief Propogation

Graphical Models

Steven J Zeil

Old Dominion Univ.

Fall 2010

1 Conditional Independence d-Separation Belief Propogation

Graphical Models

1

Conditional Independence

2

d-Separation

3

Belief Propogation

2 Conditional Independence d-Separation Belief Propogation

Graphical Models

a.k.a. Bayesian networks, probabilistic networks Nodes are hypotheses (random vars)

Values are the probabilities of the observed value of that variable

Arcs are direct influences between hypotheses Forms a directed acyclic graph (DAG) The parameters are the conditional probabilities in the arcs

3 Conditional Independence d-Separation Belief Propogation

Example

Knowing that the grass is wet, what is the probability that rain is the cause? P(R|W ) = P(W |R)P(R) P(W ) = P(W |R)P(R) P(W |R)P(R) + P(W |¬R)(P(¬R) = 0.9 × 0.4 0.9 × 0.4 + 0.2 × 0.6) = 0.75

SLIDE 2

Conditional Independence d-Separation Belief Propogation

Causes & Diagnoses

Graph shows a causal relationship. Bayes rules “reverses” the arc to give a diagnosis.

5 Conditional Independence d-Separation Belief Propogation

Conditional Independence

X and Y are independent if P(X, Y ) = P(X)P(Y ) X and Y are conditionally independent given Z if P(X, Y |Z) = P(X|Z)P(Y |Z)

P(X|Y , Z) = P(X|Z) Three canonical cases: Head-to-tail, Tail-to-tail, head-to-head

6 Conditional Independence d-Separation Belief Propogation

Head-to-Tail

P(X, Y , Z) = P(X)P(Y |X)P(Z|Y ) P(W |C) = P(W |R)P(R|C) + P(W |¬R)P(¬R|C)

7 Conditional Independence d-Separation Belief Propogation

Blocking

If we know the state of Y, we know everything we can about Z without knowingthe state of X. We say that Y blocks the path from X to Z

r, Y separates X and Z.

SLIDE 3

Conditional Independence d-Separation Belief Propogation

Tail-to-Tail

P(X, Y , Z) = P(X)P(Y |X)P(Z|X) An observed X blocks the path between Y and Z: P(X, Y |X) = P(X, Y , Z) P(X) = P(X)P(Y |X)P(Z|X) P(X) = P(Y |X)P(Z|X)

9 Conditional Independence d-Separation Belief Propogation

Head-to-Head

P(X, Y , Z) = P(X)P(Y )P(Z|X, Y ) Z blocks the path between X and Y when it is unobserved.

10 Conditional Independence d-Separation Belief Propogation

Causal vs Diagnostic

Causal inference: If the sprinkler is on, what is the probability that the grass is wet? (P(W |S)) Diagnostic inference: If the grass is wet, what is the probability that the sprinkler is on? P(S, W ) = P(W |S)P(S) P(W ) = 0.35

11 Conditional Independence d-Separation Belief Propogation

Explaining Away

Suppose that we know that it rained: P(S|R, W ) = P(W |R, S)P(S|R) P(W |R = P(W |R, S)P(S) P(W |R = 0.21 Note that P(S|R, W ) < P(S|W ). Explaining Away: Knowing that it rained, the prob that the sprinkler caused the wet grass is decreased.

SLIDE 4

Conditional Independence d-Separation Belief Propogation

Larger Systems

Larger systems formed by combining the three basic subgraphs Provides a structure & explanation of complicated relationships This graph describes P(C, S, R, W , F) How would you compute P(F|C)?

13 Conditional Independence d-Separation Belief Propogation

Example: Classification

Causal relation P(x|C) Bayes’ rule inverts P(C|x) = p(x|C)P(C) p(x)

14 Conditional Independence d-Separation Belief Propogation

Example: Naive Bayes Classification

Given C, the xj are independent P( x|C) = p(x1|C)p(x2|C) . . . p(xd|C)

15 Conditional Independence d-Separation Belief Propogation

Example: Hidden Markov

State at time t depends only on state at time t − 1 Output depends only on the current state

SLIDE 5

Conditional Independence d-Separation Belief Propogation

Path Blocking

A path from node A to node B is blocked given {C} if

The directions of edges on the path meet head-to-tail (case 1) or tail-to-tail (case 2) at a node in C, or The directions of edges meet head-to-head (case 3) and neither that node nor any of its descendants is in C.

Examples

BCDF is blocked given C BEFG is blocked given E or F BEFD is blocked unless F (or G) is given

17 Conditional Independence d-Separation Belief Propogation

d-Separation

If all paths from A to B are blocked given C, A and B are d-separated (conditionally independent) given C.

18 Conditional Independence d-Separation Belief Propogation

Belief Propogation

Use graph-based algorithms to answer queries of the form P(X|E) where The query node X is any node in the graph E is a set of evidence nodes whose values are known

19 Conditional Independence d-Separation Belief Propogation

Chains

Evidence E + in ancestors of X will flow along as diagnostic inference Evidence E − in decendents of X will flow back as causal inference E + and E − separate X from any more nodes in the chain, so we have at most two evidence nodes to consider

SLIDE 6

Conditional Independence d-Separation Belief Propogation

Chains: Propogated Info

For each node N, λ(N) = P(E −|N) π(N) = P(N|E +)

21 Conditional Independence d-Separation Belief Propogation

Chains: Meeting at the Middle

P(X|E) = P(E|X)P(X) P(E) = P(E +, E −|X)P(X) P(E) = P(E +|X)P(E −|X)P(X) P(E) = P(X|E +)P(E +)P(E −|X)P(X) P(X)P(E) = αP(X|E +)P(E −|X) = απ(X)λ(X)

22 Conditional Independence d-Separation Belief Propogation

Chains: Updating

P(X|E) = απ(X)λ(X) π(X) =

P(X|U)π(U) λ(X) =

P(Y |X)λ(Y )

23 Conditional Independence d-Separation Belief Propogation

Trees

λ(X) = P(E −

X |X)

= λY (X)λZ(X) λX(U) =

λ(X)P(X|U) π(X) = P(X|E +

X )

=

P(X|U)πX(U) πY (X) = αλZ(X)π(X)

SLIDE 7

Conditional Independence d-Separation Belief Propogation

Polytrees

π(X) = P(X|E +

X )

=

. . .

P(X|U1, U2, . . . , Uk)

k

πX(Ui) πYj(X) = α

λYs(X)π(X)

25 Conditional Independence d-Separation Belief Propogation

Polytrees

λX(Ui) = β

λ(X)

P(X|U1, U2, . . . , Uk)

πX(Ur) λ(X) =

m

λYj(X)

26 Conditional Independence d-Separation Belief Propogation