Graphical Models Steven J Zeil Old Dominion Univ. Fall 2010 1 - - PowerPoint PPT Presentation

▶
graphical models
SMART_READER_LITE
LIVE PREVIEW

Graphical Models Steven J Zeil Old Dominion Univ. Fall 2010 1 - - PowerPoint PPT Presentation

Dec 31, 2023 292 likes •586 views

Conditional Independence d-Separation Belief Propogation Graphical Models Steven J Zeil Old Dominion Univ. Fall 2010 1 Conditional Independence d-Separation Belief Propogation Graphical Models Conditional Independence 1 d-Separation 2

slide-1
SLIDE 1

Conditional Independence d-Separation Belief Propogation

Graphical Models

Steven J Zeil

Old Dominion Univ.

Fall 2010

1

slide-2
SLIDE 2

Conditional Independence d-Separation Belief Propogation

Graphical Models

1

Conditional Independence

2

d-Separation

3

Belief Propogation

2

slide-3
SLIDE 3

Conditional Independence d-Separation Belief Propogation

Graphical Models

a.k.a. Bayesian networks, probabilistic networks Nodes are hypotheses (random vars)

Values are the probabilities of the observed value of that variable

Arcs are direct influences between hypotheses Forms a directed acyclic graph (DAG) The parameters are the conditional probabilities in the arcs

3

slide-4
SLIDE 4

Conditional Independence d-Separation Belief Propogation

Example

Knowing that the grass is wet, what is the probability that rain is the cause? P(R|W ) = P(W |R)P(R) P(W ) = P(W |R)P(R) P(W |R)P(R) + P(W |¬R)(P(¬R) = 0.9 × 0.4 0.9 × 0.4 + 0.2 × 0.6) = 0.75

4

slide-5
SLIDE 5

Conditional Independence d-Separation Belief Propogation

Causes & Diagnoses

Graph shows a causal relationship. Bayes rules “reverses” the arc to give a diagnosis.

5

slide-6
SLIDE 6

Conditional Independence d-Separation Belief Propogation

Conditional Independence

X and Y are independent if P(X, Y ) = P(X)P(Y ) X and Y are conditionally independent given Z if P(X, Y |Z) = P(X|Z)P(Y |Z)

  • r

P(X|Y , Z) = P(X|Z) Three canonical cases: Head-to-tail, Tail-to-tail, head-to-head

6

slide-7
SLIDE 7

Conditional Independence d-Separation Belief Propogation

Head-to-Tail

P(X, Y , Z) = P(X)P(Y |X)P(Z|Y ) P(W |C) = P(W |R)P(R|C) + P(W |¬R)P(¬R|C)

7

slide-8
SLIDE 8

Conditional Independence d-Separation Belief Propogation

Blocking

If we know the state of Y, we know everything we can about Z without knowingthe state of X. We say that Y blocks the path from X to Z

  • r, Y separates X and Z.

8

slide-9
SLIDE 9

Conditional Independence d-Separation Belief Propogation

Tail-to-Tail

P(X, Y , Z) = P(X)P(Y |X)P(Z|X) An observed X blocks the path between Y and Z: P(X, Y |X) = P(X, Y , Z) P(X) = P(X)P(Y |X)P(Z|X) P(X) = P(Y |X)P(Z|X)

9

slide-10
SLIDE 10

Conditional Independence d-Separation Belief Propogation

Head-to-Head

P(X, Y , Z) = P(X)P(Y )P(Z|X, Y ) Z blocks the path between X and Y when it is unobserved.

10

slide-11
SLIDE 11

Conditional Independence d-Separation Belief Propogation

Causal vs Diagnostic

Causal inference: If the sprinkler is on, what is the probability that the grass is wet? (P(W |S)) Diagnostic inference: If the grass is wet, what is the probability that the sprinkler is on? P(S, W ) = P(W |S)P(S) P(W ) = 0.35

11

slide-12
SLIDE 12

Conditional Independence d-Separation Belief Propogation

Explaining Away

Suppose that we know that it rained: P(S|R, W ) = P(W |R, S)P(S|R) P(W |R = P(W |R, S)P(S) P(W |R = 0.21 Note that P(S|R, W ) < P(S|W ). Explaining Away: Knowing that it rained, the prob that the sprinkler caused the wet grass is decreased.

12

slide-13
SLIDE 13

Conditional Independence d-Separation Belief Propogation

Larger Systems

Larger systems formed by combining the three basic subgraphs Provides a structure & explanation of complicated relationships This graph describes P(C, S, R, W , F) How would you compute P(F|C)?

13

slide-14
SLIDE 14

Conditional Independence d-Separation Belief Propogation

Example: Classification

Causal relation P(x|C) Bayes’ rule inverts P(C|x) = p(x|C)P(C) p(x)

14

slide-15
SLIDE 15

Conditional Independence d-Separation Belief Propogation

Example: Naive Bayes Classification

Given C, the xj are independent P( x|C) = p(x1|C)p(x2|C) . . . p(xd|

15

slide-16
SLIDE 16

Conditional Independence d-Separation Belief Propogation

Example: Hidden Markov

State at time t depends only on state at time t − 1 Output depends only on the current state

16

slide-17
SLIDE 17

Conditional Independence d-Separation Belief Propogation

Path Blocking

A path from node A to node B is blocked given {C} if

The directions of edges on the path meet head-to-tail (case 1) or tail-to-tail (case 2) at a node in C, or The directions of edges meet head-to-head (case 3) and neither that node nor any of its descendants is in C.

Examples

BCDF is blocked given C BEFG is blocked given E or F BEFD is blocked unless F (or G) is given

17

slide-18
SLIDE 18

Conditional Independence d-Separation Belief Propogation

d-Separation

If all paths from A to B are blocked given C, A and B are d-separated (conditionally independent) given C.

18

slide-19
SLIDE 19

Conditional Independence d-Separation Belief Propogation

Belief Propogation

Use graph-based algorithms to answer queries of the form P(X|E) where The query node X is any node in the graph E is a set of evidence nodes whose values are known

19

slide-20
SLIDE 20

Conditional Independence d-Separation Belief Propogation

Chains

Evidence E + in ancestors of X will flow along as diagnostic inference Evidence E − in decendents of X will flow back as causal inference E + and E − separate X from any more nodes in the chain, so we have at most two evidence nodes to consider

20

slide-21
SLIDE 21

Conditional Independence d-Separation Belief Propogation

Chains: Propogated Info

For each node N, λ(N) = P(E −|N) π(N) = P(N|E +)

21

slide-22
SLIDE 22

Conditional Independence d-Separation Belief Propogation

Chains: Meeting at the Middle

P(X|E) = P(E|X)P(X) P(E) = P(E +, E −|X)P(X) P(E) = P(E +|X)P(E −|X)P(X) P(E) = P(X|E +)P(E +)P(E −|X)P(X) P(X)P(E) = αP(X|E +)P(E −|X) = απ(X)λ(X)

22

slide-23
SLIDE 23

Conditional Independence d-Separation Belief Propogation

Chains: Updating

P(X|E) = απ(X)λ(X) π(X) =

  • U

P(X|U)π(U) λ(X) =

  • Y

P(Y |X)λ(Y )

23

slide-24
SLIDE 24

Conditional Independence d-Separation Belief Propogation

Trees

λ(X) = P(E −

X |X)

= λY (X)λZ(X) λX(U) =

  • X

λ(X)P(X|U) π(X) = P(X|E +

X )

=

  • U

P(X|U)πX(U πY (X) = αλZ(X)π(X)

24

slide-25
SLIDE 25

Conditional Independence d-Separation Belief Propogation

Polytrees

π(X) = P(X|E +

X )

=

  • U1
  • U2

. . .

  • Uk

P(X|U1, U2, . . . , Uk)

k

  • j=1

πX(Ui) πYj(X) = α

  • x=j

λYs(X)π(X)

25

slide-26
SLIDE 26

Conditional Independence d-Separation Belief Propogation

Polytrees

λX(Ui) = β

  • X

λ(X)

  • r=i

P(X|U1, U2, . . . , Uk)

  • r=i

πX(Ur) λ(X) =

m

  • j=1

λYj(X)

26

slide-27
SLIDE 27

Conditional Independence d-Separation Belief Propogation

Junction Trees

If X does not separate E + and E − (e.g., loops in dependencies)

Moralize the graph by joining all nodes that have common children Identify cliques Embed cliques into single nodes to form a junction tree

Each compressed node is a separately solvable subproblem

27