ECE 4524 Artificial Intelligence and Engineering Applications - - PowerPoint PPT Presentation

ECE 4524 Artificial Intelligence and Engineering Applications

Lecture 19: Bayesian Networks Reading: AIAMA 14.1-14.4 Today’s Schedule:

◮ Bayesian Networks ◮ Causal Reasoning ◮ Exact Inference

Bayesian Networks (BN)

◮ A BN is a directed acyclic graph whose nodes are random

variables.

◮ The nodes are annotated with the conditional probability of

that node given its parents in the network.

◮ The graph structure encodes the assumptions of the joint

distribution/density and

◮ encodes the causal structure of the KB.

How do we guarantee a graph is a valid encoding of the joint probability?

Recall the product rule P(X1, X2, · · · , Xn) = P(Xn)

n−1

• i=1

P(Xi|Xi−1, · · · , Xn)

◮ This defines the graph structure ◮ Some ordering of the variables is better than others in the

sense it leads to graphs of less complexity.

BN Topology

The topology of the graph is related to the conditional independence of variables. Define the Markov blanket for node Xi as the set M(Xi) ≡ {Parents(Xi), Children(Xi), Parents(Children(Xi))} Then Xi is conditionally independent of any variable not in M(Xi) given the variables in M(Xi).

The nodes are annotated with the Conditional Probability Table (CPT)

There are 3 cases

◮ Discrete R.V.s ◮ Continuous R.V.s ◮ Hybrid Networks (mixtures of Discrete and Continuous R.V.s)

The simplest case is for Bernoulli R.V.s with a ”noisy or”.

Doing Inference

There are two approaches

◮ exact (today) ◮ approximate (next time)

First some terminology. We partition the variables into 3 sets

◮ Evidence Variables,E ◮ Hidden Variables,Y ◮ Query Variables,X

Inference is the procedure to answer the query P(X|E)

For Discrete BN

P(X|E) = P(X, E) P(E) = 1 P(E)

• y

P(X, E, Y ) We marginalize over the hidden variables.

Complexity of Exact Inference

◮ for singly-connected graphs (polytrees) it is linear in the

number of CPT entries.

◮ for general BNs it is in #P-Hard, strictly harder than

NP-complete problems This is why approximation algorithms are important.

Next Actions

◮ Reading on Approximate Inference (AIAMA 14.5)