CS 730/830: Intro AI Bayesian Networks Approx. Inference Exact - - PowerPoint PPT Presentation

▶
cs 730 830 intro ai
SMART_READER_LITE
LIVE PREVIEW

CS 730/830: Intro AI Bayesian Networks Approx. Inference Exact - - PowerPoint PPT Presentation

Mar 06, 2023 140 likes •380 views

CS 730/830: Intro AI Bayesian Networks Approx. Inference Exact Inference Wheeler Ruml (UNH) Lecture 23, CS 730 1 / 17 Bayesian Networks Models Example The Joint Independence Example Break Approx. Inference Exact

slide-1
SLIDE 1

CS 730/830: Intro AI

Bayesian Networks

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 1 / 17

slide-2
SLIDE 2

Bayesian Networks

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 2 / 17

slide-3
SLIDE 3

Probabilistic Models

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 3 / 17

MDPs: Naive Bayes: k-Means: Representation: variables, connectives Inference: approximate, exact

slide-4
SLIDE 4

The Alarm Domain

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 4 / 17

slide-5
SLIDE 5

The Full Joint Distribution

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 5 / 17

ultimate power: knowing the probability of every possible atomic event (combination of values)

slide-6
SLIDE 6

The Full Joint Distribution

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 5 / 17

ultimate power: knowing the probability of every possible atomic event (combination of values) simple inference via enumeration over the joint: what is distribution of X given evidence e and unobserved Y P(X|e) = P(e|X)P(X) P(e) = αP(X, e) = α

  • y

P(X, e, y) Bayes Net = joint probability distribution

slide-7
SLIDE 7

The Magic of Independence

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 6 / 17

In general: P(x1, . . . , xn) = P(xn|xn−1, . . . , x1)P(xn−1, . . . , x1)

slide-8
SLIDE 8

The Magic of Independence

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 6 / 17

In general: P(x1, . . . , xn) = P(xn|xn−1, . . . , x1)P(xn−1, . . . , x1) =

n

  • i=1

P(xi|xi−1, . . . , x1)

slide-9
SLIDE 9

The Magic of Independence

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 6 / 17

In general: P(x1, . . . , xn) = P(xn|xn−1, . . . , x1)P(xn−1, . . . , x1) =

n

  • i=1

P(xi|xi−1, . . . , x1) A Bayesian net specifies independence: P(Xi|Xi−1, . . . , X1) = P(Xi|parents(Xi))

slide-10
SLIDE 10

The Magic of Independence

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 6 / 17

In general: P(x1, . . . , xn) = P(xn|xn−1, . . . , x1)P(xn−1, . . . , x1) =

n

  • i=1

P(xi|xi−1, . . . , x1) A Bayesian net specifies independence: P(Xi|Xi−1, . . . , X1) = P(Xi|parents(Xi)) So joint distribution can be computed as P(x1, . . . , xn) =

n

  • i=1

P(xi|parents(Xi))

slide-11
SLIDE 11

The Magic of Independence

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 6 / 17

In general: P(x1, . . . , xn) = P(xn|xn−1, . . . , x1)P(xn−1, . . . , x1) =

n

  • i=1

P(xi|xi−1, . . . , x1) A Bayesian net specifies independence: P(Xi|Xi−1, . . . , X1) = P(Xi|parents(Xi)) So joint distribution can be computed as P(x1, . . . , xn) =

n

  • i=1

P(xi|parents(Xi)) For n b-ary variables with p parents, that’s nbp instead of bn!

slide-12
SLIDE 12

Example

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 7 / 17

slide-13
SLIDE 13

Break

Bayesian Networks ■ Models ■ Example ■ The Joint ■ Independence ■ Example ■ Break

  • Approx. Inference

Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 8 / 17

■

asst 12

■

project

slide-14
SLIDE 14

Approximate Inference

Bayesian Networks

  • Approx. Inference

■ Sampling ■ Likelihood Wting Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 9 / 17

slide-15
SLIDE 15

Rejection Sampling

Bayesian Networks

  • Approx. Inference

■ Sampling ■ Likelihood Wting Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 10 / 17

What is distribution of X given evidence e and unobserved Y ? Draw worlds from the joint, rejecting those that do not match e. Look at distribution of X. sample values for variables, working top down directly implements the semantics of the network ‘generative model’ each sample is linear time, but overall slow if e is unlikely

slide-16
SLIDE 16

Likelihood Weighting

Bayesian Networks

  • Approx. Inference

■ Sampling ■ Likelihood Wting Exact Inference

Wheeler Ruml (UNH) Lecture 23, CS 730 – 11 / 17

What is distribution of X given evidence e and unobserved Y ? ChooseSample (e) w ← 1 for each variable Vi in topological order: if (Vi = vi) ∈ e then w ← w · P(vi|parents(vi)) else vi ← sample from P(Vi|parents(Vi)) (afterwards, normalize samples so all w’s sum to 1) uses all samples, but needs lots of samples if e are late in ordering

slide-17
SLIDE 17

Exact Inference in Bayesian Networks

Bayesian Networks

  • Approx. Inference

Exact Inference ■ Enumeration ■ Example ■ Var. Elim. 1 ■ Var. Elim. 2 ■ EOLQs

Wheeler Ruml (UNH) Lecture 23, CS 730 – 12 / 17

slide-18
SLIDE 18

Enumeration Over the Joint Distribution

Bayesian Networks

  • Approx. Inference

Exact Inference ■ Enumeration ■ Example ■ Var. Elim. 1 ■ Var. Elim. 2 ■ EOLQs

Wheeler Ruml (UNH) Lecture 23, CS 730 – 13 / 17

What is distribution of X given evidence e and unobserved Y ? P(X|e) = P(e|X)P(X) P(e) = αP(X, e) = α

  • y

P(X, e, y) = α

  • y

n

  • i=1

P(Vi|parents(Vi))

slide-19
SLIDE 19

Example

Bayesian Networks

  • Approx. Inference

Exact Inference ■ Enumeration ■ Example ■ Var. Elim. 1 ■ Var. Elim. 2 ■ EOLQs

Wheeler Ruml (UNH) Lecture 23, CS 730 – 14 / 17

P(B|j, m) = P(j, m|B)P(B) P(j, m) = αP(B, j, m) = α

  • e
  • a

P(B, e, a, j, m) = α

  • e
  • a

n

  • i=1

P(Vi|parents(Vi)) P(b|j, m) = α

  • e
  • a

P(b)P(e)P(a|b, e)P(j|a)P(m|a) = αP(b)

  • e

P(e)

  • a

P(a|b, e)P(j|a)P(m|a) [draw tree]

slide-20
SLIDE 20

Variable Elimination

Bayesian Networks

  • Approx. Inference

Exact Inference ■ Enumeration ■ Example ■ Var. Elim. 1 ■ Var. Elim. 2 ■ EOLQs

Wheeler Ruml (UNH) Lecture 23, CS 730 – 15 / 17

P(B|j, m) = αP(B)

  • e

P(e)

  • a

P(a|B, e)P(j|a)P(m|a) factors = tables = fvarsused(dimensions). eg: fA(A, B, E), fM(A) multiplying factors: table with union of variables summing reduces table

slide-21
SLIDE 21

Variable Elimination

Bayesian Networks

  • Approx. Inference

Exact Inference ■ Enumeration ■ Example ■ Var. Elim. 1 ■ Var. Elim. 2 ■ EOLQs

Wheeler Ruml (UNH) Lecture 23, CS 730 – 16 / 17

eliminating variables: eg P(J|b) P(J|b) = αP(b)

  • e

P(e)

  • a

P(a|b, e)P(J|a)

  • m

P(m|a) all vars not ancestor of query or evidence are irrelevant!

slide-22
SLIDE 22

EOLQs

Bayesian Networks

  • Approx. Inference

Exact Inference ■ Enumeration ■ Example ■ Var. Elim. 1 ■ Var. Elim. 2 ■ EOLQs

Wheeler Ruml (UNH) Lecture 23, CS 730 – 17 / 17

■

What question didn’t you get to ask today?

■

What’s still confusing?

■

What would you like to hear more about? Please write down your most pressing question about AI and put it in the box on your way out. Thanks!