47.1 Introduction chapter overview: 46. Introduction and - - PowerPoint PPT Presentation

▶
47 1 introduction
SMART_READER_LITE
LIVE PREVIEW

47.1 Introduction chapter overview: 46. Introduction and - - PowerPoint PPT Presentation

May 12, 2023 253 likes •333 views

Foundations of Artificial Intelligence May 24, 2017 47. Uncertainty: Representation Foundations of Artificial Intelligence 47.1 Introduction 47. Uncertainty: Representation 47.2 Conditional Independence Malte Helmert and Gabriele R oger

slide-1
SLIDE 1

Foundations of Artificial Intelligence

  • 47. Uncertainty: Representation

Malte Helmert and Gabriele R¨

  • ger

University of Basel

May 24, 2017

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 1 / 24

Foundations of Artificial Intelligence

May 24, 2017 — 47. Uncertainty: Representation

47.1 Introduction 47.2 Conditional Independence 47.3 Bayesian Networks 47.4 Summary

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 2 / 24

Uncertainty: Overview

chapter overview:

◮ 46. Introduction and Quantification ◮ 47. Representation of Uncertainty

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 3 / 24

  • 47. Uncertainty: Representation

Introduction

47.1 Introduction

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 4 / 24

slide-2
SLIDE 2
  • 47. Uncertainty: Representation

Introduction

Running Example

We continue the dentist example. toothache ¬toothache catch ¬catch catch ¬catch cavity 0.108 0.012 0.072 0.008 ¬cavity 0.016 0.064 0.144 0.576

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 5 / 24

  • 47. Uncertainty: Representation

Introduction

Full Joint Probability Distribution: Discussion

Advantage: Contains all necessary information Disadvantage: Prohibitively large in practice: Table for n Boolean variables has size O(2n). Good for theoretical foundations, but what to do in practice?

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 6 / 24

  • 47. Uncertainty: Representation

Conditional Independence

47.2 Conditional Independence

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 7 / 24

  • 47. Uncertainty: Representation

Conditional Independence

Reminder: Bayes’ Rule

General version with multivalued variables and conditioned on some background evidence e: P(Y | X, e) = P(X | Y , e)P(Y | e) P(X | e)

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 8 / 24

slide-3
SLIDE 3
  • 47. Uncertainty: Representation

Conditional Independence

Multiple Evidence

If we already know that the probe catches and the tooth aches, we could compute the probability that this patient has cavity from P(Cavity | catch ∧ toothache) = αP(catch ∧ toothache | Cavity)P(Cavity). Problem: Need conditional probability for catch ∧ toothache Problem: for each value of Cavity. Problem: same scalability problem as with full joint distribution

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 9 / 24

  • 47. Uncertainty: Representation

Conditional Independence

Conditional Independence: Example

toothache ¬toothache catch ¬catch catch ¬catch cavity 0.108 0.012 0.072 0.008 ¬cavity 0.016 0.064 0.144 0.576 Variables Toothache and Catch not independent but independent given the presence or absence of cavity:

P(Toothache, Catch | Cavity) = P(Toothache | Cavity)P(Catch | Cavity)

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 10 / 24

  • 47. Uncertainty: Representation

Conditional Independence

Conditional Independence

Definition Two variables X and Y are conditionally independent given a third variable Z if P(X, Y | Z) = P(X | Z)P(Y | Z).

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 11 / 24

  • 47. Uncertainty: Representation

Conditional Independence

Conditional Independence and Multiple Evidence Example

Multiple evidence: P(Cavity | catch ∧ toothache) = αP(catch ∧ toothache | Cavity)P(Cavity) = αP(toothache | Cavity)P(catch | Cavity)P(Cavity). No need for conditional joint probabilities for conjunctions

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 12 / 24

slide-4
SLIDE 4
  • 47. Uncertainty: Representation

Conditional Independence

Conditional Independence: Decomposition of Joint Dist.

Full joint distribution: P(Toothache, Catch, Cavity) = P(Toothache, Catch | Cavity)P(Cavity) = P(Toothache | Cavity)P(Catch | Cavity)P(Cavity) Large table can be decomposed into three smaller tables. For n symptoms that are all conditionally independent given Cavity the representation grows as O(n) instead of O(2n).

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 13 / 24

  • 47. Uncertainty: Representation

Bayesian Networks

47.3 Bayesian Networks

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 14 / 24

  • 47. Uncertainty: Representation

Bayesian Networks

Bayesian Networks

Definition A Bayesian network is a directed acyclic graph, where

◮ each node corresponds to a random variable, ◮ each node X has an associated

conditional probability distribution P(X | parents(X)) that quantifies the effect of the parents on the node. Bayesian networks are also called belief networks

  • r probabilistic networks.

They are a subclass of graphical models.

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 15 / 24

  • 47. Uncertainty: Representation

Bayesian Networks

Bayesian Network: Example

.001 P(B)

Alarm Earthquake MaryCalls JohnCalls Burglary

A P(J) t f .90 .05 B t t f f E t f t f P(A) .95 .29 .001 .94 .002 P(E) A P(M) t f .70 .01

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 16 / 24

slide-5
SLIDE 5
  • 47. Uncertainty: Representation

Bayesian Networks

Semantics

The semantics for Bayesian networks expresses that

◮ the information associated to each node represents

a conditional probability distribution, and that

◮ each variable is conditionally independent

  • f its non-descendants given its parents.

Definition A Bayesian network with nodes {X1, . . . , Xn} represents the full joint probability given by P(X1 = x1 ∧ · · · ∧ Xn = xn) =

n

  • i=1

P(Xi = xi | parents(Xi)).

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 17 / 24

  • 47. Uncertainty: Representation

Bayesian Networks

Naive Construction

Order all variables, e.g.. as X1, . . . , Xn. For i = 1 to n do:

◮ Choose from X1, . . . , Xi−1 a minimal set of parents of Xi

such that P(Xi | Xi−1, . . . , X1) = P(Xi = xi | parents(Xi)).

◮ For each parent insert a link from the parent to Xi. ◮ Define conditional probability table P(Xi | parents(Xi)).

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 18 / 24

  • 47. Uncertainty: Representation

Bayesian Networks

Compactness

Compactness of Bayesian networks stems from local structures in domains, where random variables are directly influenced only by a small number of variables.

◮ n Boolean random variables ◮ each variable directly influenced by at most k others ◮ full joint probability distribution contains 2n numbers ◮ Bayesian network can be specified by n2k numbers

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 19 / 24

  • 47. Uncertainty: Representation

Bayesian Networks

Influence of Node Ordering

A bad node ordering can lead to large numbers of parents and probabiliy distributions that are hard to specify.

JohnCalls MaryCalls Alarm Burglary Earthquake MaryCalls Alarm Earthquake Burglary JohnCalls (a) (b)

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 20 / 24

slide-6
SLIDE 6
  • 47. Uncertainty: Representation

Bayesian Networks

Conditional Independence Given Parents

Each variable is conditionally independent of its non-descendants given its parents.

. . . . . . U1 X U

m

Yn Znj Y

1

Z1j

X is conditionally independent of the nodes Zij given U1 . . . Um.

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 21 / 24

  • 47. Uncertainty: Representation

Bayesian Networks

Conditional Independence Given Markov Blanket

The Markov blanket of a node consists

  • f its parents, children and children’s other parents.

. . . . . . U1 Um Yn Znj Y1 Z1j X

Each variable is conditionally independent

  • f all other nodes in the

network given its Markov blanket (gray area).

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 22 / 24

  • 47. Uncertainty: Representation

Summary

47.4 Summary

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 23 / 24

  • 47. Uncertainty: Representation

Summary

Summary & Outlook

Summary

◮ Conditional independence is weaker than (unconditional)

independence but occurs more frequently.

◮ Bayesian networks exploit conditional independence to

compactly represent joint probability distributions. Outlook

◮ There are exact and approximate inference algorithms

for Bayesian networks.

◮ Exact inference in Bayesian networks is NP-hard

(but tractable for some sub-classes such as poly-trees).

◮ All concepts can be extended to continuous random variables.

  • M. Helmert, G. R¨
  • ger (University of Basel)

Foundations of Artificial Intelligence May 24, 2017 24 / 24