ECE 6504: Advanced Topics in Machine Learning Probabilistic - - PowerPoint PPT Presentation

▶
ece 6504 advanced topics in machine learning
SMART_READER_LITE
LIVE PREVIEW

ECE 6504: Advanced Topics in Machine Learning Probabilistic - - PowerPoint PPT Presentation

Oct 14, 2022 153 likes •328 views

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics: Bayes Nets: Representation/Semantics v-structures Probabilistic influence, Active Trails Readings: Barber 3.3; KF

slide-1
SLIDE 1

ECE 6504: Advanced Topics in Machine Learning

Probabilistic Graphical Models and Large-Scale Learning

Dhruv Batra Virginia Tech

Topics:

– Bayes Nets: Representation/Semantics – v-structures – Probabilistic influence, Active Trails

Readings: Barber 3.3; KF 3.3.1-3.3.2

slide-2
SLIDE 2

Plan for today

  • Notation Clarification
  • Errata #1: Number of parameters in disease network
  • Errata #2: Car start v-structure example
  • Bayesian Networks

– Probabilistic influence & active trails – d-separation – General (conditional) independence assumptions in a BN

(C) Dhruv Batra 2

slide-3
SLIDE 3

A general Bayes net

  • Set of random variables
  • Directed acyclic graph

– Encodes independence assumptions

  • CPTs

– Conditional Probability Tables

  • Joint distribution:

(C) Dhruv Batra 3

slide-4
SLIDE 4

Factorized distributions

  • Given

– Random vars X1,…,Xn – P distribution over vars – BN structure G over same vars

  • P factorizes according to G if

Flu Allergy Sinus Headache Nose (C) Dhruv Batra 4 Slide Credit: Carlos Guestrin

slide-5
SLIDE 5

How many parameters in a BN?

  • Discrete variables X1, …, Xn
  • Graph

– Defines parents of Xi, PaXi

  • CPTs – P(Xi| PaXi)

(C) Dhruv Batra 5

slide-6
SLIDE 6

Independencies in Problem

BN:

Graph G encodes local independence assumptions

World, Data, reality:

True distribution P contains independence assertions

(C) Dhruv Batra 6 Slide Credit: Carlos Guestrin

slide-7
SLIDE 7

Bayes Nets

  • BN encode (conditional) independence assumptions.

– I(G) = {X indep of Y given Z}

  • Which ones?
  • And how can we easily read them?

(C) Dhruv Batra 7

slide-8
SLIDE 8

Local Structures

  • What’s the smallest Bayes Net?

(C) Dhruv Batra 8

slide-9
SLIDE 9

Local Structures

(C) Dhruv Batra 9

Z Y X Z Y X Z Y X Z Y X

Indirect causal effect: Indirect evidential effect: Common cause: Common effect:

slide-10
SLIDE 10

Car starts BN

  • 18 binary attributes
  • Inference

– P(BatteryAge|Starts=f)

  • 218 terms, why so fast?

(C) Dhruv Batra 10 Slide Credit: Carlos Guestrin

slide-11
SLIDE 11

Bayes Ball Rules

  • Flow of information

– on board

(C) Dhruv Batra 11

slide-12
SLIDE 12

Active trails formalized

  • Let variables O ⊆ {X1,…,Xn} be observed
  • A path X1 – X2 – · · · –Xk is an active trail if for each

consecutive triplet:

– Xi-1→Xi→Xi+1, and Xi is not observed (Xi∉O) – Xi-1←Xi←Xi+1, and Xi is not observed (Xi∉O) – Xi-1←Xi→Xi+1, and Xi is not observed (Xi∉O) – Xi-1→Xi←Xi+1, and Xi is observed (Xi∈O), or one of its descendents is observed

Slide Credit: Carlos Guestrin (C) Dhruv Batra 12

slide-13
SLIDE 13

Active trails and Independence

  • Theorem: Variables Xi and Xj

are independent given Z if

– no active trail between Xi and Xj when variables Z⊆{X1,…,Xn} are

  • bserved

A H C E G D B F K J I

(C) Dhruv Batra 13 Slide Credit: Carlos Guestrin

slide-14
SLIDE 14

Naïve Bayes:

Slide Credit: Erik Sudderth

Name That Model

(C) Dhruv Batra 14

slide-15
SLIDE 15

Slide Credit: Erik Sudderth

Name That Model

Tree-Augmented Naïve Bayes (TAN)

(C) Dhruv Batra 15

slide-16
SLIDE 16

Name That Model

Hidden Markov Model (HMM)

Y1 = {a,…z} X1 = Y5 = {a,…z} Y3 = {a,…z} Y4 = {a,…z} Y2 = {a,…z} X2 = X3 = X4 = X5 =

Figure Credit: Carlos Guestrin (C) Dhruv Batra 16