ECE 6504: Advanced Topics in Machine Learning Probabilistic - PowerPoint PPT Presentation

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics: – Bayes Nets: Representation/Semantics – v-structures – Probabilistic influence, Active Trails Readings: Barber 3.3; KF 3.3.1-3.3.2 Dhruv Batra Virginia Tech

Plan for today • Notation Clarification • Errata #1: Number of parameters in disease network • Errata #2: Car start v-structure example • Bayesian Networks – Probabilistic influence & active trails – d-separation – General (conditional) independence assumptions in a BN (C) Dhruv Batra 2

A general Bayes net • Set of random variables • Directed acyclic graph – Encodes independence assumptions • CPTs – Conditional Probability Tables • Joint distribution: (C) Dhruv Batra 3

Factorized distributions • Given Flu Allergy – Random vars X 1 , … ,X n – P distribution over vars Sinus – BN structure G over same vars Nose Headache • P factorizes according to G if (C) Dhruv Batra Slide Credit: Carlos Guestrin 4

How many parameters in a BN? • Discrete variables X 1 , … , X n • Graph – Defines parents of X i , Pa Xi • CPTs – P(X i | Pa Xi ) (C) Dhruv Batra 5

Independencies in Problem World, Data, reality: BN: True distribution P contains independence assertions Graph G encodes local independence assumptions (C) Dhruv Batra Slide Credit: Carlos Guestrin 6

Bayes Nets • BN encode (conditional) independence assumptions. – I(G) = {X indep of Y given Z} • Which ones? • And how can we easily read them? (C) Dhruv Batra 7

Local Structures • What’s the smallest Bayes Net? (C) Dhruv Batra 8

Local Structures Indirect causal effect: X Z Y Indirect evidential effect: Common effect: X Z Y X Y Common cause: Z Z X Y (C) Dhruv Batra 9

Car starts BN • 18 binary attributes • Inference – P(BatteryAge|Starts=f) • 2 18 terms, why so fast? (C) Dhruv Batra Slide Credit: Carlos Guestrin 10

Bayes Ball Rules • Flow of information – on board (C) Dhruv Batra 11

Active trails formalized • Let variables O ⊆ {X 1 , … ,X n } be observed • A path X 1 – X 2 – · · · –X k is an active trail if for each consecutive triplet: – X i-1 → X i → X i+1 , and X i is not observed (X i ∉ O ) – X i-1 ← X i ← X i+1 , and X i is not observed (X i ∉ O ) – X i-1 ← X i → X i+1 , and X i is not observed (X i ∉ O ) – X i-1 → X i ← X i+1 , and X i is observed (X i ∈ O ), or one of its descendents is observed (C) Dhruv Batra Slide Credit: Carlos Guestrin 12

Active trails and Independence A B • Theorem : Variables X i and X j are independent given Z if C – no active trail between X i and X j when variables Z ⊆ {X 1 , … ,X n } are E observed D G F H J I K (C) Dhruv Batra Slide Credit: Carlos Guestrin 13

Name That Model Naïve Bayes: (C) Dhruv Batra Slide Credit: Erik Sudderth 14

Name That Model Tree-Augmented Naïve Bayes (TAN) (C) Dhruv Batra Slide Credit: Erik Sudderth 15

Name That Model Y 1 = {a, … z} Y 2 = {a, … z} Y 3 = {a, … z} Y 4 = {a, … z} Y 5 = {a, … z} X 1 = X 2 = X 3 = X 4 = X 5 = Hidden Markov Model (HMM) (C) Dhruv Batra Figure Credit: Carlos Guestrin 16