Welcome to Welcome to the PGM Class - - PowerPoint PPT Presentation

▶
welcome to welcome to the pgm class
SMART_READER_LITE
LIVE PREVIEW

Welcome to Welcome to the PGM Class - - PowerPoint PPT Presentation

Jan 04, 2023 460 likes •876 views

Probabilistic Graphical Introduction Models Welcome to Welcome to the PGM Class

slide-1
SLIDE 1

Welcome to

Probabilistic Graphical Models

Introduction

  • Welcome to

the PGM Class

slide-2
SLIDE 2
slide-3
SLIDE 3

!"

  • !"

– #$% – "

slide-4
SLIDE 4
  • !&""&&"

– ' – (& – )& &*"

  • – )& &*"

– !+,%-./%

  • 0+"

– #%

slide-5
SLIDE 5

1&

  • )

– 1'' – "&""& – "&"

  • – "&"
  • )""

– & – "" – (*

slide-6
SLIDE 6

(2

  • 3
  • 4""&"56$78

"9'&

  • "9'&
slide-7
SLIDE 7

:';

  • 0""
  • )6
  • 3"'
  • < =<* =1<=<3=>?2!@
slide-8
SLIDE 8

Daphne Koller

Mo#va#on' and'Overview'

Probabilis#c' Graphical' Models'

Introduc#on'

slide-9
SLIDE 9

Daphne Koller

predisposing factors symptoms test results millions of pixels or thousands of superpixels each needs to be labeled {grass, sky, water, cow, horse, …} diseases treatment outcomes

slide-10
SLIDE 10

Daphne Koller

Probabilistic Graphical Models

slide-11
SLIDE 11

Daphne Koller

Models

Data Model

Declarative representation Algorithm Algorithm Algorithm elicitation domain expert Learning

slide-12
SLIDE 12

Daphne Koller

Uncertainty

  • Partial knowledge of state of the world
  • Noisy observations
  • Phenomena not covered by our model
  • Inherent stochasticity
slide-13
SLIDE 13

Daphne Koller

Probability Theory

  • Declarative representation with clear

semantics

  • Powerful reasoning patterns
  • Established learning methods
slide-14
SLIDE 14

Daphne Koller

Complex Systems

predisposing factors symptoms test results diseases treatment outcomes class labels for thousands of superpixels

Random variables X1,…, Xn Joint distribution P(X1,…, Xn)

slide-15
SLIDE 15

Daphne Koller

Graphical Models

Intelligence Difficulty Grade Letter SAT B D C A

Bayesian networks Markov networks

slide-16
SLIDE 16

Daphne Koller

Graphical Models

  • M. Pradhan, G. Provan, B. Middleton, M. Henrion, UAI 94
slide-17
SLIDE 17

Daphne Koller

Graphical Representation

  • Intuitive & compact data structure
  • Efficient reasoning using general-purpose

algorithms

  • Sparse parameterization

– feasible elicitation – learning from data

slide-18
SLIDE 18

Daphne Koller

Many Applications

  • Medical diagnosis
  • Fault diagnosis
  • Natural language

processing

  • Traffic analysis
  • Social network models
  • Message decoding
  • Computer vision

– Image segmentation – 3D reconstruction – Holistic scene analysis

  • Speech recognition
  • Robot localization &

mapping

slide-19
SLIDE 19

Daphne Koller

Image Segmentation

slide-20
SLIDE 20

Daphne Koller

  • Medical Diagnosis

Thanks to: Eric Horvitz, Microsoft Research

slide-21
SLIDE 21

Daphne Koller

Textual Information Extraction

  • Mrs. Green spoke today in New York. Green chairs

the finance committee.

slide-22
SLIDE 22

Daphne Koller

Learned Model

Multi-Sensor Integration: Traffic

  • Trained on historical data
  • Learn to predict current & future road speed, including
  • n unmeasured roads
  • Dynamic route optimization

Multiple views

  • n traffic

Incident reports Weather Thanks to: Eric Horvitz, Microsoft Research

  • I95 corridor experiment: accurate

to ±5 MPH in 85% of cases

  • Fielded in 72 cities
slide-23
SLIDE 23

Daphne Koller

Known 15/17 Supported 2/17 Reversed 1 Missed 3

Causal protein-signaling networks derived from multiparameter single-cell data Sachs et al., Science 2005

Biological Network Reconstruction

Phospho-Proteins Phospho-Lipids Perturbed in data

PKC Raf Erk Mek Plcγ PKA Akt Jnk P38 PIP2 PIP3 Subsequently validated in wetlab

This figure may be used for non-commercial and classroom purposes only. Any other uses require the prior written permission from AAAS

slide-24
SLIDE 24

Daphne Koller

Overview

  • Representation

– Directed and undirected – Temporal and plate models

  • Inference

– Exact and approximate – Decision making

  • Learning

– Parameters and structure – With and without complete data

slide-25
SLIDE 25

Daphne Koller

Preliminaries:+ Distribu0ons+

Probabilis0c+ Graphical+ Models+

Introduc0on+

slide-26
SLIDE 26

Daphne Koller

Joint Distribution

  • Intelligence (I)

– i0 (low), i1 (high),

  • Difficulty (D)

– d0 (easy), d1 (hard)

  • Grade (G)

– g1 (A), g2 (B), g3 (C)

I D G Prob. i0 d0 g1 0.126 i0 d0 g2 0.168 i0 d0 g3 0.126 i0 d1 g1 0.009 i0 d1 g2 0.045 i0 d1 g3 0.126 i1 d0 g1 0.252 i1 d0 g2 0.0224 i1 d0 g3 0.0056 i1 d1 g1 0.06 i1 d1 g2 0.036 i1 d1 g3 0.024

slide-27
SLIDE 27

Daphne Koller

Conditioning

I D G Prob. i0 d0 g1 0.126 i0 d0 g2 0.168 i0 d0 g3 0.126 i0 d1 g1 0.009 i0 d1 g2 0.045 i0 d1 g3 0.126 i1 d0 g1 0.252 i1 d0 g2 0.0224 i1 d0 g3 0.0056 i1 d1 g1 0.06 i1 d1 g2 0.036 i1 d1 g3 0.024

condition on g1

slide-28
SLIDE 28

Daphne Koller

Conditioning: Reduction

I D G Prob. i0 d0 g1 0.126 i0 d1 g1 0.009 i1 d0 g1 0.252 i1 d1 g1 0.06

slide-29
SLIDE 29

Daphne Koller

P(I, D, g1)

Conditioning: Renormalization

I D G Prob. i0 d0 g1 0.126 i0 d1 g1 0.009 i1 d0 g1 0.252 i1 d1 g1 0.06

P(I, D | g1)

I D Prob. i0 d0 0.282 i0 d1 0.02 i1 d0 0.564 i1 d1 0.134

0.447

slide-30
SLIDE 30

Daphne Koller

Marginalization

D Prob. d0 0.846 d1 0.154 I D Prob. i0 d0 0.282 i0 d1 0.02 i1 d0 0.564 i1 d1 0.134

Marginalize I

slide-31
SLIDE 31

Daphne Koller

Preliminaries:+ Factors+

Probabilis1c+ Graphical+ Models+

Introduc1on+

slide-32
SLIDE 32

Daphne Koller

Factors

  • A factor φ(X1,…,Xk)
  • Scope = {X1,…,Xk}

φ : Val(X1,…,Xk) → R

slide-33
SLIDE 33

Daphne Koller

I D G Prob. i0 d0 g1 0.126 i0 d0 g2 0.168 i0 d0 g3 0.126 i0 d1 g1 0.009 i0 d1 g2 0.045 i0 d1 g3 0.126 i1 d0 g1 0.252 i1 d0 g2 0.0224 i1 d0 g3 0.0056 i1 d1 g1 0.06 i1 d1 g2 0.036 i1 d1 g3 0.024

Joint Distribution

P(I,D,G)

slide-34
SLIDE 34

Daphne Koller

Unnormalized measure P(I,D,g1)

I D G Prob. i0 d0 g1 0.126 i0 d1 g1 0.009 i1 d0 g1 0.252 i1 d1 g1 0.06

P(I,D,g1)

slide-35
SLIDE 35

Daphne Koller

Conditional Probability Distribution (CPD)

0.3 0.08 0.25 0.4 g2 0.02 0.9 i1,d0 0.7 0.05 i0,d1 0.5 0.3 g1 g3 0.2 i1,d1 0.3 i0,d0

P(G | I,D)

slide-36
SLIDE 36

Daphne Koller

General factors

A B

φ

a0 b0 30 a0 b1 5 a1 b0 1 a1 b1 10

slide-37
SLIDE 37

Daphne Koller

a1 b1 0.5 a1 b2 0.8 a2 b1 0.1 a2 b2 a3 b1 0.3 a3 b2 0.9 b1 c1 0.5 b1 c2 0.7 b2 c1 0.1 b2 c2 0.2 a1 b1 c1 0.5·0.5 = 0.25 a1 b1 c2 0.5·0.7 = 0.35 a1 b2 c1 0.8·0.1 = 0.08 a1 b2 c2 0.8·0.2 = 0.16 a2 b1 c1 0.1·0.5 = 0.05 a2 b1 c2 0.1·0.7 = 0.07 a2 b2 c1 0·0.1 = 0 a2 b2 c2 0·0.2 = 0 a3 b1 c1 0.3·0.5 = 0.15 a3 b1 c2 0.3·0.7 = 0.21 a3 b2 c1 0.9·0.1 = 0.09 a3 b2 c2 0.9·0.2 = 0.18

Factor Product

slide-38
SLIDE 38

Daphne Koller

a1 b1 c1 0.25 a1 b1 c2 0.35 a1 b2 c1 0.08 a1 b2 c2 0.16 a2 b1 c1 0.05 a2 b1 c2 0.07 a2 b2 c1 a2 b2 c2 a3 b1 c1 0.15 a3 b1 c2 0.21 a3 b2 c1 0.09 a3 b2 c2 0.18 a1 c1 0.33 a1 c2 0.51 a2 c1 0.05 a2 c2 0.07 a3 c1 0.24 a3 c2 0.39

Factor Marginalization

slide-39
SLIDE 39

Daphne Koller

a1 b1 c1 0.25 a1 b2 c1 0.08 a2 b1 c1 0.05 a2 b2 c1 a3 b1 c1 0.15 a3 b2 c1 0.09 a1 b1 c1 0.25 a1 b1 c2 0.35 a1 b2 c1 0.08 a1 b2 c2 0.16 a2 b1 c1 0.05 a2 b1 c2 0.07 a2 b2 c1 a2 b2 c2 a3 b1 c1 0.15 a3 b1 c2 0.21 a3 b2 c1 0.09 a3 b2 c2 0.18

Factor Reduction

a1 b1 c1 0.25 a1 b1 c2 0.35 a1 b2 c1 0.08 a1 b2 c2 0.16 a2 b1 c1 0.05 a2 b1 c2 0.07 a2 b2 c1 a2 b2 c2 a3 b1 c1 0.15 a3 b1 c2 0.21 a3 b2 c1 0.09 a3 b2 c2 0.18

slide-40
SLIDE 40

Daphne Koller

Why factors?

  • Fundamental building block for defining

distributions in high-dimensional spaces

  • Set of basic operations for manipulating

these probability distributions