part 3 probabilistic inference in graphical models
play

Part 3: Probabilistic Inference in Graphical Models Sebastian - PowerPoint PPT Presentation

Jan 30, 2023 •254 likes •951 views

Belief Propagation Variational Inference Sampling Break Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado Springs, 25th June 2011 Sebastian Nowozin and Christoph H. Lampert Part 3:

  1. Belief Propagation Variational Inference Sampling Break Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado Springs, 25th June 2011 Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  2. Belief Propagation Variational Inference Sampling Break Belief Propagation Belief Propagation ◮ “Message-passing” algorithm ◮ Exact and optimal for tree-structured graphs ◮ Approximate for cyclic graphs Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  3. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � Z = exp( − E ( y )) y ∈Y � � � � = exp( − E ( y i , y j , y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  4. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � Z = exp( − E ( y )) y ∈Y � � � � = exp( − E ( y i , y j , y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  5. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � Z = exp( − E ( y )) y ∈Y � � � � = exp( − E ( y i , y j , y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  6. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � � � � Z = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  7. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � � � � Z = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  8. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � � � � Z = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  9. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains r H → Y k ∈ R Y k Y i Y j Y k Y l F G H � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) Z y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � �� � r H → Yk ( y k ) � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) r H → Y k ( y k ) y i ∈Y i y j ∈Y j y k ∈Y i Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  10. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains r H → Y k ∈ R Y k Y i Y j Y k Y l F G H � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) Z y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � �� � r H → Yk ( y k ) � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) r H → Y k ( y k ) y i ∈Y i y j ∈Y j y k ∈Y i Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  11. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains r G → Y j ∈ R Y j Y i Y j Y k Y l F G H � � � Z = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) r H → Y k ( y k ) y i ∈Y i y j ∈Y j y k ∈Y k � �� � r G → Yj ( y j ) � � = exp( − E F ( y i , y j )) r G → Y j ( y j ) y i ∈Y i y j ∈Y j Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  12. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains r G → Y j ∈ R Y j Y i Y j Y k Y l F G H � � � Z = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) r H → Y k ( y k ) y i ∈Y i y j ∈Y j y k ∈Y k � �� � r G → Yj ( y j ) � � = exp( − E F ( y i , y j )) r G → Y j ( y j ) y i ∈Y i y j ∈Y j Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  13. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees Y i Y j Y k Y l F G H I Y m � Z = exp( − E ( y )) y ∈Y � � � � � = exp( − ( E F ( y i , y j ) + · · · + E I ( y k , y m ))) y i ∈Y i y j ∈Y i y k ∈Y i y l ∈Y i y m ∈Y m Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  14. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees Y i Y j Y k Y l F G H I Y m � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) · Z y i ∈Y i y j ∈Y j y k ∈Y k            �  �   exp( − E H ( y k , y l )) exp( − E I ( y k , y m )) ·         y l ∈Y l y m ∈Y m   � �� � � �� � r H → Yk ( y k ) r I → Yk ( y k ) Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  15. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees r H → Y k ( y k ) Y i Y j Y k Y l F G H r I → Y k ( y k ) I Y m � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) · Z y i ∈Y i y j ∈Y j y k ∈Y k ( r H → Y k ( y k ) · r I → Y k ( y k )) Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  16. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees q Y k → G ( y k ) r H → Y k ( y k ) Y i Y j Y k Y l F G H r I → Y k ( y k ) I Y m � � � Z = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) · y i ∈Y i y j ∈Y j y k ∈Y k ( r H → Y k ( y k ) · r I → Y k ( y k )) � �� � q Yk → G ( y k ) Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  17. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees q Y k → G ( y k ) r H → Y k ( y k ) Y i Y j Y k Y l F G H r I → Y k ( y k ) I Y m � � � Z = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) q Y k → G ( y k ) y i ∈Y i y j ∈Y j y k ∈Y k Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  18. Belief Propagation Variational Inference Sampling Break Belief Propagation Factor Graph Sum-Product Algorithm ◮ “Message”: pair of vectors at each . . . factor graph edge ( i , F ) ∈ E r F → Y i . . . 1. r F → Y i ∈ R Y i : factor-to-variable Y i . . . message F 2. q Y i → F ∈ R Y i : variable-to-factor . . . q Y i → F message ◮ Algorithm iteratively update messages ◮ After convergence: Z and µ F can be obtained from the messages Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives MAP inference and its complexity exact & approximate MAP inference max-product and max-sum

957 views • 74 slides
Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak Ravanbakhsh Fall 2019 Learning objective Learning objective an intuition for inference in graphical models why is it difficult? exact inference by

1.49k views • 62 slides
Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications of the sum and product rules Probabilistic graphical models are graphical representations of the qualitative aspects of probability distribu-

551 views • 23 slides
Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family & Variational Inference I Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives entropy exponential family distribution duality in

1.38k views • 85 slides
Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic

Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic

School of Computer Science Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 17, Nov 12, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X 2

672 views • 18 slides
Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29,

Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29,

Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29, 2017 Reading: 1 Deep Generative Models l Explicit probabilistic models Provide an explicit parametric specification of the distribution of l

514 views • 50 slides
Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference

Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference

Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov chains the idea behind Markov Chain Monte Carlo (MCMC) two

1.03k views • 71 slides
Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas Krause Announcements Homework 3 due next Monday (Nov 23) Project poster session on Friday December 4 (tentative) Final writeup (8 pages NIPS

805 views • 36 slides
Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14

Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14

Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14 2017 Today What is probabilistic graphical model and why it is useful? Bayesian Networks Basic Inference Generative Models Fancy Inference

785 views • 51 slides
Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of

Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of

Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of Technology Spring 2018 Soleymani Approximate inference Approximate inference techniques Deterministic approximation Variational algorithms

1.95k views • 97 slides
Monte Carlo methods Draw random samples from the desired distribution Yield a stochastic

Monte Carlo methods Draw random samples from the desired distribution Yield a stochastic

School of Computer Science Approximate Inference: Monte Carlo Inference Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 18, Nov 19, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X

503 views • 24 slides
Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani

Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani

Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Some slides are adapted from Xing s slides Exact methods for inference Variable elimination Message Passing: shared terms

1.2k views • 84 slides
15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this

15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this

15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this lecture) and Nihar Shah Carnegie Mellon University Spring 2020 1 Outline Probabilistic graphical models Probabilistic inference Exact inference

665 views • 39 slides
Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models

Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models

Algorithms for Reasoning with graphical models Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models Rina Dechter (Reading Darwiche chapter 15, related papers) slides11a 828X 2019 Sampling

1.05k views • 78 slides
Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov networks: how it represents a prob. dist. independence assumptions

941 views • 57 slides
Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models

Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models

Reasoning with graphical models Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models Rina Dechter (Reading Darwiche chapter 15, cutset-sampling paper posted) slides 9b 276 2020 Overview 1.

1.02k views • 63 slides
Bayesian networks: approximate inference Machine Intelligence Thomas D. Nielsen September 2008

Bayesian networks: approximate inference Machine Intelligence Thomas D. Nielsen September 2008

Bayesian networks: approximate inference Machine Intelligence Thomas D. Nielsen September 2008 Approximative inference September 2008 1 / 25 Approximate Inference Motivation Because of the (worst-case) intractability of exact inference in

1.13k views • 37 slides
Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X

Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X

Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count X probability x 1 n 1 x 1 n 1 / m . . . . . . . . . . . . x k n k x k n k / m total m If we could sample from a variables

476 views • 47 slides
Monte Carlo Methods Lecture notes for MAP001169 Based on Script by Martin Sk old adopted by

Monte Carlo Methods Lecture notes for MAP001169 Based on Script by Martin Sk old adopted by

Monte Carlo Methods Lecture notes for MAP001169 Based on Script by Martin Sk old adopted by Krzysztof Podg orski 2 Contents 3 4 CONTENTS Part I Simulation and Monte-Carlo Integration 5 Chapter 1 Simulation and Monte-Carlo

346 views • 23 slides
Complexity Results for MCMC derived from Quantitative Bounds Jun Yang (joint work with Jeffrey

Complexity Results for MCMC derived from Quantitative Bounds Jun Yang (joint work with Jeffrey

Complexity Results for MCMC derived from Quantitative Bounds Jun Yang (joint work with Jeffrey S. Rosenthal) Department of Statistical Sciences University of Toronto SSC 2018, McGill University MCMC Complexity Bounds (J. Yang) 1 Motivation

383 views • 7 slides
Markov chain Monte Carlo sampling SPiNCOM reading group Jun. 10 th , 2016 Dimitris Berberidis 1

Markov chain Monte Carlo sampling SPiNCOM reading group Jun. 10 th , 2016 Dimitris Berberidis 1

Markov chain Monte Carlo sampling SPiNCOM reading group Jun. 10 th , 2016 Dimitris Berberidis 1 Problem statement - Motivation Goal : Draw samples from a given pdf Impact of sampling : Bayesian inference ( :unknowns,

424 views • 29 slides
Bayesian Bias Mitigation for Crowdsourcing Fabian L. Wauthier, UC Berkeley with Michael I.

Bayesian Bias Mitigation for Crowdsourcing Fabian L. Wauthier, UC Berkeley with Michael I.

University of California, Berkeley Bayesian Bias Mitigation for Crowdsourcing Fabian L. Wauthier, UC Berkeley with Michael I. Jordan 9th of May, 2012 Fabian L. Wauthier: Bayesian Bias Mitigation for Crowdsourcing, 1 The Problem of Bias in

1.6k views • 118 slides
Hierarchical Modeling Hierarchical modeling has taken over the landscape in contemporaery

Hierarchical Modeling Hierarchical modeling has taken over the landscape in contemporaery

Hierarchical Modeling Hierarchical modeling has taken over the landscape in contemporaery stochastic modeling Intent here is to show a range of examples In the development, we will also show the connections to Gibbs sampling, why Gibbs sampling

340 views • 13 slides
Felisa J. V azquez-Abad and Lachlan L. H. Andrew D epartement dinformatique et recherche

Felisa J. V azquez-Abad and Lachlan L. H. Andrew D epartement dinformatique et recherche

Filtered Gibbs Sampler fo r Estimating Filtered Gibbs Sampler fo r Estimating Blo cking Probabilities in WDM Optical Net w o rks Blo cking Probabilities in WDM Optical Net w o rks Felisa J. V azquez-Abad and

645 views • 27 slides

More recommend