another walkthrough of variational bayes
play

Another Walkthrough of Variational Bayes Bevan Jones ML for NLP - PowerPoint PPT Presentation

Mar 23, 2024 •96 likes •1.51k views

Another Walkthrough of Variational Bayes Bevan Jones ML for NLP Reading Group The University of Edinburgh 18 th of October, 2012 2 Variational Bayes? Bayes Bayes Theorem But the integral is intractable! Sampling Gibbs,

  1. Optimizing F • Apply Lagrange multipliers just like before • In this case, we have simply replaced z , x , and θ with vectors • The math is exactly the same • But we need to find the expectations we skipped before – Plug in the Dirichlet and multinomial distributions 84

  2. Optimal q(z, θ ) • Borrowed from the Mean Field Example • See slides 62-63 • All we need to do is apply the particulars of the mixture model 85

  3. Optimal q θ ( θ ) • Factorize q θ ( θ ) To get 86

  4. Optimal q φ ( φ ) : The Expectation 87

  5. Optimal q φ ( φ ) : The Expectation 88

  6. Dirichlet Distribution • Notable for being the conjugate prior of the multinomial 89

  7. Optimal q φ ( φ ) : The Numerator 90

  8. Optimal q φ ( φ ) : The Numerator Def. of Dirichlet 91

  9. Optimal q φ ( φ ) : The Numerator Previous slide 92

  10. Optimal q φ ( φ ) : The Numerator 93

  11. Optimal q φ ( φ ) : The Numerator 94

  12. Optimal q φ ( φ ) : The Numerator 95

  13. Optimal q φ ( φ ): The Normalization 96

  14. Optimal q φ ( φ ): The Normalization from previous slide 97

  15. Optimal q φ ( φ ): The Normalization 98

  16. Optimal q φ ( φ ): The Normalization Def. of Dirichlet 99

  17. Optimal q φ ( φ ): The Normalization 100

  18. Optimal q φ ( φ ) : Conjugacy Helps 101

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Variational Auto-encoders 2 VARIATIONAL AUTO-ENCODERS INTRODUCTION VARIATIONAL AUTO-ENCODERS

Variational Auto-encoders 2 VARIATIONAL AUTO-ENCODERS INTRODUCTION VARIATIONAL AUTO-ENCODERS

Lecture 3 Variational Auto-encoders 2 VARIATIONAL AUTO-ENCODERS INTRODUCTION VARIATIONAL AUTO-ENCODERS In this talk I will in some detail describe the paper of Kingma and Welling. Auto-Encoding Variational Bayes , International

706 views • 58 slides
EM & Variational Bayes Hanxiao Liu September 9, 2014 1 / 19 Outline 1. EM Algorithm 1.1

EM & Variational Bayes Hanxiao Liu September 9, 2014 1 / 19 Outline 1. EM Algorithm 1.1

EM & Variational Bayes Hanxiao Liu September 9, 2014 1 / 19 Outline 1. EM Algorithm 1.1 Introduction 1.2 Example: Mixture of vMFs 2. Variational Bayes 2.1 Introduction 2.2 Example: Bayesian Mixture of Gaussians 2 / 19 MLE by Gradient

527 views • 19 slides
Variational Inference for Bayes vMF Mixture Hanxiao Liu September 23, 2014 1 / 14 Variational

Variational Inference for Bayes vMF Mixture Hanxiao Liu September 23, 2014 1 / 14 Variational

Variational Inference for Bayes vMF Mixture Hanxiao Liu September 23, 2014 1 / 14 Variational Inference Review Lower bound the likelihood L ( ; X ) = E q log p ( X | ) log p ( X , Z | ) q ( Z ) = E q + E q log q (

310 views • 14 slides
Applications of Variational Bayes & DAGs in Neuroimaging ECE

Applications of Variational Bayes & DAGs in Neuroimaging ECE

Applications of Variational Bayes & DAGs in Neuroimaging ECE 6504: Advanced Topics in Machine Learning Rosalyn Moran rosalynj@vtc.vt.edu

665 views • 53 slides
Fast Quantification of Uncertainty and Robustness with Variational Bayes Tamara Broderick ITT

Fast Quantification of Uncertainty and Robustness with Variational Bayes Tamara Broderick ITT

Fast Quantification of Uncertainty and Robustness with Variational Bayes Tamara Broderick ITT Career Development Assistant Professor, MIT With: Ryan Giordano, Rachael Meager, Jonathan H. Huggins, Michael I. Jordan Bayesian inference

1.84k views • 170 slides
Probabilistic & Unsupervised Learning Factored Variational Approximations and Variational

Probabilistic & Unsupervised Learning Factored Variational Approximations and Variational

Probabilistic & Unsupervised Learning Factored Variational Approximations and Variational Bayes Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College

1.21k views • 116 slides
Probabilistic & Unsupervised Learning Factored Variational Approximations and Variational

Probabilistic & Unsupervised Learning Factored Variational Approximations and Variational

Probabilistic & Unsupervised Learning Factored Variational Approximations and Variational Bayes Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College

1.31k views • 108 slides
Fast Robustness Quantification with Variational Bayes Tamara Broderick ITT Career Development

Fast Robustness Quantification with Variational Bayes Tamara Broderick ITT Career Development

Fast Robustness Quantification with Variational Bayes Tamara Broderick ITT Career Development Assistant Professor, MIT With: Ryan Giordano, Rachael Meager, Jonathan Huggins, Michael I. Jordan Bayesian inference Complex, modular

1.45k views • 144 slides
Switching Linear Dynamics for Variational Bayes Filtering Philip Becker-Ehmck 1 , 2 , Jan Peters 2

Switching Linear Dynamics for Variational Bayes Filtering Philip Becker-Ehmck 1 , 2 , Jan Peters 2

Switching Linear Dynamics for Variational Bayes Filtering Philip Becker-Ehmck 1 , 2 , Jan Peters 2 , Patrick van der Smagt 1 1 Machine Learning Research Lab, Volkswagen Group 2 Department of Computer Science, Technische Universit at Darmstadt

340 views • 8 slides
Variational Autoencoders Tom Fletcher March 25, 2019 Talking about this paper: Diederik Kingma

Variational Autoencoders Tom Fletcher March 25, 2019 Talking about this paper: Diederik Kingma

Variational Autoencoders Tom Fletcher March 25, 2019 Talking about this paper: Diederik Kingma and Max Welling, Auto-Encoding Variational Bayes, In International Conference on Learning Representation (ICLR) , 2014. Autoencoders Input Latent

409 views • 15 slides
Variational Hamiltonian Monte Carlo via Score Matching Cheng Zhang (Joint work with Prof. Shahbaba

Variational Hamiltonian Monte Carlo via Score Matching Cheng Zhang (Joint work with Prof. Shahbaba

Background Markov chain Monte Carlo Fixed-Form Variational Bayes Variational Hamiltonian Monte Carlo Conclusion Rfrences Variational Hamiltonian Monte Carlo via Score Matching Cheng Zhang (Joint work with Prof. Shahbaba and Prof. Zhao)

451 views • 44 slides
(MP1 Walkthrough) Andrew Yoo (Some content taken from a previous year's walkthrough by Alberto

(MP1 Walkthrough) Andrew Yoo (Some content taken from a previous year's walkthrough by Alberto

CS 423 Operating System Design: Introduction to Linux Kernel Programming (MP1 Walkthrough) Andrew Yoo (Some content taken from a previous year's walkthrough by Alberto Alvarez) CS423: Operating Systems Design MP1 Goals Learn the basics

433 views • 27 slides
Bayes Theorem Thomas Bayes (1701-1761) Simple form of Bayes Theorem, for

Bayes Theorem Thomas Bayes (1701-1761) Simple form of Bayes Theorem, for

Nave Bayes Classifica/on Debapriyo Majumdar Data Mining Fall 2014 Indian Statistical Institute Kolkata August 14, 2014 Bayes Theorem Thomas Bayes (1701-1761) Simple form of Bayes Theorem, for two

1.04k views • 20 slides
Variational Inference for GPs: Presenters Group1: Stochastic variational inference. Slides 2 - 28

Variational Inference for GPs: Presenters Group1: Stochastic variational inference. Slides 2 - 28

Variational Inference for GPs: Presenters Group1: Stochastic variational inference. Slides 2 - 28 Chaoqi Wang Sana Tonekaboni Will Grathwohl Group2: Variational inference for GPs. Slides 29 - 57 Trefor Evans Kingsley Chang Shems Saleh James

1.32k views • 98 slides
1 Bayes Nets: Assumptions Independence in a BN Assumptions we are required to make to define

1 Bayes Nets: Assumptions Independence in a BN Assumptions we are required to make to define

1 2 Recap: Bayes Nets CS 473: Artificial Intelligence Bayes Nets: Independence A Bayes net is an efficient encoding of a probabilistic model of a domain Questions we can ask: Inference: given a fixed BN, what is P(X | e)?

197 views • 5 slides
An Introduction to An Introduction to Variational Variational Methods for Graphical Models

An Introduction to An Introduction to Variational Variational Methods for Graphical Models

An Introduction to An Introduction to Variational Variational Methods for Graphical Models Methods for Graphical Models By Jordan, M., Ghahramani, Z., Jaakkola, T.S., Saul, L.K. Basics of Basics of Variational Variational Methodology

549 views • 40 slides
Clarinet: WAN-Aware Optimization for Analyt ytics Queries Raajay Viswanathan, Ganesh

Clarinet: WAN-Aware Optimization for Analyt ytics Queries Raajay Viswanathan, Ganesh

Clarinet: WAN-Aware Optimization for Analyt ytics Queries Raajay Viswanathan, Ganesh Ananthanarayanan, Aditya Akella 1 Overview Web apps hosted on multiple DCs Low latency access to end-user 2 Overview Web apps hosted on

1.34k views • 106 slides
Deep Gaussian Processes (IPVI DGP) Haibin Yu*, Yizhou Chen* Zhongxiang Dai Bryan Kian Hsiang Low

Deep Gaussian Processes (IPVI DGP) Haibin Yu*, Yizhou Chen* Zhongxiang Dai Bryan Kian Hsiang Low

Implicit Posterior Variational Inference for Deep Gaussian Processes (IPVI DGP) Haibin Yu*, Yizhou Chen* Zhongxiang Dai Bryan Kian Hsiang Low and Patrick Jaillet Department of Computer Science National University of Singapore Department of

511 views • 25 slides
Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference Louis C. Tiao 1 Edwin V.

Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference Louis C. Tiao 1 Edwin V.

Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference Louis C. Tiao 1 Edwin V. Bonilla 2 Fabio Ramos 1 July 22, 2018 1 University of Sydney, 2 University of New South Wales Motivation: Unpaired Image-to-Image Translation

329 views • 22 slides
A Probabilistic Model for Using Social Networks in Personalized Item Recommendation Allison J.B.

A Probabilistic Model for Using Social Networks in Personalized Item Recommendation Allison J.B.

A Probabilistic Model for Using Social Networks in Personalized Item Recommendation Allison J.B. Chaney Princeton University Tina Eliassi-Rad David M. Blei Rutgers University Columbia University ajbc.io/spf Personalized Item Recommendation

971 views • 41 slides
Intractable Problems and DP with Bitmask Problem Solving Club March 1, 2017 Agenda

Intractable Problems and DP with Bitmask Problem Solving Club March 1, 2017 Agenda

Intractable Problems and DP with Bitmask Problem Solving Club March 1, 2017 Agenda Intractable problems Complexity classes P, NP, co-NP, #P, completeness How to identify common intractable problems Dynamic programming

795 views • 21 slides
On the Algorithmic Effectiveness of Digraph Decompositions and Complexity Measures Michael

On the Algorithmic Effectiveness of Digraph Decompositions and Complexity Measures Michael

On the Algorithmic Effectiveness of Digraph Decompositions and Complexity Measures Michael Lampis, Georgia Kaouri and Valia Mitsou ISAAC 2008 p. 1/18 Graph decompositions Treewidth (by Robertson and Seymour) is the most well-known and

640 views • 34 slides
t r ts tt

t r ts tt

t r ts tt trts rs rt sttt ttqs

624 views • 37 slides
On the Public Indifferentiability and Correlation Intractability of the 6-Round Feistel

On the Public Indifferentiability and Correlation Intractability of the 6-Round Feistel

On the Public Indifferentiability and Correlation Intractability of the 6-Round Feistel Construction Avradip Mandal 1 Jacques Patarin 2 Yannick Seurin 3 1 University of Luxembourg 2 University of Versailles, France 3 ANSSI, France March 20, TCC

1.05k views • 89 slides

More recommend