sub quadratic markov tree mixture models for probability
play

Sub-quadratic Markov tree mixture models for probability density - PowerPoint PPT Presentation

Jul 13, 2023 •278 likes •940 views

Sub-quadratic mixture models Sub-quadratic Markov tree mixture models for probability density estimation Sourour Ammar 1 , Ph. Leray 1 , L. Wehenkel 2 1 Equipe COnnaissances et D ecision, LINA UMR 6241 Ecole Polytechnique de lUniversit e

  1. Sub-quadratic mixture models Sub-quadratic Markov tree mixture models for probability density estimation Sourour Ammar 1 , Ph. Leray 1 , L. Wehenkel 2 1 Equipe COnnaissances et D´ ecision, LINA UMR 6241 Ecole Polytechnique de l’Universit´ e de Nantes 2 Department of EECS and GIGA-Research, University of Li` ege COMPSTAT’2010 - Paris - 22-27 August 2010 S. Ammar et al. Sub-quadratic mixture models (1/22)

  2. Sub-quadratic mixture models A simple idea Proposition Develop density estimation techniques that could scale to very high-dimensional spaces, by exploiting the Perturb & Combine idea with probabilistic graphical models. Outline Background Our proposal Some results Conclusions and Further works S. Ammar et al. Sub-quadratic mixture models (2/22)

  3. Sub-quadratic mixture models Background Perturb and combine principle P&C principle in supervised learning Principle : (Bagging, Random forests, Extremely randomized trees) How can we apply this idea to density estimation with Bayesian networks (BN)? S. Ammar et al. Sub-quadratic mixture models (3/22)

  4. Sub-quadratic mixture models Background Perturb and combine principle P&C principle in supervised learning Principle : (Bagging, Random forests, Extremely randomized trees) Learning algorithm (research proc.) How can we apply this idea to density estimation with Bayesian networks (BN)? S. Ammar et al. Sub-quadratic mixture models (3/22)

  5. Sub-quadratic mixture models Background Perturb and combine principle P&C principle in supervised learning Principle : (Bagging, Random forests, Extremely randomized trees) Learning Perturb Weak algorithm algorithm (randomization) (research proc.) How can we apply this idea to density estimation with Bayesian networks (BN)? S. Ammar et al. Sub-quadratic mixture models (3/22)

  6. � � � Sub-quadratic mixture models Background Perturb and combine principle P&C principle in supervised learning Principle : (Bagging, Random forests, Extremely randomized trees) Learning Perturb Weak algorithm algorithm (randomization) (research proc.) Weak Weak Weak algorithm algorithm algorithm 1 m 2 How can we apply this idea to density estimation with Bayesian networks (BN)? S. Ammar et al. Sub-quadratic mixture models (3/22)

  7. � � � � � � Sub-quadratic mixture models Background Perturb and combine principle P&C principle in supervised learning Principle : (Bagging, Random forests, Extremely randomized trees) Learning Perturb Weak algorithm algorithm (randomization) (research proc.) Weak Weak Weak algorithm algorithm algorithm 1 m 2 prediction 1 prediction 2 prediction m How can we apply this idea to density estimation with Bayesian networks (BN)? S. Ammar et al. Sub-quadratic mixture models (3/22)

  8. � � � � � � Sub-quadratic mixture models Background Perturb and combine principle P&C principle in supervised learning Principle : (Bagging, Random forests, Extremely randomized trees) Learning Perturb Weak algorithm algorithm (randomization) (research proc.) Weak Weak Weak algorithm algorithm algorithm 1 m 2 prediction 1 prediction 2 prediction m Combine (Weighting schema) Final prediction How can we apply this idea to density estimation with Bayesian networks (BN)? S. Ammar et al. Sub-quadratic mixture models (3/22)

  9. � � � � � � Sub-quadratic mixture models Background Perturb and combine principle P&C principle in supervised learning Principle : (Bagging, Random forests, Extremely randomized trees) Learning Perturb Weak algorithm algorithm (randomization) (research proc.) Weak Weak Weak algorithm algorithm algorithm 1 m 2 prediction 1 prediction 2 prediction m Combine (Weighting schema) Final prediction How can we apply this idea to density estimation with Bayesian networks (BN)? S. Ammar et al. Sub-quadratic mixture models (3/22)

  10. Sub-quadratic mixture models Background Density estimation with BN Density estimation with BN A B D E C S. Ammar et al. Sub-quadratic mixture models (4/22)

  11. Sub-quadratic mixture models Background Density estimation with BN Density estimation with BN ˜ S A B D E C S. Ammar et al. Sub-quadratic mixture models (4/22)

  12. Sub-quadratic mixture models Background Density estimation with BN Density estimation with BN ˜ S A B C D E 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 A B 1 1 1 1 1 0 0 0 0 1 D E C 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 S. Ammar et al. Sub-quadratic mixture models (4/22)

  13. Sub-quadratic mixture models Background Density estimation with BN Density estimation with BN ˜ S A B C D E 0 0 1 0 0 B D 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 C 0 0 0 0 1 E 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 A 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 S. Ammar et al. Sub-quadratic mixture models (4/22)

  14. Sub-quadratic mixture models Background Density estimation with BN Density estimation with BN ˜ S A B C D E 0 0 1 0 0 B D 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 C 0 0 0 0 1 E 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 A 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 S. Ammar et al. Sub-quadratic mixture models (4/22)

  15. Sub-quadratic mixture models Background Density estimation with BN Density estimation with BN ˜ S A B C D E 0 0 1 0 0 B D 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 C 0 0 0 0 1 E 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 A 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 S. Ammar et al. Sub-quadratic mixture models (4/22)

  16. � � � Sub-quadratic mixture models Background Density estimation with BN Density estimation with BN ˜ S A B C D E 0 0 1 0 0 B D 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 C 0 0 0 0 1 E 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 A 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 S. Ammar et al. Sub-quadratic mixture models (4/22)

  17. Sub-quadratic mixture models Background Density estimation with BN Bayesian averaging Instead of searching for an optimal model (structure + parameters): Assume prior probabilities over the space of structures Determine posterior probabilities of each model given a dataset approach the target distribution by avereaging the different models wighted by their posterior probabilities Caveats : Exact Bayesian averaging over large space of models is not ‘scalable’ ⇒ requires to strongly constrain the space of structures S. Ammar et al. Sub-quadratic mixture models (5/22)

  18. Sub-quadratic mixture models Outline Background Our proposal Some results Conclusions and Further works S. Ammar et al. Sub-quadratic mixture models (6/22)

  19. Sub-quadratic mixture models Proposal Strategy Use simple spaces of graphical structures ˜ S (e.g. chains, trees, poly-trees etc.) Do not assume that target distribution is representable by one of these structures Rather, assume that target distribution may be approximated well by a mixture of a reasonable number of ( S , θ ∗ ) pairs, S ∈ ˜ S S. Ammar et al. Sub-quadratic mixture models (7/22)

  20. Sub-quadratic mixture models Proposal Generic algorithm principle ˜ S A B C D E 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 S. Ammar et al. Sub-quadratic mixture models (8/22)

  21. � � � � Sub-quadratic mixture models Proposal Generic algorithm principle ˜ S A B C D E 0 0 1 0 0 0 1 1 1 1 T 1 T 2 T m 1 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 S. Ammar et al. Sub-quadratic mixture models (8/22)

  22. � � � � � � Sub-quadratic mixture models Proposal Generic algorithm principle ˜ S A B C D E 0 0 1 0 0 0 1 1 1 1 T 1 T 2 T m 1 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 θ ∗ θ ∗ θ ∗ 1 2 m S. Ammar et al. Sub-quadratic mixture models (8/22)

  23. � � � � � � Sub-quadratic mixture models Proposal Generic algorithm principle ˜ S A B C D E 0 0 1 0 0 0 1 1 1 1 T 1 T 2 T m 1 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 θ ∗ θ ∗ θ ∗ 1 2 m µ 1 µ 2 µ m S. Ammar et al. Sub-quadratic mixture models (8/22)

  24. � � � � � � Sub-quadratic mixture models Proposal Generic algorithm principle ˜ S A B C D E 0 0 1 0 0 0 1 1 1 1 T 1 T 2 T m 1 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 θ ∗ θ ∗ θ ∗ 1 2 m µ 1 µ 2 µ m Final model ( θ ∗ i , µ i ) i =1 .. m S. Ammar et al. Sub-quadratic mixture models (8/22)

  25. � � � � � � Sub-quadratic mixture models Proposal Degrees of freedom ˜ S What space ˜ S ? A B C D E 0 0 1 0 0 0 1 1 1 1 T 1 T 2 T m 1 0 0 0 0 1 1 1 1 1 0 0 0 0 1 1 0 1 0 1 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 θ ∗ θ ∗ θ ∗ 1 2 m µ 1 µ 2 µ m Final model ( θ ∗ i , µ i ) i =1 .. m S. Ammar et al. Sub-quadratic mixture models (9/22)

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Structure of mixture models Victor Medina Researcher at SBIF DataCamp Mixture Models in R

Structure of mixture models Victor Medina Researcher at SBIF DataCamp Mixture Models in R

DataCamp Mixture Models in R MIXTURE MODELS IN R Structure of mixture models Victor Medina Researcher at SBIF DataCamp Mixture Models in R Description of mixture models 1. Which is the suitable probability distribution? Get familiar with

1.07k views • 36 slides
A Method of Moments for Mixture Models and Hidden Markov Models Anima Anandkumar @ Daniel Hsu #

A Method of Moments for Mixture Models and Hidden Markov Models Anima Anandkumar @ Daniel Hsu #

A Method of Moments for Mixture Models and Hidden Markov Models Anima Anandkumar @ Daniel Hsu # Sham M. Kakade # @ University of California, Irvine # Microsoft Research, New England Outline 1. Latent class models and parameter estimation 2.

1.44k views • 82 slides
Bernoulli Mixture Models Victor Medina Researcher at SBIF DataCamp Mixture Models in R The

Bernoulli Mixture Models Victor Medina Researcher at SBIF DataCamp Mixture Models in R The

DataCamp Mixture Models in R MIXTURE MODELS IN R Bernoulli Mixture Models Victor Medina Researcher at SBIF DataCamp Mixture Models in R The handwritten digits dataset DataCamp Mixture Models in R Continuous versus discrete variables

440 views • 41 slides
Models for Probability Distributions and Density Functions 1 General Concepts Parametric:

Models for Probability Distributions and Density Functions 1 General Concepts Parametric:

Models for Probability Distributions and Density Functions 1 General Concepts Parametric: E.g., Gaussian, Gamma, Binomial Non-Parametric: E.g., kernel estimates Intermediate models: Mixture Models 2 Gaussian Mixture Model

382 views • 22 slides
Markov Systems, Markov Decision Processes, and Dynamic Programming Andrew W. Moore Note to

Markov Systems, Markov Decision Processes, and Dynamic Programming Andrew W. Moore Note to

Artificial Intelligence 15-381 April 5, 2007 Sequential Decision Problems & Markov Decision Processes Recap of last lecture Reasoning over time - Markov Processes - Hidden Markov Models - modeling state transitions - probability of

321 views • 16 slides
Probability Recap CS 188: Artificial Intelligence Hidden Markov Models Conditional probability

Probability Recap CS 188: Artificial Intelligence Hidden Markov Models Conditional probability

Probability Recap CS 188: Artificial Intelligence Hidden Markov Models Conditional probability Product rule Chain rule X, Y independent if and only if: X and Y are conditionally independent given Z if and only if: Instructors:

231 views • 11 slides
Markov processes (Markov chains) Construct a Bayes net from these variables: parents? Markov

Markov processes (Markov chains) Construct a Bayes net from these variables: parents? Markov

Markov processes (Markov chains) Construct a Bayes net from these variables: parents? Markov assumption: X t depends on bounded subset of X 0: t 1 Temporal probability models First-order Markov process: P ( X t | X 0: t 1 ) = P ( X t | X t

359 views • 7 slides
CSE P 590 A Markov Models and Hidden Markov Models

CSE P 590 A Markov Models and Hidden Markov Models

CSE P 590 A Markov Models and Hidden Markov Models http://upload.wikimedia.org/wikipedia/commons/b/ba/Calico_cat Dosage Compensation Reminder: Proteins Read DNA and X-Inactivation E.g.: 2 copies (mom/dad) of each

1.28k views • 21 slides
Clustering: Models and Algorithms Shikui Tu 2019-03-07 1 Outline Gaussian Mixture Models

Clustering: Models and Algorithms Shikui Tu 2019-03-07 1 Outline Gaussian Mixture Models

Clustering: Models and Algorithms Shikui Tu 2019-03-07 1 Outline Gaussian Mixture Models (GMM) Expectation-Maximization (EM) for maximum likelihood Model selection, Bayesian learning 2 From distance to probability distance

383 views • 37 slides
CSE 527 " Markov Models and Hidden Markov Models !

CSE 527 " Markov Models and Hidden Markov Models !

CSE 527 " Markov Models and Hidden Markov Models ! http://upload.wikimedia.org/wikipedia/commons/b/ba/Calico_cat ! Dosage Compensation Reminder: Proteins Read DNA ! and X-Inactivation ! E.g.: ! 2 copies (mom/dad) of each chromosome

770 views • 17 slides
CSE P 527 Markov Models and Hidden Markov Models 1 2

CSE P 527 Markov Models and Hidden Markov Models 1 2

CSE P 527 Markov Models and Hidden Markov Models 1 2 http://upload.wikimedia.org/wikipedia/commons/b/ba/Calico_cat Dosage Compensation and X-Inactivation 2 copies (mom/dad) of each chromosome 1-23 Mostly, both copies of each gene are

970 views • 66 slides
CSE 427 Markov Models and Hidden Markov Models 2

CSE 427 Markov Models and Hidden Markov Models 2

CSE 427 Markov Models and Hidden Markov Models 2 http://upload.wikimedia.org/wikipedia/commons/b/ba/Calico_cat Dosage Compensation and X-Inactivation 2 copies (mom/dad) of each chromosome 1-23 Mostly, both copies of each gene are

1.07k views • 60 slides
Introduction to Markov Models Estimating the probability of phrases of words, sentences, etc .

Introduction to Markov Models Estimating the probability of phrases of words, sentences, etc .

Introduction to Markov Models Estimating the probability of phrases of words, sentences, etc . But first: A few preliminaries CIS 391 - Intro to AI 2 What counts as a word? A tricky question. CIS 391 - Intro to AI 3 How to find

530 views • 31 slides
Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Mixture Models: definition Mixture models: Linear supper-position of mixtures or components | =

1.04k views • 39 slides
Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic

Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic

Putting the Pieces Together Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic Definition A Markov Logic Network (MLN) is a set of A logical KB is a set of hard constraints pairs (F, w) where on

350 views • 6 slides
Probability Recap CS 4100: Artificial Intelligence Hidden Markov Models Co Conditional

Probability Recap CS 4100: Artificial Intelligence Hidden Markov Models Co Conditional

Probability Recap CS 4100: Artificial Intelligence Hidden Markov Models Co Conditional probability Pr Product rule Ch Chain rule X, , Y in independent if if and only ly if if: X an and Y ar are co e conditional ally

267 views • 6 slides
Generative networks part 2: GANs 23 / 54 Recap on generative networks Generative networks provide

Generative networks part 2: GANs 23 / 54 Recap on generative networks Generative networks provide

Generative networks part 2: GANs 23 / 54 Recap on generative networks Generative networks provide a way to sample from any distribution. 1. Sample z , where denotes an efficiently sampleable distribution (e.g., uniform or Gaussian). 2.

486 views • 47 slides
Notes and Announcements Midterm exam: Oct 20 , Wednesday, In Class Late Homeworks Turn

Notes and Announcements Midterm exam: Oct 20 , Wednesday, In Class Late Homeworks Turn

Notes and Announcements Midterm exam: Oct 20 , Wednesday, In Class Late Homeworks Turn in hardcopies to Michelle. DO NOT ask Michelle for extensions. Note down the date and time of submission. If submitting softcopy, email to

516 views • 49 slides
Non-parametric Methods Oliver Schulte - CMPT 726 Bishop PRML Ch. 2.5 Kernel Density Estimation

Non-parametric Methods Oliver Schulte - CMPT 726 Bishop PRML Ch. 2.5 Kernel Density Estimation

Kernel Density Estimation Nearest-neighbour Non-parametric Methods Oliver Schulte - CMPT 726 Bishop PRML Ch. 2.5 Kernel Density Estimation Nearest-neighbour Outline Kernel Density Estimation Nearest-neighbour These are non-parametric

374 views • 20 slides
via Threshold-Based Pruning Edward Gan & Peter Bailis 1 MacroBase: Analytics on Fast Streams

via Threshold-Based Pruning Edward Gan & Peter Bailis 1 MacroBase: Analytics on Fast Streams

Scalable Kernel Density Classification via Threshold-Based Pruning Edward Gan & Peter Bailis 1 MacroBase: Analytics on Fast Streams Increasing Streaming Data Manufacturing, Sensors, Mobile Multi-dimensional + Latent anomalies

826 views • 31 slides
Extended Path Integral Formulation for Volumetric Transport T. Hachisuka I. Georgiev W. Jarosz

Extended Path Integral Formulation for Volumetric Transport T. Hachisuka I. Georgiev W. Jarosz

Extended Path Integral Formulation for Volumetric Transport T. Hachisuka I. Georgiev W. Jarosz J. K ivnek D. Nowrouzezahrai The University of Tokyo Solid Angle Dartmouth College Charles University in Prague McGill University [Jensen

830 views • 61 slides
Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak

Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak

Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives different goals of learning a graphical model effect of goals on the learning setup

273 views • 23 slides
Average-Case Acceleration Through Spectral Density Estimation and Universal Asymptotic Optimality

Average-Case Acceleration Through Spectral Density Estimation and Universal Asymptotic Optimality

Average-Case Acceleration Through Spectral Density Estimation and Universal Asymptotic Optimality of Polyak Momentum Fabian Pedregosa (Google Brain, Montreal) Damien Scieur (SAIT AI Lab, Montreal) Motivation Consider the convex, quadratic

1.09k views • 59 slides
Statistical Geometry Processing Winter Semester 2011/2012 Machine Learning Topics Topics

Statistical Geometry Processing Winter Semester 2011/2012 Machine Learning Topics Topics

Statistical Geometry Processing Winter Semester 2011/2012 Machine Learning Topics Topics Machine Learning Intro Learning is density estimation The curse of dimensionality Bayesian inference and estimation Bayes rule in action

1.32k views • 85 slides

More recommend