belief propagation for spatial network embeddings
play

Belief Propagation for Spatial Network Embeddings Andrew Frank - PowerPoint PPT Presentation

May 16, 2023 •232 likes •1.04k views

Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth Department of Computer Science UC Irvine August 25, 2009 Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem

  1. Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth Department of Computer Science UC Irvine August 25, 2009

  2. Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem Description Model Formulation Experimental Results Latent Space Embeddings of Social Networks 3 Problem Description Model Formulation Preliminary Results

  3. Graphical Models Markov Random Fields Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem Description Model Formulation Experimental Results Latent Space Embeddings of Social Networks 3 Problem Description Model Formulation Preliminary Results

  4. Graphical Models Markov Random Fields What Are Graphical Models? Concise representations of probabilistic models Several types: Bayesian networks (DAGs) Markov random fields (undirected graphs) Factor graphs (bipartite graphs) . . . and others!

  5. Graphical Models Markov Random Fields What Are Graphical Models? Concise representations of probabilistic models Several types: Bayesian networks (DAGs) Markov random fields (undirected graphs) Factor graphs (bipartite graphs) . . . and others! A Nodes = random variables B C Edges = dependencies E D between variables

  6. Graphical Models Markov Random Fields What Are Graphical Models? Concise representations of probabilistic models Several types: Bayesian networks (DAGs) Markov random fields (undirected graphs) Factor graphs (bipartite graphs) . . . and others! A suspects Nodes = random variables B C Edges = dependencies E D between variables {innocent,guilty}

  7. Graphical Models Markov Random Fields What Are Graphical Models? Concise representations of probabilistic models Several types: Bayesian networks (DAGs) Markov random fields (undirected graphs) Factor graphs (bipartite graphs) . . . and others! friends A suspects Nodes = random variables B C Edges = dependencies E D between variables {innocent,guilty}

  8. Graphical Models Markov Random Fields Representing Conditional Independencies Interpreting a Markov Random Field If all paths from X to Y pass through Z, then we can say X and Y are conditionally independent given Z. Graphically, with a Textually, through enumeration: Markov Random Field (MRF): A ⊥ D , E | C A B ⊥ C , D , E | A C ⊥ B | A B C D ⊥ A , B , E | C E D E ⊥ A , B , D | C . . .

  9. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D )

  10. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D )

  11. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A )

  12. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A )

  13. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A )

  14. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A )

  15. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A )

  16. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A )

  17. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A ) p ( D | C )

  18. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A ) p ( D | C )

  19. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A ) p ( D | C ) p ( E | C )

  20. Graphical Models Markov Random Fields Factorization Conditional independence lets us factor a distribution: A A ⊥ D , E | C B ⊥ C , D , E | A B C C ⊥ B | A D ⊥ A , B , E | C E D E ⊥ A , B , D | C p ( A , B , C , D , E ) = p ( A ) p ( B | A ) p ( C | A , B ) p ( D | A , B , C ) p ( E | A , B , C , D ) = p ( A ) p ( B | A ) p ( C | A ) p ( D | C ) p ( E | C ) Largest factor involves 2 variables!

  21. Graphical Models Markov Random Fields Hammersley-Clifford Theorem General factorization property of all MRFs: Hammersley-Clifford Theorem Every MRF factors as the product of potential functions defined over cliques of the graph. Potential functions are. . . Strictly positive Unnormalized

  22. Graphical Models Markov Random Fields Hammersley-Clifford Theorem General factorization property of all MRFs: Hammersley-Clifford Theorem Every MRF factors as the product of potential functions defined over cliques of the graph. A Potential functions are. . . B C Strictly positive E D Unnormalized p ( · ) ∝ f A ( A ) f B ( B ) f C ( C ) f D ( D ) f E ( E ) f AB ( A , B ) f AC ( A , C ) f CD ( C , D ) f CE ( C , E )

  23. Graphical Models Markov Random Fields Specifying a Markov Random Field Model Define the potential functions, e.g.: Let our domain be 0=innocent, 1=guilty. � . 4 B = 0 f B ( B ) = . 6 B = 1 � A = B 2 f AB ( A , B ) = 1 A � = B

  24. Graphical Models Markov Random Fields Specifying a Markov Random Field Model Define the potential functions, e.g.: Let our domain be 0=innocent, 1=guilty. � . 4 B = 0 Suspect B is acting suspicious. f B ( B ) = . 6 B = 1 � A = B 2 f AB ( A , B ) = 1 A � = B

  25. Graphical Models Markov Random Fields Specifying a Markov Random Field Model Define the potential functions, e.g.: Let our domain be 0=innocent, 1=guilty. � . 4 B = 0 Suspect B is acting suspicious. f B ( B ) = . 6 B = 1 � A = B 2 Suspects A and B are friends. f AB ( A , B ) = 1 A � = B

  26. Graphical Models Inference Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem Description Model Formulation Experimental Results Latent Space Embeddings of Social Networks 3 Problem Description Model Formulation Preliminary Results

  27. Graphical Models Inference Marginalization with MRFs Query p(A): � O ( d n ) p ( A ) = p ( A , B , C , D , E ) B , C , D , E

  28. Graphical Models Inference Marginalization with MRFs Query p(A): � O ( d n ) p ( A ) = p ( A , B , C , D , E ) B , C , D , E A Use graph structure to compute p(A) B C in O ( dn 2 ) . E D

  29. Graphical Models Inference Belief Propagation (Sum-Product Algorithm) View marginalization as a “message-passing” algorithm Variables are computational nodes. Intermediate results are “messages” between nodes. A B C E D � f ( A ) f ( B ) f ( C ) f ( D ) f ( E ) f ( A , B ) f ( A , C ) f ( C , D ) f ( C , E ) B , C , D , E

  30. Graphical Models Inference Belief Propagation (Sum-Product Algorithm) View marginalization as a “message-passing” algorithm Variables are computational nodes. Intermediate results are “messages” between nodes. A B C E D � f ( A ) f ( B ) f ( C ) f ( D ) f ( E ) f ( A , B ) f ( A , C ) f ( C , D ) f ( C , E ) B , C , D , E

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Shuffled Belief Propagation Decoding Juntan Zhang and Marc Fossorier Department of Electrical

Shuffled Belief Propagation Decoding Juntan Zhang and Marc Fossorier Department of Electrical

Shuffled Belief Propagation Decoding Juntan Zhang and Marc Fossorier Department of Electrical Engineering University of Hawaii at Manoa Honolulu, HI 96816 Outline Review of LDPC Codes Standard Belief Propagation Algorithm Shuffled

467 views • 31 slides
LEARNING AFFINITY VIA SPATIAL PROPAGATION NETWORK Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu

LEARNING AFFINITY VIA SPATIAL PROPAGATION NETWORK Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu

LEARNING AFFINITY VIA SPATIAL PROPAGATION NETWORK Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu Zhong, Ming-Hsuan Yang, Jan Kautz NVIDIA Research, March 27, 2018 WHAT IS AFFINITY? The relations between two pixels/regions A typical

945 views • 40 slides
An empirical study of Gaussian belief propagation and application in the detection of F-formations

An empirical study of Gaussian belief propagation and application in the detection of F-formations

Regularized Gaussian belief propagation Empirical Study F-formation detection Concluding Remarks An empirical study of Gaussian belief propagation and application in the detection of F-formations Francois Kamper Stellenbosch University

578 views • 20 slides
Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani

Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani

Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn 2016 Recall: Belief

1.07k views • 61 slides
Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani

Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani

Probabilistic & Unsupervised Learning Belief Propagation Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn 2014 Recall: Belief

674 views • 50 slides
Approximation-aware Dependency Parsing by Belief Propagation

Approximation-aware Dependency Parsing by Belief Propagation

Approximation-aware Dependency Parsing by Belief Propagation Matt Gormley Mark Dredze Jason Eisner September 19, 2015 TACL at EMNLP 1

833 views • 44 slides
From Elimination to Belief Propagation Recall that Induced dependency during marginalization

From Elimination to Belief Propagation Recall that Induced dependency during marginalization

School of Computer Science The Belief Propagation (Sum-Product) Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 5, Sep 31, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B

455 views • 17 slides
Belief network inference Four main approaches to determine posterior distributions in belief

Belief network inference Four main approaches to determine posterior distributions in belief

Belief network inference Four main approaches to determine posterior distributions in belief networks: Variable Elimination: exploit the structure of the network to eliminate (sum out) the non-observed, non-query variables one at a time.

330 views • 29 slides
Belief Propagation Matt Gormley Lecture 9 Sep. 25, 2019 1 Q&A Q: What if I already

Belief Propagation Matt Gormley Lecture 9 Sep. 25, 2019 1 Q&A Q: What if I already

10-418 / 10-618 Machine Learning for Structured Data Machine Learning Department School of Computer Science Carnegie Mellon University Belief Propagation Matt Gormley Lecture 9 Sep. 25, 2019 1 Q&A Q: What if I already answered a

1.18k views • 72 slides
Probabilistic Graphical Models Lecture 13 Loopy Belief Propagation CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 13 Loopy Belief Propagation CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 13 Loopy Belief Propagation CS/CNS/EE 155 Andreas Krause Announcements Homework 3 out Lighter problem set to allow more time for project Next Monday: Guest lecture by Dr. Baback Moghaddam from the

1.35k views • 40 slides
Belief Networks Some Belief Network references E. Charniak Bayesian Networks without

Belief Networks Some Belief Network references E. Charniak Bayesian Networks without

Belief Networks Some Belief Network references E. Charniak Bayesian Networks without Tears, AI Magazine Winter 1991, pp 50-63 Chris Williams, School of Informatics University of Edinburgh D. Heckerman, A Tutorial on Learning

486 views • 6 slides
Problem Session 2 Problem 1: from belief propagation to Bayes AMP state evolution Below I have

Problem Session 2 Problem 1: from belief propagation to Bayes AMP state evolution Below I have

OOPS 2020 Mean field methods in high-dimensional statistics and nonconvex optimization Lecturer: Andrea Montanari Problem session leader: Michael Celentano July 6, 2020 Problem Session 2 Problem 1: from belief propagation to Bayes AMP state

321 views • 4 slides
19 : Baham Sabbir Scribes : , Belief Propagation : Problem The With Loops R far Graph

19 : Baham Sabbir Scribes : , Belief Propagation : Problem The With Loops R far Graph

The Algorithm Lecture Junction Tree 19 : Baham Sabbir Scribes : , Belief Propagation : Problem The With Loops R far Graph Directed b ; a , b. d pca ) c clb d ) p ( b ( Pla ) dla l a) p( ) = p , A C p fz fu a. Fi

480 views • 25 slides
Parallel Splash Belief Propagation Joseph E. Gonzalez Yucheng Low Carlos Guestrin David

Parallel Splash Belief Propagation Joseph E. Gonzalez Yucheng Low Carlos Guestrin David

Parallel Splash Belief Propagation Joseph E. Gonzalez Yucheng Low Carlos Guestrin David OHallaron Computers which worked on this project: BigBro1, BigBro2, BigBro3, BigBro4, BigBro5, BigBro6, BiggerBro, BigBroFS Tashish01, Tashi02,

817 views • 64 slides
A Multi-Round Side Channel Attack on AES using Belief Propagation Hlne Le Bouder 1 Ronan

A Multi-Round Side Channel Attack on AES using Belief Propagation Hlne Le Bouder 1 Ronan

A Multi-Round Side Channel Attack on AES using Belief Propagation Hlne Le Bouder 1 Ronan Lashermes 1 Yanis Linge 2 Gal Thomas 3 Jean Yves Zie 1 INRIA Rennes, LHS/PEC 2 STMicroelectronics 3 Orange Labs Issy Les Moulineaux October 25th, 2016

315 views • 17 slides
Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1

Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1

Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1 Hilbert J.Kappen 1 M. Chertkov 2 1 Department of Biophysics Radboud University, Nijmegen, The Netherlands 2 Theoretical Division and Center of

711 views • 28 slides
Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov networks: how it represents a prob. dist. independence assumptions

941 views • 57 slides
Probabilistic Graphical Models Lecture 10 Undirected Models CS/CNS/EE 155 Andreas Krause

Probabilistic Graphical Models Lecture 10 Undirected Models CS/CNS/EE 155 Andreas Krause

Probabilistic Graphical Models Lecture 10 Undirected Models CS/CNS/EE 155 Andreas Krause Announcements Homework 2 due this Wednesday (Nov 4) in class Project milestones due next Monday (Nov 9) About half the work should be done 4 pages

388 views • 34 slides
Undirected Graphical Models Undirected Graphs Chris Williams, School of Informatics, University

Undirected Graphical Models Undirected Graphs Chris Williams, School of Informatics, University

Undirected Graphical Models Undirected Graphs Chris Williams, School of Informatics, University of Edinburgh graph G = ( X, E ) X is a set of nodes, in one-to-

80 views • 5 slides
Markov Networks Asthma Cough Potential functions defined over cliques Smoking Cancer

Markov Networks Asthma Cough Potential functions defined over cliques Smoking Cancer

Markov Networks Undirected graphical models Smoking Cancer Markov Networks Asthma Cough Potential functions defined over cliques Smoking Cancer (S,C) 1 P ( x ) c x ( ) = c False False 4.5 Z c False True 4.5

62 views • 3 slides
Learning chordal Markov networks by dynamic programming Kustaa Kangas Teppo Niinim aki Mikko

Learning chordal Markov networks by dynamic programming Kustaa Kangas Teppo Niinim aki Mikko

Learning chordal Markov networks by dynamic programming Kustaa Kangas Teppo Niinim aki Mikko Koivisto NIPS 2014 (to appear) November 27, 2014 Kustaa Kangas Learning chordal Markov networks by dynamic programming Probabilistic graphical

932 views • 50 slides
Advanced Machine Learning Introduction to Probabilistic Graphical Models Amit Sethi Electrical

Advanced Machine Learning Introduction to Probabilistic Graphical Models Amit Sethi Electrical

Advanced Machine Learning Introduction to Probabilistic Graphical Models Amit Sethi Electrical Engineering, IIT Bombay Objectives Learn about statistical dependency of variables Understand how this dependency can be coded in graphs

572 views • 18 slides
Overview Filtering images MAP, Tikhonov and Poisson model of the noise A-priori and Markov

Overview Filtering images MAP, Tikhonov and Poisson model of the noise A-priori and Markov

24/10/2014 Sistemi Intelligenti Stima MAP Alberto Borghese Universit degli Studi di Milano Laboratory of Applied Intelligent Systems (AIS-Lab) Dipartimento di Scienze dellInformazione borghese@di.unimi.it Overview Filtering images

835 views • 36 slides
Permutation Editing and Matching via Embeddings Graham Cormode, S. Muthukrishnan, Cenk Sahinalp

Permutation Editing and Matching via Embeddings Graham Cormode, S. Muthukrishnan, Cenk Sahinalp

Permutation Editing and Matching via Embeddings Graham Cormode, S. Muthukrishnan, Cenk Sahinalp (grahamc@dcs.warwick.ac.uk) Permutation Editing and Matching Why study permutations? Distances between permutations Approximating

193 views • 18 slides

More recommend