probabilistic graphical models
play

Probabilistic Graphical Models Christian Borgelt Dept. of Knowledge - PowerPoint PPT Presentation

Sep 15, 2023 •111 likes •1.06k views

Probabilistic Graphical Models Christian Borgelt Dept. of Knowledge Processing and Language Engineering Otto-von-Guericke-University of Magdeburg Universit atsplatz 2, D-39106 Magdeburg, Germany E-mail: borgelt@iws.cs.uni-magdeburg.de WWW:

  1. Probabilistic Graphical Models Christian Borgelt Dept. of Knowledge Processing and Language Engineering Otto-von-Guericke-University of Magdeburg Universit¨ atsplatz 2, D-39106 Magdeburg, Germany E-mail: borgelt@iws.cs.uni-magdeburg.de WWW: http://fuzzy.cs.uni-magdeburg.de/~borgelt 1

  2. Contents ✎ Inference Networks / Graphical Models ✎ Relational Networks: Decomposition and Reasoning ✎ Probabilistic Networks: Decomposition and Reasoning ✎ Conditional Independence Graphs ✎ Graphs and Decompositions ✎ Evidence Propagation in Graphs ✎ Danish Jersey Cattle Blood Type Determination: An Example ✎ Learning Relational Networks from Data ✎ Learning Probabilistic Networks from Data ✎ Probabilistic Networks and Causality ✎ Fault Analysis at DaimlerChrysler: An Application 2

  3. Inference Networks / Graphical Models ✎ Decomposition: Under certain conditions a distribution ✍ (e.g. a probability distribution) on a multi-dimensional domain, which encodes prior or generic knowledge about this domain, can be decomposed into a set ❢ ✍ 1 ❀ ✿ ✿ ✿ ❀ ✍ s ❣ of (overlapping) distributions on lower-dimensional subspaces. ✎ Simplified Reasoning: If such a decomposition is possible, it is sufficient to know the distributions on the subspaces to draw all inferences in the domain under consideration that can be drawn using the original distribution ✍ . ✎ Since such a decomposition is usually represented as a network and since it is used to draw inferences, it can be called an inference network . The edges of the network indicate the paths along which evidence has to be propagated. ✎ Another popular name is graphical model , where “graphical” indicates that it is based on a graph in the sense of graph theory. 3

  4. ✔ ✕ ✖ ✗ ✘ ✙ ✚ ✛ ✜ ✢ A Simple Example Example World Relation ☛ ✍ color shape size ☛ ✍ small ✡ ✍ medium ✡ ✍ small ✡ ☞ medium ✡ ☞ medium ✟ ✌ large ✠ ✌ medium ✠ ☞ medium ✠ ☞ medium ✎ 10 simple geometrical objects, 3 attributes. large ✎ One object is chosen at random and examined. ✎ Inferences are drawn about the unobserved attributes. 4

  5. ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✥ ✪ ✥ ✥ ✥ ✥ ✥ ✪ ✏ ✩ The Reasoning Space ☛ ✡ ✟ ✠ ✟ ☞ ☞ ✌ ✍ large medium medium small ✎ The reasoning space consists of a finite set Ω of world states. ✎ The world states are described by a set of attributes ❆ ✐ , whose domains ❢ ❛ ( ✐ ) 1 ❀ ✿ ✿ ✿ ❀ ❛ ( ✐ ) ❦ i ❣ can be seen as sets of propositions or events. ✎ The events in a domain are mutually exclusive and exhaustive. ✎ The reasoning space is assumed to contain the true, but unknown state ✦ 0 . 5

  6. ✦ ✦ ★ ★ ★ ★ ✦ ★ ★ ✥ ✧ ✩ ✩ ★ The Relation in the Reasoning Space Relation Relation in the Reasoning Space ☛ ✡ ✟ ✠ ☛ ✍ color shape size ☞ ☛ ✍ small ✡ ✍ ✌ medium ✡ ✍ small ✍ ✡ ☞ medium ✡ ☞ medium ✟ ✌ large large ✠ ✌ medium medium small ✠ ☞ medium ✠ ☞ medium large Each cube represents one tuple. 6

  7. ✦ ✩ ❆ ✧❆ ❅ ❅ ✩ ✩ ✩ ★ ★ ✦ ✥ ✩ ✩ Reasoning ✎ Let it be known (e.g. from an observation) that the given object is green. This information considerably reduces the space of possible value combinations. ✎ From the prior knowledge it follows that the given object must be ✍ either a triangle or a square and ✍ either medium or large. ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 7

  8. ❅ ✩ ❆ ❆ ❆ ❆ ❆ ✩ ✩ ✩ ✩ ✩ ✩ ✪ ❆ ❅ ❅ ❅ ❅ ❅ ✪❆ ✩ ✩ ✩ ✩ ✩ ❆ ✩ ✩ ✧ ❇ ❇ ❇ ❇ ✪❇ ✩ ✩ ✩ ✩ ✩ ✩ ✥ ✩ ★ ★ ✦ ★ ★ ★ ★ ✦ ✦ ★ ✩ ✩ Prior Knowledge and Its Projections ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 8

  9. ✦ ✄ ✥ ★ ✦ ✦ ✦ ✦ ★ ✧ ✩ ✩ ✂ ★ ✧ ✩ ✩ ✧ ✥ ★ ★ ✦ ★ ★ ★ ★ ✥ ★ ✧ ✧ ✥ ✥ ✥ ✩ ✩ ✩ ★ ★ ★ ✪ ✪ ✪ ✪ ✪ Cylindrical Extensions and Their Intersection ☛ ✡ ✟ ✠ ☞ Intersecting the cylindrical ex- ✌ tensions of the projection to the subspace formed by color ✍ and shape and of the projec- tion to the subspace formed by large shape and size yields the origi- medium nal three-dimensional relation. small ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 9

  10. ❅ ❈ ❅ ❅ ❅ ❅ ❅ ❈ � ❈ ❈ � ❈ ❈ ✁ ❈ ❈❅ ❅ ❈ ❈ ✄ ❈ ❅ ❅ ❅ ❈ ❈ ❈ Reasoning with Projections The same result can be obtained using only the projections to the subspaces without reconstructing the original three-dimensional space: ☛ ✡ ✟ ✠ s m l size color project extend shape ☞ ☞ ✌ ✌ project extend ✍ ✍ ☛ ✡ ✟ ✠ s m l ✛ ✘ ✛ ✘ ✛ ✘ This justifies a network representation: color shape size ✚ ✙ ✚ ✙ ✚ ✙ 10

  11. ✩ ★ ✥ ✥ ✥ ✩ ✩ ✦ ✦ ✦ ★ ✥ ✄ ✂ ★ ✦ ✦ ★ ★ ★ ✥ ✥ ✦ ★ ✥ ✥ ✥ ✧ ✧ ✧ ✥ ✥ ★ ✦ ✦ ✦ ✦ ✦ ✦ ✦ ✦ ✦ ✩ ★ ★ ✧ ✦ ✦ ★ ★ ✦ ✦ ✧ ★ ✥ ✥ ✥ ✧ ✥ ✩ ✩ ★ ✦ ✩ ✩ ✥ ✧ Using other Projections ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 11

  12. ❅ ✩ ❆ ❆ ❆ ❆ ❆ ❆ ✪❆ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✪ ❅ ❅ ❅ ✩ ❅ ❅ ✩ ✩ ✩ ❆ ✩ ✩ ✧ ❇ ❇ ❇ ❇ ✪❇ ✩ ✩ ✩ ✩ ✩ ✩ ✥ ✩ ★ ★ ✦ ★ ★ ★ ★ ✦ ✦ ★ ✦ ✦ ✩ Is Decomposition Always Possible? ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ 2 large large medium medium 1 small small ☛ ✡ ✟ ✠ ☛ ✡ ✟ ✠ ☞ ☞ ✌ ✌ ✍ ✍ large large medium medium small small 12

  13. Possibility-Based Formalization Definition: Let Ω be a (finite) sample space. A discrete possibility measure ❘ on Ω is a function ❘ : 2 Ω ✦ ❢ 0 ❀ 1 ❣ satisfying 1. ❘ ( ❀ ) = 0 and 2. ✽ ❊ 1 ❀ ❊ 2 ✒ Ω : ❘ ( ❊ 1 ❬ ❊ 2 ) = max ❢ ❘ ( ❊ 1 ) ❀ ❘ ( ❊ 2 ) ❣ . ✎ Similar to Kolmogorov’s axioms of probability theory. ✎ If an event ❊ can occur (if it is possible), then ❘ ( ❊ ) = 1 ❀ otherwise (if ❊ cannot occur/is impossible) then ❘ ( ❊ ) = 0 ✿ ✎ ❘ (Ω) = 1 is not required, because this would exclude the empty relation. ✎ From the axioms it follows ❘ ( ❊ 1 ❭ ❊ 2 ) ✔ min ❢ ❘ ( ❊ 1 ) ❀ ❘ ( ❊ 2 ) ❣ ✿ ✎ Attributes are introduced as random variables (as in probability theory). ✎ ❘ ( ❆ = ❛ ) is an abbreviation of ❘ ( ❢ ✦ ❥ ❆ ( ✦ ) = ❛ ❣ ) 13

  14. Possibility-Based Formalization (continued) Definition: Let ❳ = ❢ ❆ 1 ❀ ✿ ✿ ✿ ❀ ❆ ♥ ❣ be a set of attributes defined on a (finite) sample space Ω with respective domains dom( ❆ ✐ ), ✐ = 1 ❀ ✿ ✿ ✿ ❀ ♥ . A relation r ❳ over ❳ is the restriction of a discrete possibility measure ❘ on Ω to the set of all events that can be defined by stating values for all attributes in ❳ . That is, r ❳ = ❘ ❥ ❊ X , where � � � ❊ ✷ 2 Ω ❊ ❳ = � ✾ ❛ 1 ✷ dom( ❆ 1 ) : ✿ ✿ ✿ ✾ ❛ ♥ ✷ dom( ❆ ♥ ) : � � ❊ � = ❆ ❥ = ❛ ❥ ❆ j ✷ ❳ � � � ❊ ✷ 2 Ω = � ✾ ❛ 1 ✷ dom( ❆ 1 ) : ✿ ✿ ✿ ✾ ❛ ♥ ✷ dom( ❆ ♥ ) : � � �� � � ❊ = ✦ ✷ Ω ❆ ❥ ( ✦ ) = ❛ ❥ ✿ � ❆ j ✷ ❳ ✎ Corresponds to the notion of a probability distribution. ✎ Advantage of this formalization: No index transformation functions are needed for projections, there are just fewer terms in the conjunctions. 14

  15. Possibility-Based Formalization (continued) Definition: Let ❯ = ❢ ❆ 1 ❀ ✿ ✿ ✿ ❀ ❆ ♥ ❣ be a set of attributes and r ❯ a relation over ❯ . Furthermore, let ▼ = ❢ ▼ 1 ❀ ✿ ✿ ✿ ❀ ▼ ♠ ❣ ✒ 2 ❯ be a set of nonempty (but not neces- sarily disjoint) subsets of ❯ satisfying � ▼ = ❯✿ ▼ ✷▼ r ❯ is called decomposable w.r.t. ▼ iff ✽ ❛ 1 ✷ dom( ❆ 1 ) : ✿ ✿ ✿ ✽ ❛ ♥ ✷ dom( ❆ ♥ ) : � � � � �� � � r ❯ ❆ ✐ = ❛ ✐ = min r ▼ ❆ ✐ = ❛ ✐ ✿ ▼ ✷▼ ❆ i ✷ ❯ ❆ i ✷ ▼ If r ❯ is decomposable w.r.t. ▼ , the set of relations ❘ ▼ = ❢ r ▼ 1 ❀ ✿ ✿ ✿ ❀ r ▼ m ❣ = ❢ r ▼ ❥ ▼ ✷ ▼❣ is called the decomposition of r ❯ . 15

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov networks: how it represents a prob. dist. independence assumptions

941 views • 57 slides
Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic

Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic

Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic AI! Graphical models of variable-level (in)dependence are a broken abstraction. [VdB KRR15] Let me be provocative Probabilistic graphical models

782 views • 63 slides
Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian networks Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives likelihood function and MLE role of the sufficient statistics MLE for

644 views • 60 slides
Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak Ravanbakhsh Fall 2019 Learning objective Learning objective an intuition for inference in graphical models why is it difficult? exact inference by

1.49k views • 62 slides
Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family & Variational Inference I Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives entropy exponential family distribution duality in

1.38k views • 85 slides
The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models

The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models

School of Computer Science The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 4, Sep 26, 2007 Receptor A Receptor A Receptor B Receptor B X 1 X 1 X 1 X 2 X 2 X 2 Eric Xing

457 views • 17 slides
Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that

Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that

Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that represents a probability distribution over random variables 1 , , Probabilistic Graphical Models A graph G that represents a

1.74k views • 129 slides
Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected

Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected

Probabilistic Graphical Models Probabilistic Graphical Models parameter learning in undirected models Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives the form of likelihood for undirected models why is it difficult to

826 views • 49 slides
Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Gaussian Network Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives multivariate Gaussian density: different parametrizations marginalization and

540 views • 39 slides
Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed

Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed

Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed & undirected models Siamak Ravanbakhsh Fall 2019 Learning Objective Learning Objective understand the relationship between CIs in directed

694 views • 23 slides
CS 6782: Fall 2010 Probabilistic Graphical Models Guozhang Wang December 10, 2010 1

CS 6782: Fall 2010 Probabilistic Graphical Models Guozhang Wang December 10, 2010 1

CS 6782: Fall 2010 Probabilistic Graphical Models Guozhang Wang December 10, 2010 1 Introduction to Probabilistic Graphical Models In a probabilistic graphical model, each node represents a random variable, and the links express probabilistic

380 views • 21 slides
Probabilistic Graphical Models Probabilistic Graphical Models Loopy BP and Bethe Free Energy

Probabilistic Graphical Models Probabilistic Graphical Models Loopy BP and Bethe Free Energy

Probabilistic Graphical Models Probabilistic Graphical Models Loopy BP and Bethe Free Energy Fall 2019 Siamak Ravanbakhsh Learning objective Learning objective loopy belief propagation its variational derivation: Bethe approximation So

832 views • 58 slides
Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak

Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak

Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives different goals of learning a graphical model effect of goals on the learning setup

273 views • 23 slides
Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of

Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of

Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Basics Multivariate distributions with large number of variables Independency assumptions are useful

503 views • 36 slides
Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives MAP inference and its complexity exact & approximate MAP inference max-product and max-sum

957 views • 74 slides
Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications of the sum and product rules Probabilistic graphical models are graphical representations of the qualitative aspects of probability distribu-

551 views • 23 slides
Undirected Graphical Models Dr. Shuang LIANG School of Software Engineering TongJi University

Undirected Graphical Models Dr. Shuang LIANG School of Software Engineering TongJi University

Undirected Graphical Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Todays Topics Introduction Parameterization Gibbs Distributions Reduced Markov Networks Markov Network

721 views • 36 slides
Graphical models Sunita Sarawagi IIT Bombay http://www.cse.iitb.ac.in/~sunita 1 Probabilistic

Graphical models Sunita Sarawagi IIT Bombay http://www.cse.iitb.ac.in/~sunita 1 Probabilistic

Graphical models Sunita Sarawagi IIT Bombay http://www.cse.iitb.ac.in/~sunita 1 Probabilistic modeling Given: several variables: x 1 , . . . x n , n is large. Task: build a joint distribution function Pr( x 1 , . . . x n ) Goal: Answer several

1.42k views • 42 slides
Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of

Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of

Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2019 CS 551, Spring 2019 2019, Selim Aksoy (Bilkent University)

599 views • 26 slides
Graphical Models L eon Bottou COS 424 4/15/2010 Introduction People like drawings better

Graphical Models L eon Bottou COS 424 4/15/2010 Introduction People like drawings better

Graphical Models L eon Bottou COS 424 4/15/2010 Introduction People like drawings better than equations A graphical model is a diagram representing certain aspects of the algebraic structure of a probabilistic model. Purposes

647 views • 37 slides
Part 2: Introduction to Graphical Models Sebastian Nowozin and Christoph H. Lampert Providence,

Part 2: Introduction to Graphical Models Sebastian Nowozin and Christoph H. Lampert Providence,

Graphical Models Factor Graphs Test-time Inference Training Software Part 2: Introduction to Graphical Models Sebastian Nowozin and Christoph H. Lampert Providence, 21st June 2012 Sebastian Nowozin and Christoph H. Lampert Part 2:

809 views • 56 slides
Graphical models and inference II Milos Hauskrecht milos@pitt.edu 5329 Sennott Square, x4-8845

Graphical models and inference II Milos Hauskrecht milos@pitt.edu 5329 Sennott Square, x4-8845

CS 3750 Machine Learning Lecture 3 Graphical models and inference II Milos Hauskrecht milos@pitt.edu 5329 Sennott Square, x4-8845 http://www.cs.pitt.edu/~milos/courses/cs3750-Spring2020/ CS 3750 Advanced Machine Learning Challenges for

299 views • 27 slides
Directed Random Graphs with Given Degree Distributions Mariana Olvera-Cravioto Columbia

Directed Random Graphs with Given Degree Distributions Mariana Olvera-Cravioto Columbia

Directed Random Graphs with Given Degree Distributions Mariana Olvera-Cravioto Columbia University molvera@ieor.columbia.edu Joint work with Ningyuan Chen July 23th, 2012 ECT, Trento, Italy Directed Random Graphs with Given Degree

421 views • 39 slides
Model-based Deep Hand Pose Estimation Xingyi Zhou, Qingfu Wan, Wei Zhang, Xiangyang Xue, Yichen

Model-based Deep Hand Pose Estimation Xingyi Zhou, Qingfu Wan, Wei Zhang, Xiangyang Xue, Yichen

Motivation Previous Works Method Experiment Conclusion Model-based Deep Hand Pose Estimation Xingyi Zhou, Qingfu Wan, Wei Zhang, Xiangyang Xue, Yichen Wei Fudan University & Microsoft Research July 7, 2016 Motivation Previous Works

159 views • 14 slides

More recommend