belief network inference
play

Belief network inference Four main approaches to determine posterior - PowerPoint PPT Presentation

Mar 29, 2023 •40 likes •330 views

Belief network inference Four main approaches to determine posterior distributions in belief networks: Variable Elimination: exploit the structure of the network to eliminate (sum out) the non-observed, non-query variables one at a time.

  1. Belief network inference Four main approaches to determine posterior distributions in belief networks: Variable Elimination: exploit the structure of the network to eliminate (sum out) the non-observed, non-query variables one at a time. Search-based approaches: enumerate some of the possible worlds, and estimate posterior probabilities from the worlds generated. Stochastic simulation: random cases are generated according to the probability distributions. Variational methods: find the closest tractable distribution to the (posterior) distribution we are interested in. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 1

  2. Factors A factor is a representation of a function from a tuple of random variables into a number. We will write factor f on variables X 1 , . . . , X j as f ( X 1 , . . . , X j ). We can assign some or all of the variables of a factor: f ( X 1 = v 1 , X 2 , . . . , X j ), where v 1 ∈ dom ( X 1 ), is a factor on X 2 , . . . , X j . f ( X 1 = v 1 , X 2 = v 2 , . . . , X j = v j ) is a number that is the value of f when each X i has value v i . The former is also written as f ( X 1 , X 2 , . . . , X j ) X 1 = v 1 , etc. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 2

  3. Example factors val Y Z X Y Z val t t 0.1 t t t 0.1 r ( X = t , Y , Z ): t f t t f 0.9 f t t f t 0.2 f f r ( X , Y , Z ): t f f 0.8 f t t 0.4 f t f 0.6 f f t 0.3 f f f 0.7 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 3

  4. Example factors val Y Z X Y Z val t t 0.1 t t t 0.1 r ( X = t , Y , Z ): t f 0.9 t t f 0.9 f t 0.2 t f t 0.2 f f 0.8 r ( X , Y , Z ): t f f 0.8 f t t 0.4 f t f 0.6 r ( X = t , Y , Z = f ): f f t 0.3 f f f 0.7 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 4

  5. Example factors val Y Z X Y Z val t t 0.1 t t t 0.1 r ( X = t , Y , Z ): t f t t f 0.9 f t t f t 0.2 f f r ( X , Y , Z ): t f f 0.8 f t t 0.4 Y val f t f 0.6 r ( X = t , Y , Z = f ): t f f t 0.3 f f f f 0.7 r ( X = t , Y = f , Z = f ) = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 5

  6. Example factors val Y Z X Y Z val t t 0.1 t t t 0.1 r ( X = t , Y , Z ): t f t t f 0.9 f t t f t 0.2 f f r ( X , Y , Z ): t f f 0.8 f t t 0.4 Y val f t f 0.6 r ( X = t , Y , Z = f ): t 0.9 f f t 0.3 f 0.8 f f f 0.7 r ( X = t , Y = f , Z = f ) = 0 . 8 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 6

  7. Multiplying factors The product of factor f 1 ( X , Y ) and f 2 ( Y , Z ), where Y are the variables in common, is the factor ( f 1 × f 2 )( X , Y , Z ) defined by: ( f 1 × f 2 )( X , Y , Z ) = f 1 ( X , Y ) f 2 ( Y , Z ) . � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 7

  8. Multiplying factors example A B val t t 0.1 val A B C f 1 : t f 0.9 t t t 0.03 f t 0.2 t t f f f 0.8 t f t f 1 × f 2 : t f f B C val f t t t t 0.3 f t f f 2 : t f 0.7 f f t f t 0.6 f f f f f 0.4 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 8

  9. Multiplying factors example A B val t t 0.1 val A B C f 1 : t f 0.9 t t t 0.03 f t 0.2 t t f 0.07 f f 0.8 t f t 0.54 f 1 × f 2 : t f f 0.36 B C val f t t 0.06 t t 0.3 f t f 0.14 f 2 : t f 0.7 f f t 0.48 f t 0.6 f f f 0.32 f f 0.4 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 9

  10. Summing out variables We can sum out a variable, say X 1 with domain { v 1 , . . . , v k } , from factor f ( X 1 , . . . , X j ), resulting in a factor on X 2 , . . . , X j defined by: � ( f )( X 2 , . . . , X j ) X 1 = f ( X 1 = v 1 , . . . , X j ) + · · · + f ( X 1 = v k , . . . , X j ) � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 10

  11. Summing out a variable example val A B C t t t 0.03 val t t f 0.07 A C t f t 0.54 t t 0.57 f 3 : t f f 0.36 � B f 3 : t f f t t 0.06 f t f t f 0.14 f f f f t 0.48 f f f 0.32 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 11

  12. Summing out a variable example val A B C t t t 0.03 val t t f 0.07 A C t f t 0.54 t t 0.57 f 3 : t f f 0.36 � B f 3 : t f 0.43 f t t 0.06 f t 0.54 f t f 0.14 f f 0.46 f f t 0.48 f f f 0.32 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 12

  13. Exercise Given factors: val A B A val t t 0.6 A val s: t 0.75 t: t f 0.4 o: t 0.3 f 0.25 f t 0.2 f 0.1 f f 0.8 What is? (a) s × t (b) � A s × t (c) � B s × t (d) s × t × o (e) � A s × t × o (f) � b s × t × o � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 13

  14. Evidence If we want to compute the posterior probability of Z given evidence Y 1 = v 1 ∧ . . . ∧ Y j = v j : P ( Z | Y 1 = v 1 , . . . , Y j = v j ) = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 14

  15. Evidence If we want to compute the posterior probability of Z given evidence Y 1 = v 1 ∧ . . . ∧ Y j = v j : P ( Z | Y 1 = v 1 , . . . , Y j = v j ) P ( Z , Y 1 = v 1 , . . . , Y j = v j ) = P ( Y 1 = v 1 , . . . , Y j = v j ) = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 15

  16. Evidence If we want to compute the posterior probability of Z given evidence Y 1 = v 1 ∧ . . . ∧ Y j = v j : P ( Z | Y 1 = v 1 , . . . , Y j = v j ) P ( Z , Y 1 = v 1 , . . . , Y j = v j ) = P ( Y 1 = v 1 , . . . , Y j = v j ) P ( Z , Y 1 = v 1 , . . . , Y j = v j ) = � Z P ( Z , Y 1 = v 1 , . . . , Y j = v j ) . So the computation reduces to the probability of P ( Z , Y 1 = v 1 , . . . , Y j = v j ). We normalize at the end. � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 16

  17. Probability of a conjunction Suppose the variables of the belief network are X 1 , . . . , X n . To compute P ( Z , Y 1 = v 1 , . . . , Y j = v j ), we sum out the other variables, Z 1 , . . . , Z k = { X 1 , . . . , X n } − { Z } − { Y 1 , . . . , Y j } . We order the Z i into an elimination ordering. P ( Z , Y 1 = v 1 , . . . , Y j = v j ) = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 17

  18. Probability of a conjunction Suppose the variables of the belief network are X 1 , . . . , X n . To compute P ( Z , Y 1 = v 1 , . . . , Y j = v j ), we sum out the other variables, Z 1 , . . . , Z k = { X 1 , . . . , X n } − { Z } − { Y 1 , . . . , Y j } . We order the Z i into an elimination ordering. P ( Z , Y 1 = v 1 , . . . , Y j = v j ) � � = · · · P ( X 1 , . . . , X n ) Y 1 = v 1 ,..., Y j = v j . Z k Z 1 = � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 18

  19. Probability of a conjunction Suppose the variables of the belief network are X 1 , . . . , X n . To compute P ( Z , Y 1 = v 1 , . . . , Y j = v j ), we sum out the other variables, Z 1 , . . . , Z k = { X 1 , . . . , X n } − { Z } − { Y 1 , . . . , Y j } . We order the Z i into an elimination ordering. P ( Z , Y 1 = v 1 , . . . , Y j = v j ) � � = · · · P ( X 1 , . . . , X n ) Y 1 = v 1 ,..., Y j = v j . Z k Z 1 n � � � = · · · P ( X i | parents ( X i )) Y 1 = v 1 ,..., Y j = v j . Z k Z 1 i =1 � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 19

  20. Computing sums of products Computation in belief networks reduces to computing the sums of products. How can we compute ab + ac efficiently? � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 20

  21. Computing sums of products Computation in belief networks reduces to computing the sums of products. How can we compute ab + ac efficiently? Distribute out the a giving a ( b + c ) � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 21

  22. Computing sums of products Computation in belief networks reduces to computing the sums of products. How can we compute ab + ac efficiently? Distribute out the a giving a ( b + c ) � n How can we compute � i =1 P ( X i | parents ( X i )) Z 1 efficiently? � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 22

  23. Computing sums of products Computation in belief networks reduces to computing the sums of products. How can we compute ab + ac efficiently? Distribute out the a giving a ( b + c ) � n How can we compute � i =1 P ( X i | parents ( X i )) Z 1 efficiently? Distribute out those factors that don’t involve Z 1 . � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 23

  24. Variable elimination algorithm To compute P ( Z | Y 1 = v 1 ∧ . . . ∧ Y j = v j ): Construct a factor for each conditional probability. Set the observed variables to their observed values. Sum out each of the other variables (the { Z 1 , . . . , Z k } ) according to some elimination ordering. Multiply the remaining factors. Normalize by dividing the resulting factor f ( Z ) by � Z f ( Z ). � D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture 6.4, Page 24

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Inference in Belief Networks CMPUT 366: Intelligent Systems P&M 8.4 Lecture Outline

Inference in Belief Networks CMPUT 366: Intelligent Systems P&M 8.4 Lecture Outline

Inference in Belief Networks CMPUT 366: Intelligent Systems P&M 8.4 Lecture Outline 1. Recap 2. Factors 3. Variable Elimination 4. Efficiency Recap: Belief Networks Definition: A belief network (or Bayesian network )

1.27k views • 23 slides
Bayesian Belief Network 14.4 Inference Decision Theoretic Agents Introduction to Probability

Bayesian Belief Network 14.4 Inference Decision Theoretic Agents Introduction to Probability

RN, Chapter Bayesian Belief Network 14.4 Inference Decision Theoretic Agents Introduction to Probability [Ch13] Belief networks [Ch14] Introduction [Ch14.1-14.2] Bayesian Net Inference [Ch14.4] (Bucket Elimination) Dynamic

763 views • 33 slides
Variable Elimination 1 Inference Exact inference Enumeration Variable elimination

Variable Elimination 1 Inference Exact inference Enumeration Variable elimination

Variable Elimination 1 Inference Exact inference Enumeration Variable elimination Junction trees and belief propagation Approximate inference Loopy belief propagation Sampling methods: likelihood weighting, Markov

447 views • 12 slides
Overview Independence Belief Networks Conditional Independence Belief networks Chris

Overview Independence Belief Networks Conditional Independence Belief networks Chris

Overview Independence Belief Networks Conditional Independence Belief networks Chris Williams Constructing belief networks Inference in belief networks School of Informatics, University of Edinburgh Learning in belief

498 views • 6 slides
Belief Networks Some Belief Network references E. Charniak Bayesian Networks without

Belief Networks Some Belief Network references E. Charniak Bayesian Networks without

Belief Networks Some Belief Network references E. Charniak Bayesian Networks without Tears, AI Magazine Winter 1991, pp 50-63 Chris Williams, School of Informatics University of Edinburgh D. Heckerman, A Tutorial on Learning

486 views • 6 slides
Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical

Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical

Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical inference is the determination of an underlying probability distribution from observed data. Probabilistic Graphical Models 1 1 =

522 views • 17 slides
Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth

Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth

Belief Propagation for Spatial Network Embeddings Andrew Frank Alex Ihler Padhraic Smyth Department of Computer Science UC Irvine August 25, 2009 Outline 1 Graphical Models Markov Random Fields Inference 2 Self-Localization Problem

1.04k views • 79 slides
Travel Time Estimation using Approximate Belief States on a Hidden Markov Model Walid Krichene

Travel Time Estimation using Approximate Belief States on a Hidden Markov Model Walid Krichene

Travel Time Estimation using Approximate Belief States on a Hidden Markov Model Walid Krichene Overview Context Inference on a HMM Modeling framework and exact inference Approximate Inference: the Boyen-Koller algorithm Graph Partitioning

1.04k views • 82 slides
Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1

Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1

Approximate inference on planar graphs using Loop Calculus and Belief Propagation Vicen Gmez 1 Hilbert J.Kappen 1 M. Chertkov 2 1 Department of Biophysics Radboud University, Nijmegen, The Netherlands 2 Theoretical Division and Center of

711 views • 28 slides
A DYNAMIC BAYESIAN BELIEF NETWORK APPROACH FOR MODELING THE ATM NETWORK DELAYS Yi it Bekir

A DYNAMIC BAYESIAN BELIEF NETWORK APPROACH FOR MODELING THE ATM NETWORK DELAYS Yi it Bekir

A DYNAMIC BAYESIAN BELIEF NETWORK APPROACH FOR MODELING THE ATM NETWORK DELAYS Yi it Bekir Kaya Data Science Researcher at CAL, Aeronautics Graduate Student Prof. Gkhan nalhan Director of CAL Istanbul Technical University Controls

715 views • 67 slides
Learning a Belief Network If you know the structure have observed all of the variables

Learning a Belief Network If you know the structure have observed all of the variables

Learning a Belief Network If you know the structure have observed all of the variables have no missing data you can learn each conditional probability separately. D. Poole and A. Mackworth 2010 c Artificial Intelligence, Lecture

479 views • 17 slides
Learning a Belief Network If you know the structure have observed all of the variables

Learning a Belief Network If you know the structure have observed all of the variables

Learning a Belief Network If you know the structure have observed all of the variables have no missing data you can learn each conditional probability separately. D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture

434 views • 40 slides
Generative vs. discriminative Generative Discriminative Belief network A is more More

Generative vs. discriminative Generative Discriminative Belief network A is more More

Generative vs. discriminative Generative Discriminative Belief network A is more More robust modular Dont need precise model Class-conditional densities are specification, so long as it is likely to be local, characteristic

1.09k views • 15 slides
Meta-Amoruized Variational Inference and Learning Kristy Choi CS236: December 4th, 2019

Meta-Amoruized Variational Inference and Learning Kristy Choi CS236: December 4th, 2019

Meta-Amoruized Variational Inference and Learning Kristy Choi CS236: December 4th, 2019 Probabilistic Inference Probabilistic inference is a paruicular way of viewing the world: + = Observations Updated (posterior) Prior belief Typically

452 views • 28 slides
Bayesian Belief Networks Decision Theoretic Agents Introduction to Probability [Ch13]

Bayesian Belief Networks Decision Theoretic Agents Introduction to Probability [Ch13]

RN, Chapter 14 Bayesian Belief Networks Decision Theoretic Agents Introduction to Probability [Ch13] Belief networks [Ch14] Introduction [Ch14.1-14.2] Bayesian Net Inference [Ch14.4] (Bucket Elimination) Dynamic Belief

634 views • 49 slides
GPU INFERENCE ENGINE Michael Andersch, 7 th April 2016 WHAT IS INFERENCE, ANYWAYS? Building a

GPU INFERENCE ENGINE Michael Andersch, 7 th April 2016 WHAT IS INFERENCE, ANYWAYS? Building a

April 4-7, 2016 | Silicon Valley NVIDIA GIE: HIGH-PERFORMANCE GPU INFERENCE ENGINE Michael Andersch, 7 th April 2016 WHAT IS INFERENCE, ANYWAYS? Building a deep neural network based application Step 1: Use data to train the neural network -

1.02k views • 17 slides
Draft Internal lenses as monad morphisms Bryce Clarke CoACT, Macquarie University,

Draft Internal lenses as monad morphisms Bryce Clarke CoACT, Macquarie University,

Draft Internal lenses as monad morphisms Bryce Clarke CoACT, Macquarie University, bryce.clarke1@students.mq.edu.au CT2019, July 12 Draft Outline Motivation Formal monads, functors, and cofunctors The 2-category of internal lenses Bonus:

462 views • 30 slides
To Toward Con Con st struction of of R Res R esili ilien ili i ent Soft t S f S ft ftwa

To Toward Con Con st struction of of R Res R esili ilien ili i ent Soft t S f S ft ftwa

To Toward Con Con st struction of of R Res R esili ilien ili i ent Soft t S f S ft ftwa ware re Def D f D f fi fi fine fi ned IT d IT IT Infra IT I f I f rastruc t ructur t ure fo for Su Supp pp pporti pp orting Di

185 views • 3 slides
A duality relation between loops and trees Germn Rodrigo in collaboration with Stefano Catani,

A duality relation between loops and trees Germn Rodrigo in collaboration with Stefano Catani,

EURO EUROFLA LAVOU OUR 20 R 2008, 22 08, 22-26 -26 Sep ep, DU DURHAM RHAM A duality relation between loops and trees Germn Rodrigo in collaboration with Stefano Catani, T anju Gleisberg, Frank Krauss, and Jan C. Winter JHEP 09 (2008)

580 views • 31 slides
Homework #7, Spring 2013 Version 2.3 corrected May 17 Background You are interested in

Homework #7, Spring 2013 Version 2.3 corrected May 17 Background You are interested in

Homework #7, Spring 2013 Version 2.3 corrected May 17 Background You are interested in estimating the date of divergence between two isolated populations of the Scots pine, Pinus sylvestris . These populations were reported by Naydenov et al.

303 views • 7 slides
Estimating Cloud Application Performance Based on Micro-Benchmark Profiling Joel Scheuner,

Estimating Cloud Application Performance Based on Micro-Benchmark Profiling Joel Scheuner,

Estimating Cloud Application Performance Based on Micro-Benchmark Profiling Joel Scheuner, Philipp Leitner Joel Scheuner ! scheuner@chalmers.se " joe4dev # @joe4dev Supported by Context: Public Infrastructure-as-a-Service Clouds IaaS

607 views • 26 slides
19th International Conference on Computational Statistics COMPSTAT 2010 Nuria Ruiz-Fuentes

19th International Conference on Computational Statistics COMPSTAT 2010 Nuria Ruiz-Fuentes

19th International Conference on Computational Statistics COMPSTAT 2010 Nuria Ruiz-Fuentes Paula R. Bouzas, Juan E. Ruiz-Castro University of Jan, Spain University of Granada, Spain Cox Process N (t) Information process Intensity

463 views • 13 slides
A Tree-Loop Duality Relation at Two Loops and Beyond Isabella Bierenbaum In collaboration with:

A Tree-Loop Duality Relation at Two Loops and Beyond Isabella Bierenbaum In collaboration with:

A Tree-Loop Duality Relation at Two Loops and Beyond Isabella Bierenbaum In collaboration with: S. Catani, P. Draggiotis, G. Rodrigo References: Catani, Gleisberg, Krauss, Rodrigo, Winter JHEP 0809 (2008) 065 Bierenbaum, Catani, Draggiotis,

456 views • 24 slides
UP- SKILLING FOR TODAYS FORESTRY AND TOMORROWS TIMBER PRODUCTS Roots for Future Growth

UP- SKILLING FOR TODAYS FORESTRY AND TOMORROWS TIMBER PRODUCTS Roots for Future Growth

UP- SKILLING FOR TODAYS FORESTRY AND TOMORROWS TIMBER PRODUCTS Roots for Future Growth 2011 - 2014 ambitious & strategic vision for the Scottish FTT industry. The four strategic aims were : Grow and communicate the

633 views • 26 slides

More recommend