mpe inference in conditional linear gaussian networks
play

MPE inference in Conditional Linear Gaussian Networks Antonio Salmern - PowerPoint PPT Presentation

MPE inference in Conditional Linear Gaussian Networks Antonio Salmern 1 Rafael Rum 1 Helge Langseth 2 Anders L. Madsen 3 , 4 Thomas D. Nielsen 4 1 Dept. Mathematics, University of Almera, Spain 2 Dept. Computer and Information Science.


  1. MPE inference in Conditional Linear Gaussian Networks Antonio Salmerón 1 Rafael Rumí 1 Helge Langseth 2 Anders L. Madsen 3 , 4 Thomas D. Nielsen 4 1 Dept. Mathematics, University of Almería, Spain 2 Dept. Computer and Information Science. Norwegian University of Science and Technology, Trondheim, Norway 3 Hugin Expert A/S, Aalborg, Denmark 4 Dept. Computer Science, Aalborg University, Denmark ECSQARU 2015, Compiegne, July 17, 2015 1

  2. Introduction ◮ The AMiDST project: Analysis of MassIve Data STreams http://www.amidst.eu ECSQARU 2015, Compiegne, July 17, 2015 2

  3. Introduction ◮ The AMiDST project: Analysis of MassIve Data STreams http://www.amidst.eu ◮ Large number of variables ◮ Queries to be answered in real time ◮ Hybrid Bayesian networks (involving discrete and continuous variables) ◮ Conditional linear Gaussian networks ECSQARU 2015, Compiegne, July 17, 2015 3

  4. Conditional Linear Gaussian networks A Conditional Linear Gaussian (CLG) network is a hybrid Bayesian network where ◮ The conditional distribution of each discrete variable X D given its parents is a multinomial ◮ The conditional distribution of each continuous variable Z with discrete parents X D and continuous parents X C , is p ( z | X D = x D , X C = x C ) = N ( z ; α ( x D ) + β ( x D ) T x C , σ ( x D )) for all x D and x C , where α and β are the coefficients of a linear regression model of Z given X C , potentially different for each configuration of X D . ECSQARU 2015, Compiegne, July 17, 2015 4

  5. Conditional Linear Gaussian networks. Example P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) Y f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) f ( t | w , S = 0 ) = N ( t ; − w , 1 ) W S f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) U T ECSQARU 2015, Compiegne, July 17, 2015 5

  6. Querying a Bayesian network (I) ◮ Probabilistic inference: Computing the posterior distribution of a target variable: � � p ( x , x E ) d x C x C x D p ( x i | x E ) = � � p ( x , x E ) d x C i x Ci x Di ECSQARU 2015, Compiegne, July 17, 2015 6

  7. Querying a Bayesian network (II) ◮ Maximum a posteriori (MAP): For a set of target variables X I , the goal is to compute x ∗ I = arg max p ( x I | X E = x E ) x I where p ( x I | X E = x E ) is obtained by first marginalizing out from p ( x ) the variables not in X I and not in X E ECSQARU 2015, Compiegne, July 17, 2015 7

  8. Querying a Bayesian network (II) ◮ Maximum a posteriori (MAP): For a set of target variables X I , the goal is to compute x ∗ I = arg max p ( x I | X E = x E ) x I where p ( x I | X E = x E ) is obtained by first marginalizing out from p ( x ) the variables not in X I and not in X E ◮ Most probable explanation (MPE): A particular case of MAP where X I includes all the unobserved variables ECSQARU 2015, Compiegne, July 17, 2015 8

  9. MPE in CLG networks ◮ Can be carried out using bucket elimination (Dechter, 1999): ◮ A bucket containing probability functions is kept for each variable. ◮ Initially, an ordering of the variables in the network is established, and each conditional distribution in the network is assigned to the bucket corresponding to the variable in its domain holding the highest rank. ◮ Afterwards, the buckets are processed in a sequence opposite to the initial ordering of the variables. ◮ Each bucket is processed by combining all the functions it contains and by marginalizing the main variable in that bucket by maximization. ECSQARU 2015, Compiegne, July 17, 2015 9

  10. Example P ( Y ) = ( 0 . 5 , 0 . 5 ) Y P ( S ) = ( 0 . 1 , 0 . 9 ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) W S f ( t | w , S = 0 ) = N ( t ; − w , 1 ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) U T Elimination order: Y , S , W , T , U ECSQARU 2015, Compiegne, July 17, 2015 10

  11. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) h W ( Y , S ) B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) f ( w | Y ) h T ( w , S ) h U ( w ) B W : f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 11

  12. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) h W ( Y , S ) B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) f ( w | Y ) h T ( w , S ) h U ( w ) B W : f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 12

  13. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) h W ( Y , S ) B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) f ( w | Y ) h T ( w , S ) h U ( w ) B W : f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 13

  14. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) h W ( Y , S ) B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) 1 f ( w | Y ) h T ( w , S ) √ B W : 2 π f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 14

  15. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) h W ( Y , S ) B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) 1 f ( w | Y ) h T ( w , S ) √ B W : 2 π f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 15

  16. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) h W ( Y , S ) B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) 1 1 f ( w | Y ) √ √ B W : 2 π 2 π f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 16

  17. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) h W ( Y , S ) B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) 1 1 f ( w | Y ) √ √ B W : 2 π 2 π f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 17

  18. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) 1 2 π ) 3 ( B S : P ( S ) √ f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) 1 1 f ( w | Y ) √ √ B W : 2 π 2 π f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 18

  19. Example Elimination order: Y , S , W , T , U P ( Y ) h S ( Y ) B Y : P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) 1 2 π ) 3 ( B S : P ( S ) √ f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) f ( w | Y = 1 ) = N ( w ; 2 , 1 ) 1 1 f ( w | Y ) √ √ B W : 2 π 2 π f ( t | w , S = 0 ) = N ( t ; − w , 1 ) B T : f ( t | w , S ) f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) f ( u | w ) B U : ECSQARU 2015, Compiegne, July 17, 2015 19

  20. Example Elimination order: Y , S , W , T , U 1 2 π ) 3 0 . 9 ( B Y : P ( Y ) √ P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) 1 2 π ) 3 ( √ B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) 1 1 f ( w | Y = 1 ) = N ( w ; 2 , 1 ) B W : f ( w | Y ) √ √ 2 π 2 π f ( t | w , S = 0 ) = N ( t ; − w , 1 ) f ( t | w , S ) B T : f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) B U : f ( u | w ) ECSQARU 2015, Compiegne, July 17, 2015 20

  21. Example Elimination order: Y , S , W , T , U 1 2 π ) 3 0 . 9 ( B Y : P ( Y ) √ P ( Y ) = ( 0 . 5 , 0 . 5 ) P ( S ) = ( 0 . 1 , 0 . 9 ) 1 2 π ) 3 ( √ B S : P ( S ) f ( w | Y = 0 ) = N ( w ; − 1 , 1 ) 1 1 f ( w | Y = 1 ) = N ( w ; 2 , 1 ) B W : f ( w | Y ) √ √ 2 π 2 π f ( t | w , S = 0 ) = N ( t ; − w , 1 ) f ( t | w , S ) B T : f ( t | w , S = 1 ) = N ( t ; w , 1 ) f ( u | w ) = N ( u ; w , 1 ) B U : f ( u | w ) The MPE configuration is obtained tracing back the steps ECSQARU 2015, Compiegne, July 17, 2015 21

  22. What we have learnt so far ◮ Marginalizing continuous variables is easy if they are the first to be marginalized out ◮ The price to pay is that, in the worst case, a function containing all the discrete variables would be created ◮ This complexity blow-up can be avoided in many cases by allowing orderings for constructing the buckets where discrete and continuous variables can be arranged with no restrictions ◮ But then a new problem arises, as the maximization operation becomes more complex ECSQARU 2015, Compiegne, July 17, 2015 22

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend