maximum likelihood vs least squares for estimating

play

Maximum Likelihood vs. Least Squares for Estimating Mixtures of - PowerPoint PPT Presentation

Aug 29, 2022 •681 likes •1.41k views

Maximum Likelihood vs. Least Squares for Estimating Mixtures of Truncated Exponentials Helge Langseth 1 Thomas D. Nielsen 2 Rafael Rum 3 Antonio Salmern 3 1 Department of Computer and Information Science The Norwegian University of Science and

Maximum Likelihood vs. Least Squares for Estimating Mixtures of Truncated Exponentials Helge Langseth 1 Thomas D. Nielsen 2 Rafael Rumí 3 Antonio Salmerón 3 1 Department of Computer and Information Science The Norwegian University of Science and Technology, Trondheim (Norway) 2 Department of Computer Science Aalborg University, Aalborg (Denmark) 3 Department of Statistics and Applied Mathematics University of Almería, Almería (Spain) INFORMS, Seattle, November 2007 H. Langseth et al. ML vs. LS for estimating MTEs
Outline 1 Motivation. 2 The MTE (Mixture of Truncated Exponentials) model. 3 Maximum Likelihood (ML) estimation for MTEs. 4 Least Squares (LS) estimation of MTEs. 5 Experimental analysis. 6 Conclusions. H. Langseth et al. ML vs. LS for estimating MTEs
Outline 1 Motivation. 2 The MTE (Mixture of Truncated Exponentials) model. 3 Maximum Likelihood (ML) estimation for MTEs. 4 Least Squares (LS) estimation of MTEs. 5 Experimental analysis. 6 Conclusions. H. Langseth et al. ML vs. LS for estimating MTEs
Outline 1 Motivation. 2 The MTE (Mixture of Truncated Exponentials) model. 3 Maximum Likelihood (ML) estimation for MTEs. 4 Least Squares (LS) estimation of MTEs. 5 Experimental analysis. 6 Conclusions. H. Langseth et al. ML vs. LS for estimating MTEs
Outline 1 Motivation. 2 The MTE (Mixture of Truncated Exponentials) model. 3 Maximum Likelihood (ML) estimation for MTEs. 4 Least Squares (LS) estimation of MTEs. 5 Experimental analysis. 6 Conclusions. H. Langseth et al. ML vs. LS for estimating MTEs
Outline 1 Motivation. 2 The MTE (Mixture of Truncated Exponentials) model. 3 Maximum Likelihood (ML) estimation for MTEs. 4 Least Squares (LS) estimation of MTEs. 5 Experimental analysis. 6 Conclusions. H. Langseth et al. ML vs. LS for estimating MTEs
Outline 1 Motivation. 2 The MTE (Mixture of Truncated Exponentials) model. 3 Maximum Likelihood (ML) estimation for MTEs. 4 Least Squares (LS) estimation of MTEs. 5 Experimental analysis. 6 Conclusions. H. Langseth et al. ML vs. LS for estimating MTEs
Motivation Graphical models are a common tool for decision analysis. Problems in which continuous and discrete variables interact are frequent. A very general solution is the use of MTE models. When learning from data, the only existing method in the literature is based on least squares. The feasibility of ML estimation seems to be worth studying. H. Langseth et al. ML vs. LS for estimating MTEs
Motivation Graphical models are a common tool for decision analysis. Problems in which continuous and discrete variables interact are frequent. A very general solution is the use of MTE models. When learning from data, the only existing method in the literature is based on least squares. The feasibility of ML estimation seems to be worth studying. H. Langseth et al. ML vs. LS for estimating MTEs
Motivation Graphical models are a common tool for decision analysis. Problems in which continuous and discrete variables interact are frequent. A very general solution is the use of MTE models. When learning from data, the only existing method in the literature is based on least squares. The feasibility of ML estimation seems to be worth studying. H. Langseth et al. ML vs. LS for estimating MTEs
Motivation Graphical models are a common tool for decision analysis. Problems in which continuous and discrete variables interact are frequent. A very general solution is the use of MTE models. When learning from data, the only existing method in the literature is based on least squares. The feasibility of ML estimation seems to be worth studying. H. Langseth et al. ML vs. LS for estimating MTEs
Motivation Graphical models are a common tool for decision analysis. Problems in which continuous and discrete variables interact are frequent. A very general solution is the use of MTE models. When learning from data, the only existing method in the literature is based on least squares. The feasibility of ML estimation seems to be worth studying. H. Langseth et al. ML vs. LS for estimating MTEs
Bayesian networks X 1 X 3 X 2 X 4 X 5 D.A.G. The nodes represent random variables. Arc ⇒ dependence. n p ( x ) = p ( x i | π i ) x ∈ Ω X � i = 1 H. Langseth et al. ML vs. LS for estimating MTEs
Bayesian networks X 1 X 2 X 3 X 4 X 5 D.A.G. The nodes represent random variables. Arc ⇒ dependence. p ( x 1 , x 2 , x 3 , x 4 , x 5 ) = p ( x 1 ) p ( x 2 | x 1 ) p ( x 3 | x 1 ) p ( x 5 | x 3 ) p ( x 4 | x 2 , x 3 ) . H. Langseth et al. ML vs. LS for estimating MTEs
The MTE model (Moral et al. 2001) Definition (MTE potential) X : mixed n -dimensional random vector. Y = ( Y 1 , . . . , Y d ) , Z = ( Z 1 , . . . , Z c ) its discrete and continuous parts. A function f : Ω X �→ R + 0 is a Mixture of Truncated Exponentials potential (MTE potential) if for each fixed value y ∈ Ω Y of the discrete variables Y , the potential over the continuous variables Z is defined as:   m c b ( j )   f ( z ) = a 0 + a i exp i z j � � i = 1 j = 1   for all z ∈ Ω Z , where a i , b ( j ) are real numbers. i Also, f is an MTE potential if there is a partition D 1 , . . . , D k of Ω Z into hypercubes and in each D i , f is defined as above. H. Langseth et al. ML vs. LS for estimating MTEs
The MTE model (Moral et al. 2001) Definition (MTE potential) X : mixed n -dimensional random vector. Y = ( Y 1 , . . . , Y d ) , Z = ( Z 1 , . . . , Z c ) its discrete and continuous parts. A function f : Ω X �→ R + 0 is a Mixture of Truncated Exponentials potential (MTE potential) if for each fixed value y ∈ Ω Y of the discrete variables Y , the potential over the continuous variables Z is defined as:   m c b ( j )   f ( z ) = a 0 + a i exp i z j � � i = 1 j = 1   for all z ∈ Ω Z , where a i , b ( j ) are real numbers. i Also, f is an MTE potential if there is a partition D 1 , . . . , D k of Ω Z into hypercubes and in each D i , f is defined as above. H. Langseth et al. ML vs. LS for estimating MTEs
The MTE model (Moral et al. 2001) Example Consider a model with continuous variables X and Y , and discrete variable Z . X Y Z H. Langseth et al. ML vs. LS for estimating MTEs
The MTE model (Moral et al. 2001) Example One example of conditional densities for this model is given by the following expressions: � 1 . 16 − 1 . 12 e − 0 . 02 x if 0 . 4 ≤ x < 4 , f ( x ) = 0 . 9 e − 0 . 35 x if 4 ≤ x < 19 . 1 . 26 − 1 . 15 e 0 . 006 y  if 0 . 4 ≤ x < 5 , 0 ≤ y < 13 ,   1 . 18 − 1 . 16 e 0 . 0002 y  if 0 . 4 ≤ x < 5 , 13 ≤ y < 43 ,  f ( y | x ) = 0 . 07 − 0 . 03 e − 0 . 4 y + 0 . 0001 e 0 . 0004 y if 5 ≤ x < 19 , 0 ≤ y < 5 ,   − 0 . 99 + 1 . 03 e 0 . 001 y   if 5 ≤ x < 19 , 5 ≤ y < 43 . if z = 0 , 0 . 4 ≤ x < 5 ,  0 . 3   if z = 1 , 0 . 4 ≤ x < 5 ,  0 . 7  f ( z | x ) = if z = 0 , 5 ≤ x < 19 , 0 . 6   if z = 1 , 5 ≤ x < 19 .  0 . 4  H. Langseth et al. ML vs. LS for estimating MTEs
The MTE model (Moral et al. 2001) Example One example of conditional densities for this model is given by the following expressions: � 1 . 16 − 1 . 12 e − 0 . 02 x if 0 . 4 ≤ x < 4 , f ( x ) = 0 . 9 e − 0 . 35 x if 4 ≤ x < 19 . 1 . 26 − 1 . 15 e 0 . 006 y  if 0 . 4 ≤ x < 5 , 0 ≤ y < 13 ,   1 . 18 − 1 . 16 e 0 . 0002 y  if 0 . 4 ≤ x < 5 , 13 ≤ y < 43 ,  f ( y | x ) = 0 . 07 − 0 . 03 e − 0 . 4 y + 0 . 0001 e 0 . 0004 y if 5 ≤ x < 19 , 0 ≤ y < 5 ,   − 0 . 99 + 1 . 03 e 0 . 001 y   if 5 ≤ x < 19 , 5 ≤ y < 43 . if z = 0 , 0 . 4 ≤ x < 5 ,  0 . 3   if z = 1 , 0 . 4 ≤ x < 5 ,  0 . 7  f ( z | x ) = if z = 0 , 5 ≤ x < 19 , 0 . 6   if z = 1 , 5 ≤ x < 19 .  0 . 4  H. Langseth et al. ML vs. LS for estimating MTEs
The MTE model (Moral et al. 2001) Example One example of conditional densities for this model is given by the following expressions: � 1 . 16 − 1 . 12 e − 0 . 02 x if 0 . 4 ≤ x < 4 , f ( x ) = 0 . 9 e − 0 . 35 x if 4 ≤ x < 19 . 1 . 26 − 1 . 15 e 0 . 006 y  if 0 . 4 ≤ x < 5 , 0 ≤ y < 13 ,   1 . 18 − 1 . 16 e 0 . 0002 y  if 0 . 4 ≤ x < 5 , 13 ≤ y < 43 ,  f ( y | x ) = 0 . 07 − 0 . 03 e − 0 . 4 y + 0 . 0001 e 0 . 0004 y if 5 ≤ x < 19 , 0 ≤ y < 5 ,   − 0 . 99 + 1 . 03 e 0 . 001 y   if 5 ≤ x < 19 , 5 ≤ y < 43 . if z = 0 , 0 . 4 ≤ x < 5 ,  0 . 3   if z = 1 , 0 . 4 ≤ x < 5 ,  0 . 7  f ( z | x ) = if z = 0 , 5 ≤ x < 19 , 0 . 6   if z = 1 , 5 ≤ x < 19 .  0 . 4  H. Langseth et al. ML vs. LS for estimating MTEs
Learning MTEs from data In this work we are concerned with the univariate case. The learning task involves three basic steps: Determination of the splits into which Ω X will be partitioned. Determination of the number of exponential terms in the mixture for each split. Estimation of the parameters. H. Langseth et al. ML vs. LS for estimating MTEs

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

More recommend