Model selection and parameter estimation with covariates in - PowerPoint PPT Presentation

Model selection and parameter estimation with missing Model selection and parameter estimation with covariates in logistic regression missing covariates in logistic regression models models Fabrizio Workshop on Model Selection 2008 Consentino & Gerda Claeskens Introduction Fabrizio Consentino & Gerda Claeskens Model selection criteria ORSTAT and Leuven Statistics Research Center Katholieke Universiteit Leuven Estimation Applications 24 July 2008 Conclusions

Overview Model selection and parameter estimation with missing 1 Introduction covariates in logistic regression models 2 Model selection criteria Fabrizio Consentino & Gerda Claeskens 3 Estimation Introduction Model selection 4 Applications criteria Estimation Applications 5 Conclusions Conclusions

Introduction Model selection and parameter estimation with missing covariates in logistic Problems regression models Model Selection: Fabrizio Consentino & Searching for the ’best’ model in order to explain the Gerda Claeskens phenomena of interest. Introduction Missing Data: Model Presence of missing observations in the data sets of selection criteria interest. Estimation Applications Conclusions

Introduction Model selection and parameter estimation Missing Data with missing covariates in logistic Missingness patterns: regression models Structure of the missing observations Fabrizio � 1 Consentino & X observed Gerda M = Claeskens 0 otherwise Introduction Missingness mechanisms: Model selection To describe the missing indicator M criteria Estimation Missing at random (MAR) Applications ⇒ f ( M | X obs , X mis ; θ ) = f ( M | X obs ) Conclusions

Introduction Model selection and parameter estimation with missing Assumptions covariates in logistic regression models Response variable Y fully observed Fabrizio Consentino & Design matrix X contains missing values Gerda Claeskens It can be partitioned in X = ( X obs , X mis ) Introduction Model MAR assumption selection criteria Estimation f ( Y , X ; θ ) = f ( Y | X ; β ) f ( X ; α ) Applications Conclusions

Introduction Model Method of Weights - EM algorithm selection and parameter Introduced by Ibrahim (1990), it is used in missing covariates data. estimation with missing covariates in It provides a weighted log-likelihood function in the E-step: logistic regression Q i ( θ | θ ( k ) ) = � w i log f ( y i , x i ; θ ) dx mis , i models Fabrizio Consentino & with w i = f ( x mis , i | x obs , i , y i ; θ ( k ) ) Gerda Claeskens Q ( θ | θ ( k ) ) = � n i =1 Q i ( θ | θ ( k ) ) is evaluated with a Monte Carlo EM Introduction algorithm and a Gibbs sampler along with the adaptive rejection Model selection algorithm of Gilks and Wild (1992) for sampling from criteria ( x mis , i | x obs , i , y i ; θ ( k ) ) Estimation f ( Y , X ; θ ) = f ( Y | X ; β ) f ( X ; α ) Applications Q i ( θ | θ ( k ) ) = � w i log f ( y i | x i ; β ) dx mis , i + � w i log f ( x i ; α ) dx mis , i Conclusions = Q (1) ( β | θ ( k ) ) + Q (2) ( α | θ ( k ) ) i i

Model Selection Criteria Model selection and parameter estimation with missing AIC and modifications covariates in logistic regression One of the most popular criteria. models Fabrizio AIC is twice a penalized log likelihood value, Consentino & Gerda Claeskens AIC = 2 log L n (ˆ θ ) − 2 length( θ ) Introduction Model Modification in the penalty term: selection p = tr ( � I − 1 � criteria Takeuchi: ˆ J ) n Estimation Hurvich and Tsai: ˆ p = 2 length( θ ) n − length ( θ ) − 1 Applications Conclusions

Model Selection Criteria Model selection and Derivation parameter estimation with missing covariates in Kullback-Leibler distance: logistic regression KL ( g , f θ ) = E g [log { g ( Y , X ) / f ( Y , X ; θ ) } ] models Fabrizio “Adjusted” likelihood function log ˜ f θ ( y , x ) = Q ( θ | θ ) Consentino & Gerda Claeskens Q ( θ | θ ) = � n � log f ( y i , x obs , i , x mis , i ; θ ) f ( x mis , i | x obs , i , y i , θ ) dx mis , i i =1 Introduction Kullback-Leibler distance: Model selection KL ( g , ˜ f θ ) = [ E g { log g ( Y , X ) } − E g { log ˜ f θ ( Y , X ) } ] / n criteria � � Estimation x ) log ˜ x ; � K n = g ( y , x ) g (˜ y , ˜ f (˜ y , ˜ θ ) d ˜ y d ˜ x d y d x / n Applications Conclusions An estimator of K n is � K n = Q ( � θ | � θ ) / n

Model Selection Criteria Model Criteria selection and parameter Following Takeuchi’s information criterion (Takeuchi, 1976), we estimation with missing define the model robust criterion TIC for missing covariate values as covariates in logistic regression TIC = 2 Q ( � θ | � θ ) − 2 tr { � J ( � θ ) � I − 1 ( � θ ) } models Fabrizio where Consentino & Gerda Claeskens n � θ ) = − 1 θ ) = 1 � I ( � Q ( � ¨ θ | � θ ) and � J ( � Q i ( � ˙ θ | � θ ) ˙ Q i ( � θ | � θ ) ′ . Introduction n n i =1 Model selection If the matrices I and J are equal, then the penalty in the expression criteria of the TIC reduces to the number of parameters in the model. This Estimation simplification leads to a version of Akaike’s information criterion Applications suitable for use with missing covariate information. Conclusions AIC = 2 Q ( � θ | � θ ) − 2 length( θ ) .

Model Selection Criteria Model selection and parameter estimation with missing Criteria covariates in logistic regression Claeskens and Consentino (2008, Biometrics) proposed models criteria using only Q (1) Fabrizio Consentino & Gerda Claeskens TIC 1 = 2 Q (1) ( � β | � β ) − 2 tr { � J ( � β ) � I − 1 ( � β ) } Introduction AIC 1 = 2 Q (1) ( � β | � β ) − 2 length( β ) Model selection ’Full’ Q function: Q (1) + Q (2) criteria Estimation If no missingness: Q = log L n and Q (2) = 0 Applications Conclusions

Applications Model selection and parameter estimation Simulation setting with missing covariates in logistic regression Logistic regression model models Fabrizio X 3 and X 4 generated independently from a standard Consentino & normal distribution. Gerda Claeskens X 1 and X 2 contain missing observations and are modeled Introduction using a bivariate normal regression ( X 1 , X 2 ) ∼ N 2 ( µ , Σ), Model µ ′ = ( µ i 1 , µ i 2 ) selection criteria The regressors are the fully observed covariates in X Estimation µ it = α t 0 + α t 1 x i 3 + α t 2 x i 4 , t = 1 , 2 Applications Conclusions

Applications Model selection and parameter estimation Simulation results with missing covariates in logistic regression % missing Criteria Model Correctly Model Correctly models x 1 , x 2 selection specified selection specified Fabrizio n = 50 n = 100 Consentino & C O U C O U Gerda 5% 5% TIC 1 0.530 0.280 0.190 0.810 0.650 0.340 0.010 0.990 Claeskens AIC 1 0.563 0.223 0.214 0.786 0.653 0.333 0.014 0.986 AIC orig 0.570 0.227 0.203 0.797 0.677 0.303 0.020 0.980 AIC cc 0.527 0.253 0.220 0.780 0.680 0.300 0.020 0.980 Introduction 10% 5% TIC 1 0.503 0.297 0.200 0.800 0.630 0.360 0.010 0.990 Model AIC 1 0.547 0.247 0.206 0.784 0.663 0.330 0.007 0.993 selection AIC orig 0.577 0.220 0.203 0.797 0.677 0.303 0.020 0.980 criteria AIC cc 0.507 0.230 0.263 0.737 0.670 0.310 0.020 0.980 15% 15% TIC 1 0.477 0.340 0.183 0.817 0.567 0.423 0.010 0.990 Estimation AIC 1 0.527 0.263 0.210 0.790 0.653 0.333 0.014 0.986 AIC orig 0.577 0.220 0.203 0.797 0.677 0.303 0.020 0.980 Applications AIC cc 0.443 0.233 0.324 0.676 0.640 0.317 0.043 0.957 Conclusions

Applications Model selection and parameter estimation with missing covariates in Simulation results logistic regression models % missing Criteria Model Correctly Fabrizio x 1 , x 2 selection specified Consentino & n = 50 Gerda C O U Claeskens 5% 5% TIC 1 0.530 0.280 0.190 0.810 MCAR AIC 1 0.563 0.223 0.214 0.786 AIC orig 0.570 0.227 0.203 0.797 Introduction AIC cc 0.527 0.253 0.220 0.780 Model 5% 5% TIC 1 0.433 0.423 0.144 0.856 selection AIC 1 0.470 0.340 0.190 0.810 MAR criteria AIC orig 0.583 0.280 0.137 0.863 AIC cc 0.437 0.193 0.370 0.630 Estimation Applications Conclusions

Model Selection Criteria Model selection and parameter Distribution selection estimation with missing covariates in Focussing on f ( X ; α ). logistic regression Q (2) used for deciding which distribution describes better models Fabrizio the missing covariates. Consentino & Gerda Criteria used Claeskens Introduction TIC = 2 Q ( � θ | � θ ) − 2 tr { � J ( � θ ) � I − 1 ( � θ ) } Model selection AIC = 2 Q ( � θ | � criteria θ ) − 2 length( θ ) . Estimation with Q ( � θ | � θ ) = Q (1) ( � β | � θ ) + Q (2) ( � α | � θ ) Applications Conclusions Main drawback: computationally intense.