Flexible Discriminant Analysis Using Motivation MGLMM Multivariate - PowerPoint PPT Presentation

Flexible Discriminant Analysis Using Multivariate Mixed Models D. Hughes Flexible Discriminant Analysis Using Motivation MGLMM Multivariate Mixed Models Discriminant Analysis ISDR Example Conclusions David Hughes 2015

Flexible Outline Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis ISDR Example 1. Motivation Conclusions 2. Multivariate Generalized Linear Mixed Models (MGLMM) 3. Longitudinal Discriminant Analysis 4. ISDR Example 5. Conclusions

Flexible Motivation Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis ISDR Example ◮ Complex data. Conclusions

Flexible Motivation Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis ISDR Example ◮ Complex data. Conclusions ◮ Longitudinal

Flexible Motivation Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis ISDR Example ◮ Complex data. Conclusions ◮ Longitudinal ◮ Multivariate

Flexible Motivation Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis ISDR Example ◮ Complex data. Conclusions ◮ Longitudinal ◮ Multivariate ◮ Different types of data

Flexible Motivation Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis ISDR Example ◮ Complex data. Conclusions ◮ Longitudinal ◮ Multivariate ◮ Different types of data ◮ Complicated correlation structure

Flexible Motivation Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis ISDR Example ◮ Complex data. Conclusions ◮ Longitudinal ◮ Multivariate ◮ Different types of data ◮ Complicated correlation structure

Flexible Available Methods Discriminant Analysis Using Multivariate Mixed Models D. Hughes ◮ Univariate models using a classical linear mixed model (e.g Motivation MGLMM Brant et al. (2003), Lix and Sajobi (2010), Tomasko et al. Discriminant (1999) and Wernecke et al. (2004)). Analysis ◮ Fails to account properly for the dependence between markers ISDR Example in our case. Conclusions ◮ Multivariate Models for continuous markers using multivariate mixed models (eg Morrell et al. (2012) using linear mixed models and Marshall et al. (2009) using non-linear mixed models). ◮ Not applicable if some of the markers are not continuous. ◮ Pairwise models for continuous and binary markers (Fieuws et al. (2008)). ◮ This method in principle is suitable for our purposes but in this talk we outline a more flexible approach.

Flexible A more flexible approach Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM ◮ Typical assumption about the random effects distribution can Discriminant be relaxed by using a mixture of normal distributions (Kom´ arek Analysis et al. (2010)). ISDR Example ◮ This methodology only considers three continuous markers. Conclusions ◮ Cluster Analysis with continuous, binary and count variables with mixture distributions for the random effects is possible (Kom´ arek and Kom´ arekov´ a (2013)) ◮ In Cluster Analysis the groups are unknown whereas in our case groups are known beforehand. ◮ Software is available in the mixAK package in R created by Arnoˇ st Kom´ arek.

Flexible Progress Map Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis Dataset for Analysis ISDR Example Conclusions Fitting of the multivariate mixed-effects model (MGLMM) Discriminant model built using parameters of MGLMM Allocate new patients to diagnostic groups

Flexible Definitions Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation ◮ Y i , r , j is the j ‘th observation of the r ‘th marker for patient i and MGLMM Discriminant is measured at time t i , r , j . Analysis ◮ We consider r = 1 , . . . , R markers on i = 1 , . . . , N patients. ISDR Example Conclusions ◮ Y i , r is a vector containing all observations of marker r for patient i . ◮ Y i is a stacked vector containing all the observations of all markers for patient i . ◮ Distribution of each marker may depend on additional covariates such as time, Age, Gender. ◮ It is possible for each marker to be measured at different time points and a different number of times.

Flexible Multivariate Generalized Linear Mixed Models Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation ◮ To allow for different types of marker we model each marker MGLMM using a generalised linear mixed model Discriminant Analysis h − 1 [ E ( Y i , r | α r , b i , r )] = X i , r α r + Z i , r b i , r (1) ISDR Example r Conclusions

Flexible Multivariate Generalized Linear Mixed Models Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation ◮ To allow for different types of marker we model each marker MGLMM using a generalised linear mixed model Discriminant Analysis h − 1 [ E ( Y i , r | α r , b i , r )] = X i , r α r + Z i , r b i , r (1) ISDR Example r Conclusions ◮ h r is a link function used depending on the type of longitudinal marker. ◮ α r is a vector of fixed parameters for marker r . ◮ b i , r is a vector of random effects for patient i for marker r (i.e subject specific parameters). ◮ X and Z are matrices containing covariate information for each patient.

Flexible Joint Distribution of the random effects Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation ◮ The dependence between markers is captured by the joint MGLMM distribution of the random effects b i = ( b i , 1 , . . . , b i , R ), Discriminant i = 1 , . . . , N . Analysis ISDR Example ◮ The most common assumption is that the random effects Conclusions follow a Normal distribution. b i ∼ N ( µ, D ) (2) ◮ This assumption can be difficult to verify and additional flexibility can be achieved by allowing a mixture of Normal distributions. K � b i ∼ w k N ( µ k , D k ) (3) k =1

Flexible Parameter Estimation Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM ◮ We need to estimate the following parameters. Discriminant Analysis ISDR Example Conclusions

Flexible Parameter Estimation Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM ◮ We need to estimate the following parameters. Discriminant Analysis ◮ Fixed effects α = ( α 1 , . . . , α R ) ISDR Example ◮ Possible dispersion parameters φ = ( φ 1 , . . . , φ R ) Conclusions ◮ Mixture weights w = ( w 1 , . . . , w K ) ◮ Mean vector of random effects µ = ( µ 1 , . . . , µ K ) ◮ Covariance matrix of random effects ( vec ( D 1 ) , . . . , vec ( D K )) ◮ In all, we need to estimate, θ = ( α , φ , w , µ , vec ( D 1 ) , . . . , vec ( D K )) (4)

Flexible MCMC estimates Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant ◮ Full maximum likelihood estimates are difficult to obtain due Analysis to the complexity of the likelihood. ISDR Example Conclusions ◮ We instead use a Bayesian approach based on MCMC. ◮ We utilise weakly informative priors and a block Gibbs sampler. ◮ A benefit of this method, not explored in this talk is that credible intervals for the group membership probabilities are readily available. These could be incorporated into a classification procedure in some cases.

Flexible Progress Map Discriminant Analysis Using Multivariate Mixed Models D. Hughes Motivation MGLMM Discriminant Analysis Dataset for Analysis ISDR Example Conclusions Fitting of the multivariate mixed-effects model (MGLMM) Discriminant model built using parameters of MGLMM Allocate new patients to diagnostic groups

Flexible Longitudinal Discriminant Analysis Discriminant Analysis Using Multivariate Mixed Models D. Hughes ◮ Fit MGLMM to data in each diagnostic group g , g = 1 , . . . , G to obtain MCMC parameter estimates, ˆ Motivation θ g . MGLMM ◮ Use the fitted GLMM model to derive the discriminant rule Discriminant Analysis that assigns the patients into two (or more) diagnostic groups. ISDR Example ◮ Let ˆ P g , new be the probability that a new observation Y i , is Conclusions from group g . ◮ The prior probability of being in group g is denoted π g . ◮ Using Bayes rule it can be seen that π g ˆ f g , new ˆ P g , new = (5) � G − 1 h =0 π h ˆ f h , new ◮ Assign new patients to disease group if ˆ P disease , new is greater than a specified value. If not assign to the group for which ˆ P g , new is largest.

Flexible Specifying the predictive density f g , new Discriminant Analysis Using Multivariate Mixed Models D. Hughes ◮ Marginal Prediction Motivation MGLMM f marg Discriminant g , new = p ( y new | θ g ) (6) Analysis ISDR Example ◮ Conditional Prediction Conclusions g , new = p ( y new | b new = ˜ f cond b g , new , θ g ) (7) ◮ Random Effects Prediction g , new = p (˜ f rand b g , new | θ g ) (8) ◮ These values are calculated using numerical integration methods such as Gauss Quadrature since they involve complex integrals that cannot be solved analytically.