A profile likelihood approach to longitudinal data Speaker: Ziqi - - PowerPoint PPT Presentation
A profile likelihood approach to longitudinal data Speaker: Ziqi Chen Central South University Workshop on Flexible Models for Longitudinal and Survival Data with Applications in Biostatistics July, 27th, 2015 A profile likelihood approach to
Speaker: Ziqi Chen Central South University
Workshop on Flexible Models for Longitudinal and Survival Data with Applications in Biostatistics
July, 27th, 2015
A profile likelihood approach to longitudinal data
Longitudinal data arise frequently in biomedical and health studies in which repeated measurements from the same subject are
regression parameters are important for longitudinal data analysis.
A profile likelihood approach to longitudinal data
Liang and Zeger (1986) developed the generalized estimating equation (GEE) for correlated data. GEE estimators are efficient when the working correlation structure is correctly specified, which is not an easy task however. Misspecification
efficiency (Wang and Carey 2003), although consistency remains holds. Qu, Lindsay, and Li (2000) proposed the quadratic inference function (QIF) to improve efficiency of the GEE by combining strength of several correlation structures.
A profile likelihood approach to longitudinal data
Ye and Pan (2006) proposed the simultaneous GEE equations to estimate both the mean regression coefficients and covariance structure parameters. Leung, Wang and Zhu (2009) proposed a hybrid method that combines multiple GEEs based on different working correlation models, using the empirical likelihood method (Qin and Lawless, 1994). In order to attain efficient estimators (i.e., their asymptotic variances achieve the efficiency bounds), the aforementioned papers all need correct specification of the correlation structure for longitudinal data.
A profile likelihood approach to longitudinal data
If we know the full likelihood function of the longitudinal data, we could achieve the maximum likelihood estimators of the regression parameters, which is efficient. Motivated by the modified Cholesky decomposition, Pourahmadi (1999,2000) proposed to estimate the mean regression coefficients efficiently by simultaneously estimating the covariance matrices based on the the multivariate normal distribution. However, it is difficult to specify the full likelihood function when responses are non-normal because of the correlated nature of longitudinal data.
A profile likelihood approach to longitudinal data
By regressing the error on its predecessors (Pourahmadi, 1999), we treat the prediction error density as an unknown nonparametric function and propose to estimate it by kernel
achieve the estimators of the regression parameters by maximizing the so-called profile likelihood. Our proposed method performs well without specification of the likelihood as well as the correlation structure.
A profile likelihood approach to longitudinal data
Suppose the response variable for the i−th subject is measured m times, yi = (yi1, · · · , yim)T, where yi’s are independently distributed, i = 1, · · · , n, where n is the sample size. The covariate corresponding to yij is xij, which is a p−dimensional vector. Denote Xi = (xi1, · · · , xim)T , which is m × p−dimensional matrix for the i−th subject. Let µij = E(yij|Xi) = xT
ij β, where β is a p−dimensional
parameter vector with true value being β∗. Denote ǫij = yij − xT
ij β∗ and ǫi = (ǫi1, · · · , ǫim)T . We assume
that the error vector ǫi is independent of the covariate matrix Xi, and ǫi (i = 1, · · · , n) are independently and identically distributed.
A profile likelihood approach to longitudinal data
The independence estimating procedure
We first naively assume that the responses of the i−th subject are independent of each other, for i = 1, · · · , n. The maximum likelihood estimator of β could be got if the density functions of ǫ1j (j = 1, · · · , m) are known. However, these density functions are typically unknown in practice. Let ǫij(β) = yij − xT
ij β. We propose to estimate the density
function of ǫ1j(β) for given β by kernel smoothing as ˆ fǫ1j(β)(u) = 1 nh
n
K(ǫij(β) − u h ), for j = 1, · · · , m, where K is a scalar kernel and h is an appropriate bandwidth.
A profile likelihood approach to longitudinal data
The independence estimating procedure
We propose to obtain the estimator of β by maximizing the profile likelihood equation (Chen, et al., 2014): ˆ βI
MPL = arg max β m
n
log ˆ fǫ1j(β)(ǫij(β)). (1)
A profile likelihood approach to longitudinal data
The independence estimating procedure
Since xij is independent of ǫij, we know
m
<
m
(2) for any β = β∗.
A profile likelihood approach to longitudinal data
The independence estimating procedure
We have that by simple calculations
m
1 n
n
log ˆ fǫ1j(β)(ǫij(β)) →P
m
uniformly in β, by Theorem 2.1 of Newey and McFadden (1994), we obtain ˆ βI
MPL →P β∗.
A profile likelihood approach to longitudinal data
The independence estimating procedure
Even though the consistency of ˆ βI
MPL holds, it may not be
efficient estimator, since ˆ βI
MPL is got based on the
independence assumption, that is, the within-subject correlation has not been taken into account. This is consistent with the literature in GEE that one could not get the fully efficient estimator when independence working correlation matrix is used (Liang and Zeger, 1986; Diggle, 2002). The asymptotical normality theory in the next subsection and the simulation studies further verify this point.
A profile likelihood approach to longitudinal data
The efficient estimating approach
Let Cov(ǫi) = Σ. By the modified Cholesky decomposition (Pourahmadi, 1999), there exist a lower triangular matrix P with ones as diagonal entries and −φjl as the (j, l)th element and a diagonal matrix D = diag(σ2
i1, · · · , σ2 im) such that
PΣPT = D. Based on this decomposition, one regresses ǫij on its predecessors ǫi1, · · · , ǫi(j−1) with corresponding regression coefficients being φj1, · · · , φj(j−1) and denotes ηij to be the corresponding prediction error, that is, ηij = ǫij −
j−1
φjlǫil, for j = 2, · · · , m; i = 1, · · · , n. Let ηi1 = yi1 − µi1, for i = 1, · · · , n.
A profile likelihood approach to longitudinal data
The efficient estimating approach
We have that ηi1, · · · , ηim are uncorrelated random variables since their covariance matrix D is a diagonal matrix. The specification of φjl’s determines the correlation structure
correlation structure when φjl’s are all zero and owns the AR-1 correlation structure if φjl is zero for j − l ≥ 2.
A profile likelihood approach to longitudinal data
The efficient estimating approach
Define ηij(β, φj) = yij − x
′
ijβ − j−1 l=1(yil − x
′
ilβ)φjl, let 0 l=1
equal zero throughout this paper and let φj = (φj1, · · · , φj(j−1)). When the distribution of ǫi is not available, instead of using the maximum likelihood method, we propose η1j follows completely nonparametric distribution for j = 1, · · · , m and use the kernel nonparametric technique to estimate the density function of η1j.
A profile likelihood approach to longitudinal data
The efficient estimating approach
We assume φj (j = 2, · · · , m) are known and propose to estimate the density function of η1j(β, φj) for given β by kernel smoothing as ˆ fη1j(β,φj)(u) = 1 nh
n
K(ηij(β, φj) − u h ), for j = 1, · · · , m, (4) where K is a scalar kernel and h is an appropriate bandwidth.
A profile likelihood approach to longitudinal data
The efficient estimating approach
With the estimated prediction error densities, we propose to get the estimate of β through the maximum profile likelihood (MPL; Chen et al., 2014), that is, ˆ βMPL is got through maximizing the following profile likelihood equation:
m
n
log ˆ fη1j(β,φj)(ηij(β, φj)). (5)
A profile likelihood approach to longitudinal data
The efficient estimating approach
We conclude that the choice of φjl thus the correlation structure used has no impact on the consistency property of the estimated β. This coincides with that in the GEE estimating literature. Specifically, for any φj (j = 2, · · · , m), recall ǫi is independent
m
<
m
A profile likelihood approach to longitudinal data
The efficient estimating approach
since
m
1 n
n
log ˆ fη1j(β)(ηij(β, φj)) →P
m
holding uniformly in β, we have ˆ βMPL is a consistent estimator (Newey and McFadden, 1994).
A profile likelihood approach to longitudinal data
The efficient estimating approach
Remark 1. The proposed profile likelihood estimator in this section is motivated by Cholesky decomposition of the covariance matrix Σ. However, the consistency property of the proposed estimator does not rely on existence of the covariance matrix
Specifically, we regress the error on its predecessors motivated by the Cholesky decomposition and estimate the prediction error density. The proposed approach is based on maximizing the estimated likelihood and does not directly include the covariance of error ǫi or operate on its empirical counterpart.
A profile likelihood approach to longitudinal data
The efficient estimating approach
Remark 1. Note that, the GEE requires the covariance matrix of error directly. As is shown in the simulation study, when the covariance matrix diverges, our proposed method performs very well, however the GEE breaks down.
A profile likelihood approach to longitudinal data
The efficient estimating approach
Remark 2. Our proposed method is robust against outliers. Intuitively, by the property of the commonly used kernel functions (e.g., gaussian, epanechnikov kernels), an outlier being distant from
value, thus has little impact on the estimation of β. The robust performance is illustrated further via the simulation study.
A profile likelihood approach to longitudinal data
The efficient estimating approach
Let Φ = (φ2, · · · , φm), Wi(Φ) = f
′
η11(β∗)(ηi1(β∗))
fη11(β∗)(ηi1(β∗)), f
′
η12(β∗,φ2)(ηi2(β∗, φ2))
fη12(β∗,φ2)(ηi2(β∗, φ2)), · · · , f
′
η1m(β∗,φm)(ηim(β∗, φm))
fη1m(β∗,φm)(ηim(β∗, φm)) T ,
A profile likelihood approach to longitudinal data
The efficient estimating approach
Qi(Φ) = diag
f
′
η11(β∗)(ηi1(β∗))
fη11(β∗)(ηi1(β∗))}2, { f
′
η12(β∗,φ2)(ηi2(β∗, φ2))
fη12(β∗,φ2)(ηi2(β∗, φ2))}2, · · · , { f
′
η1m(β∗,φm)(ηim(β∗, φm))
fη1m(β∗,φm)(ηim(β∗, φm))}2 , P be a lower unitriangular matrix with the (j, l)−th below diagonal entry being −φjl.
A profile likelihood approach to longitudinal data
The efficient estimating approach
Theorem Under mild conditions, for any Φ given, √n(ˆ βMPL − β∗) = −Ω−1 1 √n
n
(Xi − EXi)TPTWi(Φ) + op(1), where Ω = E{(X1 − EX1)TPTQ1(Φ)P(X1 − EX1)}.
A profile likelihood approach to longitudinal data
The efficient estimating approach
We show the efficiency property of ˆ βMPL under assumption that the error is normally distributed. Assume EX = 0 without loss of generality. When the true value of Φ (i.e., Φ∗) is used when maximizing (5), since Wi(Φ∗) =
σ∗2
11
, −ηi2(β∗, φ∗
2)
σ∗2
12
, · · · , −ηim(β∗, φ∗
m)
σ∗2
1m
1j being the variance of ηij(β∗, φ∗ j ), we have that
through simple calculations √n(ˆ βMPL − β∗) →L N
1 Σ∗−1X1)}−1
, where Σ∗ is the true value of Σ. It is easily seen that ˆ βMPL at this time is asymptotically as efficient as the maximum likelihood estimator based on the true distribution of the error.
A profile likelihood approach to longitudinal data
The efficient estimating approach
We conclude the choice of Φ does not affect the asymptotical consistency as well as normality of the MPL estimator, however determines efficiency of the resulted MPL estimator. This is consistent with properties in GEE estimating literature. Further, as long as ηi1(β∗), ηi2(β∗, φ∗
2), · · · , ηim(β∗, φ∗ m) are
independent of each other, we achieve fully efficient estimator ˆ βMPL when Φ is set to Φ∗ regardless of the error distribution.
A profile likelihood approach to longitudinal data
The efficient estimating approach
One typically does not know the true value of Φ. We finally discuss the estimation of Φ. Let ˆ µij = x
′
ij ˆ
βI
MPL and let
ˆ ηij(φj) = yij − ˆ µij − j−1
l=1(yil − ˆ
µil)φjl (j = 2, · · · , m). We propose to obtain the estimator of φj (i.e., ˆ φj) by maximizing
n
log ˆ fˆ
η1j(φj)(ˆ
ηij(φj)) for j = 2, · · · , m, where ˆ fˆ
η1j(φj)(u) = 1
nh
n
K( ˆ ηij(φj) − u h ). (6)
A profile likelihood approach to longitudinal data
The efficient estimating approach
ˆ φj ( j = 2, · · · , m) is a consistent estimator. When φj in (5) is replaced by ˆ φj, the resulted MPL estimator of β, i.e., ˆ β∗
MPL,
has the asymptotical normality property in Theorem 1 by letting Φ = Φ∗: √n(ˆ β∗
MPL − β∗) →L N(0, Ω∗−1),
(7) where Ω∗ = E{(X1 − EX1)T P∗TQ1(Φ∗)P∗(X1 − EX1)} with P∗ being the true value of P. Equation (7) suggests that the covariance matrix of ˆ β∗
MPL is
approximately (nΩ∗)−1.
A profile likelihood approach to longitudinal data
The efficient estimating approach
We estimate EQ1(Φ∗) as ˆ EQ1(Φ∗) := diag 1 n
n
{ ˆ f
′
η11(ˆ β∗
MPL)(ηi1(ˆ
β∗
MPL))
ˆ fη11(ˆ
β∗
MPL)(ηi1(ˆ
β∗
MPL))
}2, · · · , 1 n
n
{ ˆ f
′
η1m(ˆ β∗
MPL,ˆ
φm)(ηim(ˆ
β∗
MPL, ˆ
φm)) ˆ fη1m(ˆ
β∗
MPL,ˆ
φm)(ηim(ˆ
β∗
MPL, ˆ
φm)) }2 , where ˆ f
′
η1j(β,φj)(u) = − 1
nh2
n
K
′(ηij(β, φj) − u
h ), for j = 1, · · · , m.
A profile likelihood approach to longitudinal data
The efficient estimating approach
we estimate Ω∗ by ˆ Ω := 1 n
n
(Xi − ¯ X)T ˆ PT{ˆ EQ1(Φ∗)}ˆ P(Xi − ¯ X), where ¯ X = 1
n
n
i=1 Xi and ˆ
P is the lower triangular matrix with 1′s as diagonal entries and −ˆ φjl as the (j, l)th element. Thus, the covariance matrix of ˆ β∗
MPL could be estimated by
(nˆ Ω)−1. We can show that ˆ Ω−1 converges in probability to Ω∗−1 under the Frobenius norm.
A profile likelihood approach to longitudinal data
The efficient estimating approach
For j = 1, · · · , m, we use the same bandwidth h but this is
different bandwidths for different prediction error densities. Specifically, for j = 1, · · · , m, the maximum-profile-likelihood cross-validation bandwidth for estimating fη1j(β,φj)(·) is given by ˆ hj = arg max
h>0 n
log{ 1 (n − 1)h
K(ηkj(ˆ βI
MPL, φj) − ηij(ˆ
βI
MPL, φj)
h )}
A profile likelihood approach to longitudinal data
In this section we investigate the finite sample performances
yij = x
′
ijβ∗ + ǫij,
for i = 1, · · · , n; j = 1, · · · , 5, where β∗ is a p−dimensional vector of parameters. We generate 200 data sets respectively for all the studies. We use the simulated average mean square error (SAMSE), which is got by averaging ||ˆ β − β∗||2/p over 200 simulated samples, to measure the accuracy of estimators (Wang, 2011). We compare our proposed MPL estimator ˆ β∗
MPL with GEE
estimators using three different working correlation structures: independence, exchangeable and the AR-1 working correlation matrices (Zhou and Qu, 2012; Chen et al., 2012).
A profile likelihood approach to longitudinal data
In order to explain that MPL accounting for within-subject correlation using our proposed estimating procedure via regressing the error on its predecessors could improve accuracy
parameters (i.e., ˆ βI
MPL) using the independence MPL
estimating method described in Section 2.1. Throughout the numerical demonstration, the Gaussian kernel function K(u) =
1 √ 2π exp(−u2/2) is used.
A profile likelihood approach to longitudinal data
Study 1. The sample size n is taken as 100 and the dimension of the covariates is set to be p = 3. The dimension of the parameter is low and fixed in this study. The covariate xij is set to be (xij1, xij2, xij3)
′, which follows the multivariate normal distribution
with mean zero and with covariance matrix Σ(1) with (Σ)(1)
i,j = 0.5|i−j| for 1 ≤ i, j ≤ 3. Let β∗ = (1, 0.5, −0.5)
′. The
errors are set as follows: ǫi1 = ηi1, ǫi2 = ηi2 + 0.5ǫi1, ǫi3 = ηi3 + 0.4ǫi1 + 0.4ǫi2, ǫi4 = ηi4 + 0.3ǫi1 + 0.3ǫi2 + 0.3ǫi1, ǫi5 = ηi5 + 0.1ǫi1 + 0.1ǫi2 + 0.1ǫi1 + 0.1ǫi5, where ηi1, ηi2 and ηi3 follow the standard normal distribution, ηi4 and ηi5 follow the T−distribution with 3 degrees of freedom, ηij (j = 1, · · · , 5) are all independent of each other.
A profile likelihood approach to longitudinal data
Study 2. We consider the sample size n = 50, 100, 200, 500 and the dimension of the parameter pn = [2.5n1/3], where [q] is the largest integer not greater than q. In this study, β∗ = (0.41
′
k, −0.31
′
k, 0.21
′
k, −0.11
′
pn−3k)
′ , where 1k denotes a
k−dimensional vector of ones and k = [pn/4]. We take the covariate xij = (xij1, · · · , xijpn)
′, which follows the multivariate
normal distribution with mean zero and with covariance matrix Σ(2) with (Σ)(2)
i,j = 0.5|i−j| for 1 ≤ i, j ≤ pn. The error used in this
study is that of Study 1. This study is to investigate how the proposed approach works when the dimension of the parameter pn grows with the sample size n.
A profile likelihood approach to longitudinal data
Study 3. The sample size in this study is set to be n = 100. We consider two cases of dimensions of the parameter, i.e., p = 3 and p = 11. The covariates for the case of p = 3 and p = 11 are generated via the methods of Study 1 and Study 2, respectively. The error ǫi = (ǫi1, · · · , ǫi5)
′ is generated from the multivariate
normal distribution with mean zero and covariance matrix Σ(3), which is an AR-1 correlation matrix with autocorrelation coefficient 0.8.
A profile likelihood approach to longitudinal data
Study 4. The error ǫi = (ǫi1, · · · , ǫi5)
′ follows the multivariate
normal distribution with mean zero and covariance matrix Σ(4), which is an exchangeable correlation matrix with all pairs of
generated following the procedure in Study 3. The main purpose
new method to the correlation structures.
A profile likelihood approach to longitudinal data
Study 5. The error ǫi = (ǫi1, · · · , ǫi5)
′ is generated from the
multivariate T distribution with location parameter zero, scale matrix Σ(4) and degrees of freedom 3. We generate the datasets following the procedure in Study 3. This study is to investigate the performance of the proposed approach when the error is heavy-tailed.
A profile likelihood approach to longitudinal data
Study 6. The error ǫi = (ǫi1, · · · , ǫi5)
′ is generated from the
multivariate Cauchy distribution with location parameter zero and scale matrix Σ(4) and we generate the datasets following the procedure in Study 3. This study is designed to investigate how the proposed approach performs when covariance matrix of the error is infinite.
A profile likelihood approach to longitudinal data
Study 7. The error ǫi follows the mixture of multivariate normal distributions, i.e., 0.9N(0, Σ(4)) + 0.1N(0, 100Σ(4)) and the datasets are sampled using the procedure in Study 3. The main purpose of this study is to investigate robustness of the proposed method in terms of resistance to outliers. Study 3-7 could also play the role in investigating how the proposed method works when the true covariance matrix does not admit an explicit Cholesky decomposition for different error distributions.
A profile likelihood approach to longitudinal data
Table 1
A profile likelihood approach to longitudinal data
First, we see clearly that the proposed MPL method has smaller SAMSEs than the GEE for all the errors, sometimes by a large margin, except for the GEE adopting correct correlation structures under the multivariate normal errors (Study 3 and 4).
A profile likelihood approach to longitudinal data
Second, the multivariate normal errors in Study 3 and Study 4 favor the GEE using AR-1 and exchangeable correlation structures, respectively, thus our proposed method produces sightly larger SAMSEs than the GEE method using correct correlation structures. However for both studies, our proposed method outperforms the GEE choosing incorrect correlation
well without specification of the correlation matrix, thus is robust regardless of the correlation structure.
A profile likelihood approach to longitudinal data
Third, consistent with the general knowledge in longitudinal data analysis, our proposed method has taken into account the within-subject correlation and thus has smaller SAMSEs than the MPL with independence estimating procedure. Fourth, even if the generating covariance matrix does not admit a clear Cholesky decomposition, our proposed method continues to perform very well as seen from the results of Study 3-7.
A profile likelihood approach to longitudinal data
Fifth, study 2 shows that our proposed approach performs satisfactorily when the dimension of the parameter pn grows with the sample size n. Sixth, the proposed method is resistant to heavy-tailed errors as is shown by Study 5.
A profile likelihood approach to longitudinal data
Seventh, when the covariance matrix of the error diverges (e.g., Cauchy distribution), GEE breaks down, however, our method works well as is indicated in Study 6. Finally, the proposed method is robust against outliers as is seen in Study 7, and the SAMSE of the new method remains no more than 0.02 regardless of the error distribution. These demonstrate its robustly good performance in estimating parameters. Extensive studies show that our proposed method on longitudinal data outperforms GEE for the case that the covariance matrix of the responses for each subject is covariate-dependent, although it is assumed that the covariance matrix is static when introducing our method.
A profile likelihood approach to longitudinal data
Table 1
A profile likelihood approach to longitudinal data
“SD” represents the sample standard deviation over 200 estimates and is regarded as the true standard error. “SE” represents the sample average of 200 standard errors using the covariance estimating method described in the end of Section 2.2. Table 2 compares SD with SE for all models used above for the case of n = 100 and p = 3. We observe that the covariance estimating method works remarkably well. Similar phenomena are also observed for
A profile likelihood approach to longitudinal data
We proposed a novel profile likelihood based method for longitudinal data analysis. The proposed method takes the within-subject correlation into account and works well without specification of the likelihood as well as the correlation structure. Our theoretical and numerical results show that our proposed methods produce consistent and efficient estimators.
A profile likelihood approach to longitudinal data