A comparison of Raos score test and likelihood ratio tests for three - - PowerPoint PPT Presentation

A comparison of Rao’s score test and likelihood ratio tests for three separable covariance matrix structures

Katarzyna Filipiak1, Daniel Klein2, Anuradha Roy3

1Institute of Mathematics, Pozna´

n University of Technology, Poland

2Institute of Mathematics, P. J. ˇ

Saf´ arik University, Koˇ sice, Slovakia

3Department of Management Science and Statistics,

University of Texas at San Antonio, USA

XLII Conference on Mathematical Statistics

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 1 / 20

Data

q – number of characteristics p – number of time points n – number of individuals Xi, i = 1,...,n – i.i.d. observation matrices

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 2 / 20

Observation matrices

X1 =     x1,1,1 x1,1,2 ... x1,1,p x1,2,1 x1,2,2 ... x1,2,p . . . . . . ... . . . x1,q,1 x1,q,2 ... x1,q,p     . . . Xn =     xn,1,1 xn,1,2 ... xn,1,p xn,2,1 xn,2,2 ... xn,2,p . . . . . . ... . . . xn,q,1 xn,q,2 ... xn,q,p    

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 3 / 20

Two-level multivariate model

Model

Xi ∼ Nq,p(M,Ω) Xi - matrix of observations M - matrix of means Ω - variance-covariance matrix (p.d.)

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 4 / 20

Two-level multivariate model

Model

vecXi ∼ Npq(vecM,Ω) vecXi ∼ Npq(µ,Ω) Vector of unknown parameters: θ =

• θ1

θ2

• =
• µ

vechΩ

• pq

pq(pq+1)/2 Estimability of Ω: n > pq

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 5 / 20

Covariance structure

Ω

pq×pq = Ψ p×p⊗ Σ q×q

Ψ – variance-covariance matrix of the repeated measurements on a given characteristic (the same for all characteristics) Σ – variance-covariance matrix of the q response variables at any given time point (the same for all time points) Number of parameters:

p(p+1) 2

+ q(q+1)

2

−1

Roy & Khattree (2003), Lu & Zimmerman (2005), Roy (2007)

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 6 / 20

Covariance structure

Ω

pq×pq = Ψ p×p⊗ Σ q×q

Ψ – AR(1) structure Ψ =        1 ρ ρ2 ... ρp−1 ρ 1 ρ ... ρp−2 ρ2 ρ 1 ... ρp−3 . . . . . . . . . ... . . . ρp−1 ρp−2 ρp−3 ... 1        Number of parameters: 1+ q(q+1)

2

Roy and Khatree, 2005, Roy and Leiva, 2008

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 7 / 20

Covariance structure

Ω

pq×pq = Ψ p×p⊗ Σ q×q

Ψ – CS structure Ψ =       1 ρ ρ ... ρ ρ 1 ρ ... ρ ρ ρ 1 ... ρ . . . . . . . . . ... . . . ρ ρ ρ ... 1       Number of parameters: 1+ q(q+1)

2

Roy and Khatree, 2005, Roy and Leiva, 2008, Filipiak et al., 2016

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 8 / 20

Hypothesis

Hypothesis

H0 : Ω = Ψ⊗Σ, Ψ UN HA : Ω UN Degrees of freedom: ν = pq(pq+1)

2

− q(q+1)

2

− p(p+1)

2

−1

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 9 / 20

Hypothesis

Hypothesis

H0 : Ω = Ψ⊗Σ, Ψ AR(1) or CS HA : Ω UN Degrees of freedom: ν = pq(pq+1)

2

− q(q+1)

2

−1

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 10 / 20

Likelihood ratio test

X = [vecX1,vecX2,...,vecXn] ∈ Rpq,n – data matrix lnL(µ,Ω;X) – log-likelihood function (partially differentiable with respect to each coordinate of θ for every X).

Likelihood ratio (LR)

Λ = maxH0 L maxHA L

Likelihood ratio test statistics

−2lnΛ ∼

app

χ2

ν

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 11 / 20

Rao’s score test

Rao’s score (RS)

s′( θ)F −1( θ)s( θ)

• θ – MLE of θ

s(θ) =

• s′

1(θ),s′ 2(θ)

′ =

• ∂ lnL

∂vec′µ, ∂ lnL ∂vech′Ω

′ – score vector F(θ) = −E

• ∂s(θ)

∂θ ′

• – Fisher information matrix (invertible)

Rao’s score test statistics

s′( θ)F −1( θ)s( θ) ∼

app

χ2

ν

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 12 / 20

Hypothesis

H0 : Ω = Ψ⊗Σ, Ψ UN, AR(1) or CS HA : Ω UN

RS test statistics

RS = n 2tr

• 1

nXQ1nX′(

Ψ−1 ⊗ Σ−1)−Ipq 2 Q1n = In − 1

n1n1′ n

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 13 / 20

Remark

RST statistic depends only on the data matrix X, an estimate of Ψ and Σ. Thus, the minimum number of samples needed to calculate RST statistic is: max{p,q}+1 if Ψ is UN; q+1 if Ψ is AR(1) or CS, whereas the minimum number of samples needed to calculate LRT statis- tic is pq+1, since it depends on the MLE of a pq×pq variance-covariance matrix Ω.

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 14 / 20

Theorem

Theorem

The distributions of both LRT and RST statistics under the null hypothesis

• f separability of Ψ⊗Σ with first component as UN, AR(1) or CS, do not

depend on the true values of µ or Σ. Moreover, if Ψ is UN or CS, the distributions of both test statistics do not depend on the true value of Ψ.

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 15 / 20

Simulation study - empirical Type I error

H0 p q n

• αLRT
• αRST

n

• αLRT
• αRST

UN ⊗ UN 5 5 6 — 0.536 30 1.000 0.028 AR(1) ⊗ UN 12 2 — 0.361 0.999 0.030 CS ⊗ UN 12 2 — 0.363 0.999 0.030 UN ⊗ UN 5 5 10 — 0.150 50 0.794 0.019 AR(1) ⊗ UN 12 2 — 0.131 0.691 0.020 CS ⊗ UN 12 2 — 0.132 0.691 0.020 UN ⊗ UN 5 5 15 — 0.067 100 0.187 0.014 AR(1) ⊗ UN 12 2 — 0.069 0.151 0.014 CS ⊗ UN 12 2 — 0.069 0.150 0.014 UN ⊗ UN 5 5 20 — 0.045 150 0.083 0.013 AR(1) ⊗ UN 12 2 — 0.046 0.069 0.014 CS ⊗ UN 12 2 — 0.046 0.069 0.014 UN ⊗ UN 5 5 25 — 0.033 200 0.052 0.012 AR(1) ⊗ UN 12 2 1.000 0.034 0.044 0.013 CS ⊗ UN 12 2 1.000 0.034 0.045 0.013

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 16 / 20

Simulation study - Empirical null distribution

n p q H0 LRT 95th RST 95th 4 5 2 UN ⊗ UN — — 3 3 AR(1) ⊗ UN — (62.223) 3 3 CS ⊗ UN — (62.262) 10 5 2 UN ⊗ UN — 55.044 3 3 AR(1) ⊗ UN (145.064) (56.266) 3 3 CS ⊗ UN (145.131) (56.237) 25 5 2 UN ⊗ UN 69.799 54.298 3 3 AR(1) ⊗ UN (66.399) (54.563) 3 3 CS ⊗ UN (66.379) (54.496) 50 5 2 UN ⊗ UN 60.457 53.907 3 3 AR(1) ⊗ UN (59.065) (53.734) 3 3 CS ⊗ UN (59.034) (53.727) 100 5 2 UN ⊗ UN 56.523 53.603 3 3 AR(1) ⊗ UN (55.923) (53.558) 3 3 CS ⊗ UN (55.892) (53.527) 200 5 2 UN ⊗ UN 54.861 53.474 3 3 AR(1) ⊗ UN (54.533) (53.348) 3 3 CS ⊗ UN (54.536) (53.373) ∞ 5 (3) 2 (3) (Sep. struc) 53.384 53.384

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 17 / 20

Monkey data set (McKiernan et al, 2009)

n = 12 subjects (rhesus monkeys) q = 3 characteristics (body weight, % body fat, % maximum mass of upper leg) p = 3 time points

6th year 9th year 12th year Weight % BF % MM Weight % BF % MM Weight % BF % MM 1 15.97 28.31 92.9 15.80 30.58 64.3 14.93 32.64 54.6 2 12.25 16.52 94.6 13.98 25.12 93.2 13.96 26.74 82.6 3 18.52 31.55 96.9 19.09 33.54 88.6 19.09 34.64 85.4 4 11.85 27.25 93.6 14.69 39.39 85.8 14.79 44.14 71.5 5 14.48 19.36 99.0 15.06 23.42 88.7 14.56 24.62 90.3 6 14.70 26.38 94.9 13.90 26.93 88.4 13.06 27.11 78.2 7 13.26 22.54 96.3 14.00 30.82 81.8 13.46 30.54 74.6 8 13.67 21.42 88.4 16.70 26.62 82.0 15.47 27.32 77.8 9 15.33 24.85 98.9 14.68 30.58 69.1 11.58 24.11 60.5 10 10.96 16.79 95.3 10.69 17.55 90.7 8.32 9.46 82.0 11 12.92 22.54 94.9 12.02 20.43 87.9 11.61 24.15 84.5 12 11.10 15.65 94.6 14.58 29.52 83.5 13.30 30.93 77.1

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 18 / 20

Monkey data set (McKiernan et al, 2009)

LRT RST UN ⊗ UN AR(1) ⊗ UN CS ⊗ UN UN ⊗ UN AR(1) ⊗ UN CS ⊗ UN 1,2,3 Value 86.706 106.979 118.609 49.894 49.862 61.388 p−val : χ2

ν

0.039 0.094 0.0095 p−val : END (0.05,0.1) (0.01,0.05) < 0.01 (0.05,0.1) (0.1,0.15) (0.01,0.05) 1,2 Value 11.493 21.753 30.251 11.295 17.532 26.254 p−val : χ2

ν

0.430 0.194 0.025 0.414 0.419 0.070 p−val : END (0.85,0.9) (0.6,0.65) (0.2,0.25) 0.75 (0.5,0.55) (0.05,0.1) 2,3 Value 19.241 45.215 54.675 16.458 30.364 40.096 p−val : χ2

ν

0.116 0.225 0.024 0.0012 p−val : END (0.45,0.50) (0.01,0.05) (< 0.01) (0.30,0.35) (0.01,0.05) < 0.01 1,3 Value 28.619 47.578 57.264 25.842 29.381 36.715 p−val : χ2

ν

0.007 0.018 0.031 0.004 p−val : END (0.1,0.15) (0.01,0.05) < 0.01 (0.01,0.05) (0.01,0.05) < 0.01

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 19 / 20

Main References

Filipiak, K., Klein, D., and Roy, A. (2016). Score test for a separable covariance structure with the first component as compound symmetric correlation matrix. J. Multivariate Anal. 150, 105–124. Filipiak, K., Klein, D., and Roy, A. (2016). A comparison of likelihood ratio tests and Rao’s score test for three separable covariance matrix structures. Biometrical Journal. DOI: 10.1002/bimj.201600044. Lu, N. and Zimmerman, D. (2005). The likelihood ratio test for a separable covariance matrix. Statist. Probab.

• Lett. 73, 449–457.

McKiernan, S.H., Colman, R.J., Lopez, M., Beasley, T.M., Weindruch, R., and Aiken, J.M. (2009). Longitudinal analysis of early stage Sarcopenia in aging rhesus monkeys. Experimental Gerontology 44, 170–176. Roy, A. (2007). A note on testing of Kronecker product covariance structures for doubly multivariate data.

• Proc. Amer. Statist. Assoc., Statistical Computing Section, 2157–2162.

Roy, A. and Khattree, R. (2003). Tests for mean and covariance structures relevant in repeated measures based discriminant analysis. J. Appl. Statist. Sci. 12(2), 91–104. Roy, A. and Khattree, R. (2005). Testing the hypothesis of a Kroneckar product covariance matrix in multivariate repeated measures data. Proc. of SUGI 30, Philadelphia. Roy, A. and Leiva, R. (2008). Likelihood ratio tests for triply multivariate data with structured correlation on spatial repeated measurements. Statist. Probab. Lett. 78, 1971–1980.

• K. Filipiak (Pozna´

n, Poland) Rao’s score test 01.12.2016 20 / 20