Estimation equations for multivariate linear models with Kronecker - - PowerPoint PPT Presentation

Estimation equations for multivariate linear models with Kronecker structured covariance matrices

Szczepa´ nska-´ Alvareza, Chengcheng Haob, Yuli Liangc, Dietrich von Rosend,e

a Department of Mathematical and Statistical Methods, Pozna´

n University of Life Sciences, Poland,

b School of Business Information, Shanghai University of International Business

and Economics, China,

c Statistics Sweden, Sweden, d Department of Energy and Technology, Swedish University of Agricultural

Sciences, Sweden,

e Department of Mathemathics, Link¨

• ping University, Sweden

2.12.2016

Model

Consider independent and identically matrix normally distributed

• bservations

Xi ∼ Np,q(µ, Ψ, Σ), i = 1, ..., n, vecXi ∼ Npq(vecµ, Σ ⊗ Ψ), where E[Xi] = µ - the expected value, D[Xi] = Σ ⊗ Ψ - the dispersion matrix, Ψ - the p × p matrix describing the unknown covariance structure between the rows of Xi, Σ - the q × q matrix describing the unknown covariance structure between the columns of Xi

Data Y

Let Y(i) = Xi − 1 n

n

• i=1

Xi, Moreover, Y = (Y(1), Y(2), ..., Y(n)), Y(i) = Y1i Y2i

• ,

Y : p × nq, Y(i) : p × q, Y1i : r × q, Y2i : (p − r) × q, i = 1, 2, ..., n, so Y = Y11 Y12 ... Y1n Y21 Y22 ... Y2n

• =

Y1 Y2

• ,

where Y1 : r × nq, Y2 : (p − r) × nq.

Data ˜ Y′

Let ˜ Y′ = ( ˜ Y′

(1), ˜

Y′

(2), ..., ˜

Y′

(n)),

˜ Y′

(i) =

˜ Y′1i ˜ Y′2i

• ˜

Y′ = Y′

11

Y′

12

... Y′

1n

Y′

21

Y′

22

... Y′

2n

• =

˜ Y′1 ˜ Y′2

• where

˜ Y′ : q × np, ˜ Y′1i : q × r, ˜ Y′2i : q × (p − r), i = 1, 2, ..., n.

Case 1

The matrix Σ is unstructured and Ψ is a partitioned matrix of the form Ψ = A(θ) B B′ Ω

• ,

where A(θ): r × r, 1 < r < p, depends on an unknown parameter θ, B: r × (p − r), Ω: (p − r) × (p − r) - unknown matrices.

Case 1 - Theorem 1

Given that the maximum likelihood estimator for θ in A(θ) can be

• btained the maximum likelihood estimators of µ, Σ and Ψ satisfy

the following equations:

• µ

= 1 n

n

• i=1

Xi, dA( θ)−1 d θ vec(qnA( θ) − Y1(In ⊗ Σ−1)Y′

1) = 0,

np Σ =

• Y

′ 1(In ⊗ A(

θ)−1) Y1 + ( Y

′ 2 −

Y

′ 1(In ⊗

δ))(In ⊗ Ψ

−1 2•1)

×( Y

′ 2 −

Y

′ 1(In ⊗

δ))′,

• δ

= (Y1(In ⊗ Σ

−1)Y′ 1)−1Y2(In ⊗

Σ

−1)Y′ 1,

Case 1 - Theorem 1

where

• Ψ

=

• A(

θ)

• B
• B

′

• Ω
• ,

and

• B

= A( θ) δ,

• Ω =
• Ψ2•1 +

δ

′A(

θ) δ, qn Ψ2•1 = (Y2 − δ

′Y1)(In ⊗

Σ

−1)(Y2 −

δ

′Y1)′.

Case 1 - Corollary 1

Under the assumptions of Theorem 1, if qnA( θ) − Y1(In ⊗ Σ

−1)Y′ 1 = 0,

then,

• µ

= 1 n

n

• i=1

Xi

• Ψ

= 1 nq Y(In ⊗ Σ

−1)Y′,

• Σ

= 1 np

• Y

′(In ⊗

Ψ

−1)

Y.

flip-flop algorithm

P.Dutilleul (1999)

• Since (cΣ) ⊗ ( 1

c Ψ), all the parameters of Σ and Ψ are not

defined uniquely.

• The direct product Σ ⊗ Ψ is uniquely defined.
• The convergence of the MLE algorithm may be assessed by try-

ing various initial solution. If all of the initial solutions tried result in the same direct product ˆ Σ⊗ ˆ Ψ and the corresponding final solu- tions are ˆ Σ, ˆ Ψ satisfy the criterion of the second derivatives, then any of the final solutions ˆ Σ, ˆ Ψ should provide maximum likelihood estimates for Σ and Ψ; otherwise, they correspond, at the least, to local extrema of the likelihood function.

Case 1 - Corollary 2

Under the assumptions of Theorem 1, if A( θ) = 1, then

• µ

= 1 n

n

• i=1

Xi

• Ψ

= 1 nq Y(In ⊗ Σ

−1)Y′,

• Σ

= 1 np

• Y

′(In ⊗

Ψ

−1)

Y.

Case 2

The matrix Σ is unstructured and Ψ is a block partitioned matrix, Ψ = A(θ) B B′ Ω

• ,

where A(θ) - a compound symmetric structure, i.e., A(θ) = (1 − θ)Ir + θ1r1′

r, where 1r denotes the column vector

• f size r with all elements equal to 1.

Case 2 - Theorem 2

The maximum likelihood estimators of µ, Σ and Ψ satisfy the following equations:

• θ

= 1 nqr(r − 1)tr(1r1′

rY1(In ⊗

Σ

−1)Y′ 1) −

1 r − 1,

• µ

= 1 n

n

• i=1

Xi, A( θ) = (1 − θ)Ir + θ1r1′

r,

np Σ =

• Y

′ 1(In ⊗ A(

θ)−1) Y1 + ( Y

′ 2 −

Y

′ 1(In ⊗

δ))(In ⊗ Ψ

−1 2•1)

×( Y

′ 2 −

Y

′ 1(In ⊗

δ))′,

• δ

= (Y1(In ⊗ Σ

−1)Y′ 1)−1Y2(In ⊗

Σ

−1)Y′ 1,

Case 2 - Theorem 2

where

• Ψ

=

• A(

θ)

• B
• B

′

• Ω
• ,

and

• B

= A( θ) δ,

• Ω =
• Ψ2•1 +

δ

′A(

θ) δ, qn Ψ2•1 = (Y2 − δ

′Y1)(In ⊗

Σ

−1)(Y2 −

δ

′Y1)′.

Case 3

Both matrices Σ and Ψ follow a compound symmetric covariance structure, i.e. Ψ = (1 − ρ)Ip + ρ1p1′

p,

Σ = σ1Iq + σ2(1q1′

q − Iq),

where ρ, σ1 and σ2 are unknown parameters.

Case 3 - Theorem 3

The maximum likelihood estimators of µ, Σ and Ψ satisfy

• µ

= 1 n

n

• i=1

Xi,

• Ψ

= (1 − ρ)Ip + ρ1p1′p,

• Σ

=

• σ1Iq +

σ2(1q1′q − Iq), where

• σ1

=

1 q(

λ1 + (q − 1) λ2),

• σ2 = 1

q(

λ1 − λ2),

• ρ =
• λ3/

λ4−1

• λ3/

λ4+p−1,

• λ1,

λ2 - distinct eigenvalues of Σ

• λ3,

λ4 - distinct eigenvalues of Ψ

Case 3 - Theorem 3

and ˆ λ1 =

1 np(ˆ

λ−1

3 t1 + ˆ

λ−1

4 t2),

ˆ λ2 =

1 np(q−1)(ˆ

λ−1

3 t3 + ˆ

λ−1

4 t4),

nqˆ λ3 − nqˆ λ2

3ˆ

λ−1

4

− ˆ λ−1

1 t1 − ˆ

λ−1

2 t3 +

ˆ λ−1

1 ˆ

λ−2

4 ˆ

λ2

3t2

(p − 1) + ˆ λ−1

2 ˆ

λ−2

4 ˆ

λ2

3t4

(p − 1) = 0, , p = ˆ λ3 + ˆ λ4(p − 1), with t1 = tr{P1pY(In ⊗ P1q)Y′}, t2 = tr{Q1pY(In ⊗ P1q)Y′}, t3 = tr{P1pY(In ⊗ Q1q)Y′}, t4 = tr{Q1pY(In ⊗ Q1q)Y′}, where P1p = 1

p1p1′ p and Q1p = Ip − P1p and the observation

matrix Y is the centered observation matrix.

Case 4

The matrix Σ is unstructured and in Ψ is the matrix which all diagonal elements equal 1. Ψ = T

− 1

2

d TT − 1

2

d ,

where T : p × p - the symmetric matrix, Td : p × p - the diagonal matrix with diagonal elements the same as matrix T.

Case 4 - Theorem 4

Maximum likelihood equations are given by the following relations: npΣ = ˜ Y′(In ⊗ (T

1 2

d T−1T

1 2

d )−1)˜

Y −2T−1T

1 2

d AT

1 2

d T−1 + (T−1T

1 2

d AT

1 2

d T−1)d + (T−1T

1 2

d A)dT − 1

2

d

= 0, where A = 2nqT

1 2

d T−1T

1 2

d − nqIp − Y(In ⊗ Σ−1)Y′

and (T−1T

1 2

d AT

1 2

d T−1)d,

(T−1T

1 2

d A)d

denote diagonal matrices.

Literature

Dutilleul P. (1999). The MLE algorithm for the matrix normal

• distribution. J. Statist. Comput. Simul, vol. 64, 105-123.

Srivastava M. S., von Rosen T. and von Rosen D.(2008). Models with a Kronecker Product Covariance Structure: Estimation and

• Testing. Mathematical Methods of Statistics, vol. 17, No. 4,

357–370. Szczepa´ nska-´ Alvarez A, Hao Ch., Liang Y., von Rosen D. (2016). Estimation equations for multivariate linear models with Kronec- ker structured covariance matrices. Communications in Statistics- Theory and Methods. DOI10.1080/03610926.2016.1165852