Properties of estimators of doubly exchangeable covariance matrix - - PowerPoint PPT Presentation

Properties of estimators of doubly exchangeable covariance matrix and structured mean vector for three-level multivariate observations

Arkadiusz Kozio l XLII Konferencja ”Statystyka Matematyczna” 27.11.2016 - 02.12.2016, B¸ edlewo 1

• 1. Three-level multivariate data - Introduction

Multi-level multivariate observations are becoming increasingly visible across all fields of biomedical, medical and engineering among many others these days. Example of three-level multivariate observations: Investigator measured of intraocular pressure (IOP) and central corneal thickness (CCT) which are

• btained from both the eyes (sites), each at three time points at an interval
• f three months for 30 patients. It is clear that for this data set m = 2, u = 2

and v = 3. 2

• 2. Doubly exchangeable covariance structure in three-level

multivariate data

The assumption of double exchangeability reduces the number of un- known parameters considerably, thus allows more dependable or reliable parameter estimates. The unstructured variance-covariance matrix has vum(vum + 1)/2 unknown parameters, which can be large for arbitrary values of m, v or u. In order to reduce the number of unknown parameters, it is then essential to assume some appropriate structure on the variance-covariance matrix. One may assume a DE covariance structure in this situation, where the data is multivariate in three levels. DE covariance structure has only 3m(m+1)/2 unknown param- eters. 3

2.1. Doubly exchangeable covariance structure in three-level multivariate data - form

The (vum × vum)−dimensional DE covariance structure is defined as Γ =      Σ0 Σ1 . . . Σ1 . . . ... . . . . . . ... . . . Σ1 Σ1 . . . Σ0      = Iv ⊗ (Σ0 − Σ1) + J v ⊗ Σ1, where Iv is the v × v identity matrix, 1v is a v × 1 vector of ones, J v = 1v1′

v.

4

2.1. Doubly exchangeable covariance structure in three-level multivariate data - form

In (1) matrices Σ0 and Σ1 have the following form: Σ0 = Iu ⊗ (Γ0 − Γ1) + J u ⊗ Γ1, and Σ1 = J u ⊗ Γ2. This covariance structure can equivalently be written in terms of Γ0, Γ1 and Γ2 as Γ = Iv ⊗ Iu ⊗ Γ0 + Iv ⊗ (J u − Iu) ⊗ Γ1 + (J v − Iv) ⊗ J u ⊗ Γ2. 5

2.2. Doubly exchangeable covariance structure in three-level multivariate data - assumptions

• 1. Γ0 is a positive definite symmetric m × m matrix,
• 2. Γ1 and Γ2 are a symmetric m × m matrices,
• 3. Γ0 − Γ1 is positive definite matrix,
• 4. Γ0 + (u − 1)Γ1 − uΓ2 is positive definite matrix,
• 5. Γ0 + (u − 1)Γ1 + (v − 1)uΓ2 is positive definite matrix,

so that the vum × vum matrix Γ is positive definite

• for a proof, see Lemma

2.1 in Roy and Leiva (2011)

• .

6

• 3. Model with structured mean vector - notation

Let yr,ts be a m-variate vector of measurements on the rth individual at the tth time point and at the sth site; r = 1, . . . , n, t = 1, . . . , v, s = 1, . . . , u. The n individuals are all independent. Let yr = (y′

r,11, . . . , y′ r,vu)′ be the vum-variate vector of all measurements

corresponding to the rth individual. Finally, let y1, y2, . . . , yn be a random sample of size n drawn from the popu- lation Nvum (1vu ⊗ µ, Γ), where µ ∈ Rm and Γ is assumed to be a vum×vum positive definite matrix. 7

3.1. Model with structured mean vector - assumption

In this model we assume that covariance structure is DE and mean vector has a following structure: 1nvu ⊗ µ where µ has m components. y

nvum×1 = vec( Y ′ vum×n) ∼ N

• (1nvu ⊗ Im)µ, In ⊗ Γvum
• .

This means that n independent random column vectors are identically dis- tributed (vum × vum)−dimensional variance covariance matrix. 8

3.2. Orthogonal projector on the subspace of mean vector

It will be noted by P and used to show that if In ⊗ Ivum ∈ ϑ = sp{V }, where V = In ⊗ Γ, it follows that P y is the best linear unbiased estimator (BLUE) if and only if P commutes with all covariance matrix V . Orthogonal projectors on the subspace of mean vector for model with struc- tured mean vector: P = 1 nJ n ⊗ 1 vJ v ⊗ 1 uJ u ⊗ Im, where J n = 1n1′

n, J v = 1v1′ v and J u = 1u1′ u are matrices of ones.

9

3.3. Orthogonal projector on the subspace of mean vector

Result 1. The projection matrix P commutes with the covariance matrix V , i.e., P V = V P , where V = In ⊗ Γ, the covariance matrix of y.

• Lemma. Let ϑ denote the subspace spanned by V , i.e., ϑ = sp{V }. Then,

ϑ is a quadratic subspace, meaning that ϑ is a linear space and if V ∈ ϑ then V 2 ∈ ϑ

• see Seely (1971) for the definition
• .

10

3.4. BLUE for µ

Because for considered model orthogonal projector on the space generated by the mean vector commutes with all covariances matrices, there exists BLUE for each estimable function of mean. Moreover BLUE are least squares es- timators (LSE), in view of Result 1. Thus, µ is the unique solution of the following normal equation: (1nvu ⊗ Im)′(1nvu ⊗ Im)µ = (1nvu ⊗ Im)′y or nvuImµ = [Im, Im, . . . , Im]y, which means that:

• µ =

1 nvu

n

• r=1

v

• t=1

u

• s=1

yr,ts. 11

3.5. Base for the quadratic subspace ϑ

We define Aii = Eii and Aij = Eij + Eji, for i < j; and j = 1, . . . , m, as a base for symmetric matrices Γ. The (m × m)−dimensional matrices Eij has 1 only at the ijth element, and 0 at all other elements. Then it is clear that the base for diagonal matrices of the form In⊗Iv⊗Iu⊗Γ0 is constituted by matrices K(0)

ij = In ⊗ Iv ⊗ Iu ⊗ Aij,

for i ≤ j, j = 1, . . . , m, the base for matrices of the form In ⊗ Iv ⊗ (J u − Iu) ⊗ Γ1 is constituted by matrices K(1)

ij = In ⊗ Iv ⊗ (J u − Iu) ⊗ Aij,

for i ≤ j, j = 1, . . . , m 12

3.5. Base for the quadratic subspace ϑ

and the base for matrices of the form In ⊗ (J v − Iv) ⊗ J u ⊗ Γ2 is constituted by matrices K(2)

ij = In ⊗ (J v − Iv) ⊗ J u ⊗ Aij,

for i ≤ j, j = 1, . . . , m. 13

3.6. The complete and minimal sufficient statistics for family of normal distributions

Let M = In⊗Iv⊗Iu⊗Im−P . So, M is idempotent. Now, since P V = V P , and ϑ is a quadratic space, MϑM = Mϑ is also a quadratic space. Result 2. The complete and minimal sufficient statistics for the mean vector and the variance-covariance matrix are: (1′

nvu ⊗ Im)y

y′MK(l)

ij My, l = 0, 1, 2, i ≤ j, i, j = 1, . . . , m,

14

3.7. BQUE for parameters of covariance matrix

Since P commutes with the covariance matrix of y, for each parameter of covariance there exists BQUE if and only if sp{MV M}, is a quadratic subspace (see Zmy´ slony (1976, 1980) and Gnot et al. (1976, 1977a,c)) or Jordan algebra (see Jordan et al. (1934)), where V stands for covariance matrix of y. It is clear that if sp{V } is a quadratic subspace and if for each Σ ∈ sp{V } commutativity P Σ = ΣP holds, then sp{MV M} = sp{MV } is also a quadratic subspace. According to the coordinate free approach, the expectation of Myy′M can be written as a linear combination of matrices MK(0)

ij , MK(1) ij and MK(2) ij

with unknown coefficients σ(0)

ij , σ(1) ij and σ(2) ij , respectively.

15

3.7. BQUE for parameters of covariance matrix

Defining m(m+1)

2

column vectors σ(l) = [σ(l)

ij ] for i ≤ j = 1, . . . , m; l = 0, 1, 2,

we see that the normal equations have the following block diagonal structure:     a b c b d e c e f   ⊗ I m(m+1)

2

    σ(0) σ(1) σ(2)   =   r(0) r(1) r(2)   , (1) where for i ≤ j = 1, . . . , m; a = tr

• M(K(0)

ij )2

, b = tr

• MK(0)

ij K(1) ij

• ,

c = tr

• MK(0)

ij K(2) ij

• , d = tr
• M(K(1)

ij )2

, e = tr

• MK(1)

ij K(2) ij

• and

f = tr

• M(K(2)

ij )2

, while the

m(m+1) 2

× 1 vector r(l) =

1 2−δij

• r′K(l)

ij r

• for

l = 0, 1, 2, δij is the Kronecker delta and r stands for the residual vector, i.e., r = My = (Invum − P )y. 16

3.7. BQUE for parameters of covariance matrix

Let C0, C1 and C2 be defined as follows: C0 =

v

• t=1

u

• s=1

n

• r=1
• yr,ts −

µ yr,ts − µ ′ , C1 =

v

• t=1

u

• s=1

u

• s∗=1

s=s∗ n

• r=1
• yr,ts∗ −

µ yr,ts − µ ′ , C2 =

v

• t=1

v

• t∗=1

t=t∗ u

• s=1

u

• s∗=1

n

• r=1
• yr,t∗s∗ −

µ yr,ts − µ ′ , where µ =

1 nvu

n

r=1

v

t=1

u

s=1 yr,ts.

17

3.7. BQUE for parameters of covariance matrix

Now the right hand side of equation (1) can be expressed by C0, C1 and C2 respectively, and then we have:     a b c b d e c e f   ⊗ Im     Γ0 Γ1 Γ2   =   C0 C1 C2   . Solving this equation we get   Γ0 Γ1 Γ2   =      

(n−1)vu+1 n(n−1)v2u2 1 n(n−1)v2u2 1 n(n−1)v2u2 1 n(n−1)v2u2 (n−1)vu+u−1 n(n−1)v2u2(u−1) 1 n(n−1)v2u2 1 n(n−1)v2u2 1 n(n−1)v2u2 nv−1 n(n−1)v2(v−1)u2

   ⊗ Im      C0 C1 C2   . 18

3.7. BQUE for parameters of covariance matrix

Estimators for Γ0, Γ1 and Γ2 are:

• Γ0 = (n − 1)vu + 1

n(n − 1)v2u2 C0 + 1 n(n − 1)v2u2C1 + 1 n(n − 1)v2u2C2,

• Γ1 =

1 n(n − 1)v2u2C0 + (n − 1)vu + u − 1 n(n − 1)v2u2(u − 1)C1 + 1 n(n − 1)v2u2C2,

• Γ2 =

1 n(n − 1)v2u2C0 + 1 n(n − 1)v2u2C1 + nv − 1 n(n − 1)v2(v − 1)u2C2.

• Theorem. Estimators for µ and Γ are consistent. Moreover, the family of

distributions of these estimators is complete. 19

3.8. Consistency of estimators

var( σ(0)

ij )

= 2((n − 1)uv + 1) (n − 1)nu2v2 tr(AijΓ0AijΓ0) + 4(u − 1) (n − 1)nu2v2tr(AijΓ0AijΓ1) + 4(v − 1) (n − 1)nuv2tr(AijΓ0AijΓ2) +2(u − 1)((n − 1)uv + u − 1) (n − 1)nu2v2 tr(AijΓ1AijΓ1) +4(u − 1)(v − 1) (n − 1)nuv2 tr(AijΓ1AijΓ2) +2(v − 1)(nv − 1) (n − 1)nv2 tr(AijΓ2AijΓ2). 20

3.8. Consistency of estimators

var( σ(1)

ij )

= 2((n − 1)uv + u − 1) (n − 1)n(u − 1)u2v2 tr(AijΓ0AijΓ0) +4((u − 2)u((n − 1)v + 1) + 1) (n − 1)n(u − 1)u2v2 tr(AijΓ0AijΓ1) + 4(v − 1) (n − 1)nuv2tr(AijΓ0AijΓ2) +2(u((u − 3)u + 3)((n − 1)v + 1) − 1) (n − 1)n(u − 1)u2v2 tr(AijΓ1AijΓ1) +4(u − 1)(v − 1) (n − 1)nuv2 tr(AijΓ1AijΓ2) +2(v − 1)(nv − 1) (n − 1)nv2 tr(AijΓ2AijΓ2). 21

3.8. Consistency of estimators

var( σ(2)

ij )

= 2(nv − 1) (n − 1)nu2(v − 1)v2tr(AijΓ0AijΓ0) + 4(u − 1)(nv − 1) (n − 1)nu2(v − 1)v2tr(AijΓ0AijΓ1) + 4(n(v − 2)v + 1) (n − 1)nu(v − 1)v2tr(AijΓ0AijΓ2) + 2(u − 1)2(nv − 1) (n − 1)nu2(v − 1)v2tr(AijΓ1AijΓ1) +4(u − 1)(n(v − 2)v + 1) (n − 1)nu(v − 1)v2 tr(AijΓ1AijΓ2) +2(nv((v − 3)v + 3) − 1) (n − 1)n(v − 1)v2 tr(AijΓ2AijΓ2). 22

3.8. Consistency of estimators

It follows that for any fixed Γ if n → ∞ then var( σ(0)

ij ) → 0,

var( σ(1)

ij ) → 0,

var( σ(2)

ij ) → 0.

Thus estimators σ(0)

ij ,

σ(1)

ij and

σ(2)

ij are consistent.

Note that the estimators for µ and the estimators for elements of covari- ance matrix are one-to-one functions of minimal sufficient statistic. 23

3.8. Consistency of estimators

We consider the situation where Γ0 = I, an identity matrix, and Γ1 = 0, the matrix of zeros. If Γ0 = I and Γ1 = Γ2 = 0, then Γ is the identity matrix Γ = I. For the identity matrix Γ the formulas of variances of estimators for σ(0), σ(1) and σ(2) are: var( σ(0)

ij )

= 2((n − 1)uv + 1) (n − 1)nu2v2 , var( σ(1)

ij )

= 2((n − 1)uv + u − 1) (n − 1)n(u − 1)u2v2 , and var( σ(2)

ij )

= 2(nv − 1) (n − 1)nu2(v − 1)v2. 24

• 4. Comparison of BUE in two models

Consider two models, Mo1 and Mo2, both with a doubly exchangeable co- variance structure, in which the first one has the unstructured mean vector 1n⊗µ (where µ has vum components) and the second one has the structured mean vector 1nvu ⊗ µ (where µ has m components). Variances of covariance parameters under models Mo1 are σ(0)

[1]ij,

σ(1)

[1]ij,

σ(2)

[1]ij,

and Mo2 are σ(0)

[2]ij,

σ(1)

[2]ij and

σ(2)

[2]ij. Denote P [1] and P [2] as orthogonal pro-

jectors for unstructured and structured space generated by mean vector, re-

• spectively. Since R(P [2]) ⊂ R(P [1]) then matrices P [1]P [2] = P [2]P [1] = P [2].

One can easily check that the expectation of σ(0)

[1]ij,

σ(1)

[1]ij and

σ(2)

[1]ij calculated

for Mo1 are unbiased under Mo2. 25

• 4. Comparison of BUE in two models

One can show that differences of variances for both models satisfy inequalities: var( σ(0)

[2]ij) − var(

σ(0)

[1]ij) < 0,

var( σ(1)

[2]ij) − var(

σ(1)

[1]ij) < 0,

var( σ(2)

[2]ij) − var(

σ(2)

[1]ij) < 0.

26

• 4. Comparison of BUE in two models

Considering the situation where Γ0 = I and Γ1 = Γ2 = 0 thus V = I we get: var( σ(0)

[2]ij) − var(

σ(0)

[1]ij) = −

2(uv − 1) (n − 1)nu2v2, var( σ(1)

[2]ij) − var(

σ(1)

[1]ij) = −

2(u(v − 1) + 1) (n − 1)n(u − 1)u2v2, var( σ(2)

[2]ij) − var(

σ(2)

[1]ij) = −

2 (n − 1)nu2(v − 1)v2. For graphical illustration of these differences, author fixed Γ0 = I and Γ1 = Γ2 = 0. For each figure, values for n are chosen from 3 to 25 and for u and v from 2 to 10. For the plot of n and u, v is treated as constant and v = 2. Similarly, For the plot of n and v, u is treated as constant and u = 2. 27

Figure 1: var( σ(0)

[2]ij) − var(

σ(0)

[1]ij) for n and u, plotting separate figure for

parameters n and v is redundant because difference is symmetric with respect to u and v. 28

Figure 2: (A) var( σ(1)

[2]ij) − var(

σ(1)

[1]ij) for n and u, and (B) var(

σ(1)

[2]ij) −

var( σ(1)

[1]ij) for n and v.

29

Figure 3: (A) var( σ(2)

[2]ij) − var(

σ(2)

[1]ij) for n and u, and (B) var(

σ(2)

[2]ij) −

var( σ(2)

[1]ij) for n and v.

30

References

[1] Fonseca, M., Mexia, J.T. and Zmy´ slony, R., 2010. Least squares and gen- eralized least squares in models with orthogonal block structure. Journal

• f Statistical Planning and Inference, 140(5), 1346-1352.

[2] Drygas, H., 1970. The Coordinate-Free Approach to Gauss-Markov Es- timation, Berlin, Heidelberg: Springer. 31

[3] Gnot, S., Klonecki, W. and Zmy´ slony, R. 1977. Uniformly minimum variance unbiased estimation in various classes of estimators. Statistics 8(2), 199-210. [4] Gnot, S., Klonecki, W. and Zmy´ slony, R. 1980. Best unbiased estimation: a coordinate free-approach. Probability and Statistics, 1(1), 1-13. [5] Jordan, P., Neumann, von, J. and Wigner, E., 1934. On an algebraic gen- eralization of the quantum mechanical formalism. The Annals of Math- ematics, 35(1), 29-64. [6] Kruskal, W., 1968. When are Gauss-Markov and Least Squares Estima- tors Identical? A Coordinate-Free Approach. The Annals of Mathemat- ical Statistics, 39(1), pp.70-75. 32

[7] Lehmann, E.L. and Casella, G., 1998. Theory of Point Estimation Second Edition, Springer. [8] Roy, A. and Leiva, R. 2011. Estimating and testing a structured co- variance matrix for three-level multivariate data, Communications in Statistics - Theory and Methods, 40(11), 1945-1963. [9] Roy, A., Fonseca, M. 2012. Linear models with doubly exchangeable distributed errors, Comm. Statist. Theory Methods, 41, 25452569. [10] Roy, A., Zmy´ slony, R., Fonseca, M. and Leiva, R. 2016. Optimal es- timation for doubly multivariate data in blocked compound symmetric covariance structure, Journal of Multivariate Analysis. 33

[11] Kozio l, A., Roy, A., Zmy´ slony, R., Fonseca, M. and Leiva, R. 2016. Best unbiased estimates for parameters of three-level multivariate data with doubly exchangeable covariance structure, Statistics (submitted). [12] Seely, J.F., 1971. Quadratic subspaces and completeness. The Annals of Mathematical Statistics, 42(2), 710-721. [13] Seely, J.F., 1972. Completeness for a family of multivariate normal dis-

• tributions. The Annals of Mathematical Statistics, 43, 1644-1647.

[14] Seely, J.F., 1977. Minimal sufficient statistics and completeness for mul- tivariate normal families. Sankhya (Statistics). The Indian Journal of

• Statistics. Series A, 39(2), 170-185.

[15] Zmy´ slony, R. 1976. On estimation of parameters in linear models, Ap- plicationes Mathematicae XV 3(1976), 271-276. 34

[16] Zmy´ slony, R. 1978. A characterization of best linear unbiased estimators in the general linear model, Lecture Notes in Statistics, 2, 365-373. [17] Zmy´ slony, R. 1980. Completeness for a family of normal distributions, Mathematical Statistics, Banach Center Publications 6, 355-357. 35