On Synchronized Permutation tests in Two-way ANOVA Dario Basso 1 , - - PowerPoint PPT Presentation

On Synchronized Permutation tests in Two-way ANOVA

Dario Basso1, Luigi Salmaso1, Fortunato Pesarin2

1: Department of Management and Engineering, University of Padova 2: Department of Statistics, University of Padova dario@stat.unipd.it

mODa8, June 3 - 8, 2007 - Almagro

– p.1/29

The I × J Balanced Design

In I × J replicated designs, the responses are assumed to follow the linear model: yijk = µ + αi + βj + γij + εijk where: αi is the effect of the ith level of Factor A, i = 1, . . . , I; βj is the effect of the jth level of Factor B, j = 1, . . . , J; γij is the effect of the ijth level of interaction AB; εijk (k = 1, . . . , n) are i.i.d. random errors with: E[εijk] = 0; V ar[εijk] = σ2. The effects are assumed to satisfy the side-conditions:

i

αi =

j

βj = 0;

i

γij = 0 ∀j,

j

γij = 0 ∀i.

– p.2/29

Representation of the response’s structure

Factor B . . . Bj . . . Bh . . . . . . ... . . . . . . . . . ... Ai . . . µ + αi + βj + γij + εijk . . . µ + αi + βh + γih + εihk . . . Factor A . . . ... . . . ... . . . ... As . . . µ + αs + βj + γsj + εsjk . . . µ + αs + βh + γsh + εshk . . . . . . ... . . . . . . . . . ...

– p.3/29

Hypotheses of Interest

H0A : αi = 0 ∀i vs. H1A : ∃ αi = 0 H0B : βj = 0 ∀j vs. H1B : ∃ βj = 0 H0AB : γij = 0 ∀i, j vs. H1AB: ∃ γij = 0 The aim is to find three uncorrelated test statistics (saparately) for each null hypothesis.

– p.4/29

Normality Assumption: The Parametric Solution

If εijk ∼ N(0, σ2), define: MSA =

i [¯

yi·· − ¯ y···]2/(I − 1) MSB =

i [¯

y·j· − ¯ y···]2/(J − 1) MSAB =

i

j [¯

yij· − ¯ yi·· − ¯ y·j· + ¯ y···]2/(I − 1)(J − 1) MSε =

i

j

k

• yijk − ¯

yij·

FA = MSA/MSε ∼ FI−1,IJ(n−1) if H0A is true. FB = MSB/MSε ∼ FJ−1,IJ(n−1) if H0B is true. FAB = MSAB/MSε ∼ F(I−1)(J−1),IJ(n−1) if H0AB is true. Can the Normality if errors be assumed? The two-way ANOVA test statistics are positively correlated.

– p.5/29

Correlation Among Two-Way ANOVA Test Statistics

The test statistics for main factors: FA = MSA/MSε FB = MSB/MSε are positively correlated. In fact: COV [FA, FB] = E[FA · FB] − E[FA]E[FB] = E[MSA]E[MSB]E[MS−2

ε

] + − E[MSA]E[MSB]E[MS−1

ε

]2 = E[MSA]E[MSB]V ar[MS−1

ε

] > 0.

– p.6/29

Correlation Among Two-Way ANOVA Test Statistics

It can be proved that the correlation between FA and FB is: ρ(FA, FB) = σ4 [(I − 1) + α][(J − 1) + β]

˜ σ2(FA)˜ σ2(FB) where: ˜ σ2(FA) = σ4

˜ σ2(FB) = σ4

• β2 + [(J − 1) + 2β][(I − 1) + IJ(n − 1) − 2]
• .

α =

i α2 i /σ2

β =

j β2 j /σ2

– p.7/29

Error Exchangeability Assumption: Nonparametric Solution

A jointly sufficient statistic for the three null hypotheses is: Y = [y11, y12, . . . , yij, . . . , yIJ] where yij is the vector of n units in block AiBj. ⇒ Units are not exchangeable between blocks, unless H0A ∩ H0B ∩ H0AB is true. ⇒ To separately test for factor A effects we need that exchangeability is satisfied under H0A regardless H0B ∪ H0AB is true or not. Therefore....

– p.8/29

Permuting within columns

Factor B . . . Bj . . . Bh . . . . . . ... . . . . . . . . . ... Ai . . . yij1 yij2 . . . yijn . . . yih1 yih2 . . . yihn . . . Factor A . . .

• ν∗

is|j/n

• ν∗

is|h/n

• As

. . . ysj1 ysj2 . . . ysjn . . . ysh1 ysh2 . . . yshn . . . . . . ... . . . . . . . . . ... If ν∗

is|j = ν∗ is|h no proper solution is possible.

– p.9/29

Testing Factor A

Suppose that ν∗

is|j units are exchanged between block AiBj and AsBj, j = 1, . . . , J.

The totals of blocks AiBj and AsBj have the following structure:

n

k

y∗

ijk = ν∗ is|j [µ + αs + βj + γsj] + (n − ν∗ is|j) [µ + αi + βj + γij] + n

k

ε∗

ijk, n

k

y∗

sjk = ν∗ is|j [µ + αi + βj + γij] + (n − ν∗ is|j) [µ + αs + βj + γsj] + n

k

ε∗

sjk.

Let us consider the structure of the difference of these totals, to compare αi and αs at level Bj:

aT ∗ is|j =

k

y∗

ijk −

k

y∗

sjk = (n − 2ν∗ is|j)[αi − αs + γij − γsj] + ε∗ is|j

Similarly, at level Bj:

aT ∗ is|h =

k

y∗

ihk −

k

y∗

shk = (n − 2ν∗ is|h)[αi − αs + γih − γsh] + ε∗ is|h

– p.10/29

Testing Factor A

Because of the side-conditions on γ’s, the sum over all columns:

j aT ∗ is|j =

j

(n − 2ν∗

is|j)[αi − αs + γij − γsj] +

j

ε∗

is|j,

• nly depends on effects αi and αs and on exchangeable errors if and only if ν∗

is|j = ν∗ is, ∀j.

We have to synchronize the permutations with respect to the levels of factor B. When ν∗

is|j = ν∗ is, ∀j,

j aT ∗ is|j = (n − 2ν∗ is)J[αi − αs] +

j

ε∗

is|j,

and (when ν∗

is = ν∗) the test statistic: aT ∗ A =

i<s

[

j aT ∗ is|j]2,

• nly depends on effects of factor A and on exchangeable errors, hence it is a separate

exact test on factor A.

– p.11/29

Permuting within rows

Factor B . . . Bj . . . Bh . . . . . . ... . . . ⇐ ⇒ . . . ... Ai . . . yij1 yij2 . . . yijn ⇐ ⇒ yih1 yih2 . . . yihn . . . ν∗

jh|i/n

Factor A . . . ... . . . ⇐ ⇒ . . . ... As . . . ysj1 ysj2 . . . ysjn ⇐ ⇒ ysh1 ysh2 . . . yshn . . . ν∗

jh|s/n

. . . ... . . . ⇐ ⇒ . . . ... If ν∗

jh|i = ν∗ jh|s no proper solution is possible.

– p.12/29

Testing Factor B

Let:

bT ∗ jh|i = (n − 2ν∗ jh|i) [βj − βh + γij − γih] +

k ε∗ ijk −

k ε∗ ihk

be the difference between the totals of blocks AiBj and AiBh after we have exchanged ν∗

jh|i units (out of n);

if ν∗

jh|i = ν∗ jh ∀i, because of the side-conditions on γ’s, the sum: I

i=1 bT ∗ jh|i = (n − 2ν∗ jh)I [βj − βh] + I

i=1

k

ε∗

ijk −

k

ε∗

ihk

Similarly, (when ν∗

jh = ν∗) by interchanging indeces and by applying synchronized

permutations within rows, the test statistic:

bT ∗ B =

j<h

i bT ∗ jh|i

allows for nonparametric exact testing on Factor B.

– p.13/29

Testing the Interaction

Consider the synchronized permutations within columns; recall:

aT ∗ is|j

= (n − 2ν∗

is)[αi − αs + γij − γsj] + ε∗ is|j aT ∗ is|h

= (n − 2ν∗

is)[αi − αs + γih − γsh] + ε∗ is|h.

The difference:

aT ∗ is|j − aT ∗ is|h = (n − 2ν∗ is)[γij − γsj − γih + γsh] + ε∗ is|j − ε∗ is|h

• nly depends on the interaction effects, therefore (when ν∗

is = ν∗), the test statistic: aT ∗ AB =

i<s

j<h

aT ∗ is|j − aT ∗ is|h

allows for nonparametric exact testing on Interaction. By interchanging indeces (and by synchronizing within rows), there is another test statistic:

bT ∗ AB =

j<h

i<s

bT ∗ jh|i − bT ∗ jh|s

which allows for nonparametric exact testing on Interaction.

– p.14/29

Constrained Synchronized Permutations (CSPs)

Factor B . . . Bj . . . Bh . . . . . . ... . . . . . . . . . ... Ai . . . yij1 yij2 . . . yijn . . . yih1 yih2 . . . yihn . . . . . . . . . Factor A . . . ν∗ ν∗ ν∗ ν∗ ν∗ As . . . ysj1 ysj2 . . . ysjn . . . ysh1 ysh2 . . . yshn . . . . . . . . . . . . ... . . . . . . . . . ... Note that ν∗ is itself a random variable. In this example ν∗ = 1 . ν∗ units from the same original positions are exchanged between each pair of blocks.

– p.15/29

Unconstrained Synchronized Permutations (USPs)

Factor B . . . Bj . . . Bh . . . . . . ... . . . . . . . . . ... Ai . . . yij1 yij2 . . . yijn . . . yih1 yih2 . . . yihn . . . . . . . . . Factor A . . . ν∗ ν∗ ν∗ ν∗ ν∗ As . . . ysj1 ysj2 . . . ysjn . . . ysh1 ysh2 . . . yshn . . . . . . . . . . . . ... . . . . . . . . . ... Note that ν∗ is itself a random variable. In this example ν∗ = 1. ν∗ units are exchanged between each pair of blocks (regardless their original positions).

– p.16/29

Cardinality of CSPs and USPs

CSPs depend on the rearrangement made in the 1st pair of blocks (AiB1 and AsB1), hence the cardinality of CSPs is: CCSP =

n

• USPs depend on all possible ways to exchange ν∗ units between pairs of blocks for all

columns and all possible pairs of rows, hence: CUSP =

n

ν∗=0

ν∗

– p.17/29

Cardinality of the support of aT ∗

A with CSPs

The squaring operator in the test statistic produces a symmetry among the permutation values of aT ∗

• A. For instance, if n = 3:

. . . Bj . . . Bh . . . Ai . . . yij1 yij2 yij3 . . . yih1 yih2 yih3 . . . . . .

• ν∗ = 1 ν∗ = 2
• ν∗ = 1 ν∗ = 2
• As

. . . ysj1 ysj2 ysj3 . . . ysh1 ysh2 ysh3 . . . Hence the cardinality of the support of aT ∗

A when CSPs are applied is:

BCSP

aT ∗ A =

n

• /2

– p.18/29

Cardinality of the support of aT ∗

A with USPs

Due to the symmetry in the distribution of aT ∗

A, the cardinality aT ∗ A with USPs is:

BUSP

aT ∗ A =

(n−1)/2

ν∗=0

ν∗

when n is odd, BUSP

aT ∗ A =

n/2−1

ν∗=0

ν∗

+

n/2

2

I(I−1)/2

when n is even.

– p.19/29

Uncorrelation Among Synchronized Permutation Tests

Synchronized permutation test statistics for main factors are uncorrelated: suppose we are testing for factor H0A (i.e. we exchange units within columns): the total of each column is constant, therefore:

i bT ∗ jh|i =

i

[

k

y∗

ijk −

k

y∗

ihk] =

i

k

y∗

ijk −

i

k

y∗

ihk = Y·j − Y·h

hence:

bT ∗ B =

j<h

[

i bT ∗ jh|i] 2 ≡

j<h

(Y·j − Y·h)2 = bTB where bTB is the observed value of the test statistic for factor B. Since bT ∗

B is constant, ρ(aT ∗ A, bT ∗ B) = 0.

It has been empirically proved (via simulations) that: ρ(aT ∗

A, aT ∗ AB) = 0

ρ(bT ∗

B, bT ∗ AB) = 0

– p.20/29

A Simulation Under H0A

p−value C.d.f 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 3 replicates 0.0 0.2 0.4 0.6 0.8 1.0 5 replicates 0.0 0.2 0.4 0.6 0.8 1.0 10 replicates CSP USP

Cumulative distribution function of paT ∗

A under H0A in a 3 × 2 Design.

Normal Errors with σ = 1.

– p.21/29

Uncorrelation Among Synchronized Permutation Tests

50 100 150 200 20 40 60 80 100 T(A) T(B) 5 10 15 20 10 20 30 F(A) F(B)

0.2 0.4 0.6 0.8 1.0 0.2 0.4 0.6 0.8 1.0 p[T(A)] p[T(B)] 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 p[F(A)] p[F(B)]

Settings: I = J = 2, n = 3. Data under H0A ∩ H0B. εijk ∼ N(0, 1).

– p.22/29

Uncorrelation Among Synchronized Permutation Tests

11000 13000 15000 17000 20 40 60 80 100 T(A) T(B) 2000 4000 6000 8000 10000 10 20 30 F(A) F(B) 0.07 0.08 0.09 0.10 0.11 0.12 0.13 0.2 0.4 0.6 0.8 1.0 p[T(A)] p[T(B)] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.0 0.2 0.4 0.6 0.8 1.0 p[F(A)] p[F(B)]

Settings: I = J = 2, n = 3. Data under H0B ∩ H1A : α1 = −α2 = 10σ .

– p.23/29

Uncorrelation Among Synchronized Permutation Tests

12000 14000 16000 2500 3500 4500 T(A) T(B) 2000 4000 6000 8000 10000 500 1500 2500 F(A) F(B) 0.08 0.09 0.10 0.11 0.12 0.13 0.08 0.10 0.12 p[T(A)] p[T(B)] 0 e+00 2 e−08 4 e−08 0.0 e+00 6.0 e−06 1.2 e−05 p[F(A)] p[F(B)]

Settings: I = J = 2, n = 3. Data under H1A : α1 = −α2 = 10σ and H1B : β1 = −β2 = 5σ – p.24/29

A Power Comparison

p−value C.d.f

0.0 0.2 0.4 0.6 0.8 1.0

USP 3 replicates CSP 5 replicates CSP 10 replicates

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

ANOVA 3 replicates

0.0 0.2 0.4 0.6 0.8 1.0

ANOVA 5 replicates

0.0 0.2 0.4 0.6 0.8 1.0

ANOVA 10 replicates 3 levels H0 2 levels H0 3 levels H1 2 levels H1

Cumulative distribution functions of paT ∗

A and paT ∗ B in a 3 × 2 Design.

Cumulative distribution functions of pFA and pFB in a 3 × 2 Design. Settings: α1 = −α3 = σ/2, α2 = 0; β1 = −β2 = σ/2. Normal Errors with σ = 1.

– p.25/29

References

Basso D., Chiarandini M., Salmaso L. (2006). Synchronized Permutation Tests in I × J

• Designs. Journal of Statistical Planning and Inference, in press, available on line.

Salmaso L. (2003). Synchronized Permutation Tests in Factorial Designs. Communications in Statistics-Theory and Methods, 32, 1419-1437. Pesarin F . (2001). Multivariate Permutation Tests with Applications to Biostatistics.

Wiley-Chichester.

– p.26/29

Obtaining Monte Carlo CSPs for aT ∗

A

Let y be a 2n × 1 vector of data Let π(y) = y∗ be a random permutation of y; For any pair of rows, i and s, consider the pooled vectors yis|j =

• yij, ysj
• , ∀j;

Apply the same permutation π to all yis|j, obtaining yis|j∗ = π(yis|j), ∀j; Split any yis|j∗ into yij∗ and ysj∗, i.e. assign the first n units of yis|j∗ to yij∗ and the

• ther units of yis|j∗ to ysj∗, ∀j.

– p.27/29

Obtaining Monte Carlo USPs for aT ∗

A

Consider the case when n is even. Define: pν∗ =

ν∗

µ=0

µ

ν∗ = 0, 1, . . . , n. Build the vector (n + 1) × 1: Pν∗ = [p0, p1, . . . , pn]′ = {Pr [ν∗ = i] , i = 0, . . . , n} ; Obtain u, a realization from U[0, 1]. for k = 0, 1, . . . , n, if u ≤ pk then ν∗ = k. For any couple of rows i and s, do: Shuffle the units within each block; Exchange the first ν∗ units from block AiBj and AsBj, ∀j. If n is even define pν∗ in accordance to the event {n is even}.

– p.28/29

(Discrete) Uniform Distribution of aT ∗

A

In synchronized permutation testing: P [aT ∗

A = at∗ A] = P [aT ∗ A = at∗ A|ν∗] × pν∗

If CSPs are applied, then P[aT ∗

A|ν∗ = x] = P[aT ∗ A|ν∗ = n − x], hence:

P [aT ∗

A = at∗ A] =

1

n

1

n

2

n

1 BCSP

aT ∗ A

If USPs (n is odd) are applied, then: P [aT ∗

A = at∗ A]

=

ν∗

µ=0

µ

1

ν∗

= 1

µ=0

µ

= 1 BUSP

aT ∗ A

– p.29/29