The Poisson trick for matched two-way tables a case for putting the - - PowerPoint PPT Presentation

Plan Key ideas Example Plots Bilinear models Case of two matched tables References

The Poisson trick for matched two-way tables

a case for putting the fish in the bowl (a case for putting the bird in the cage) Simplice Dossou-Gb´ et´ e1, Antoine de Falguerolles2,*

• 1. Universit´

e de Pau et des Pays de l’Adour

• 2. Universit´

e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

31 January 2011

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References

Plan Key ideas Matched two-way tables Objectives Poisson trick The suicide data: age, method and gender Data CAs for the two matched tables Plots Bird Fish Bilinear models restricted two-way interaction Case of two matched tables Poisson-Multinomial trick for two matched tables References

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References

Key ideas

◮ Matched two-way tables ◮ Analysis of dissimilarity/similarity between tables ◮ Poisson trick

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Matched two-way tables

matched two-way tables

The m tables of counts classified by factor A (row) and factor B (column), Y SAB

k

, their margins Y SA

k

and Y SB

k

and total count Y S

k

ySAB

1

ySA

1

(ySB

1 )′

yS

1

. . . ySAB

s

ySA

s

(ySB

s

)′ yS

s

. . . ySAB

#S

ySA

#S

(ySB

#S)′

yS

#S

The marginal two-way table (and its margins) yAB yA (yB)′ y∅

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Objectives

Objectives

Similarity/Dissimilarity between tables row profiles or column profiles May involve some preprocessing of

◮ tables by unifying margins by biproportional fitting (RAS,

Iterative Proportional Fitting, matrix Raking)

◮ row profiles (column profiles) by weighting tables, profiles into

tables, common metric

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Poisson trick

Poisson trick

◮ Y SAB sab

independent Poisson E[Y SAB

sab ]

= var(Y SAB

sab )

E[Y SAB

sab ]

= m(βAB

ab + restricted(βSAB sab )) ◮ Y SAB sab | #S s=1 Y SAB sab

= yAB

ab

multinomial with

◮ known parameter: y AB

ab

◮ probabilities:

m(βAB

ab + restricted(βSAB sab ))

m

k=1 m(βAB ab + restricted(βSAB sab )) = m(βAB ab + restricted(βSAB sab ))

y AB

ab

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Poisson trick

Poisson trick for two matched tables

Particular case: two matched tables (#M = 2)

◮ independant Poisson counts E[Y SAB sab ] (s = 1, 2)

◮ exponential mean function (log link function): m = exp,

m−1 = log

◮ model: all two-way interactions of A , B and F

E[Y SAB

sab ]

= exp(βAB

ab + βSA sa + βSB sb )

◮ Y SAB 2ab

binomial B(yAB

ab , πAB 2ab)

◮ model: additivity of effects of A and B

logit(πAB

2ab)

= βSA

2a + βSB 2b

Works also with the inclusion of a reduced rank interaction in the predictor

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Data

Male

Method Age c1 c2 c3 c4 c5 c6 c7 c8 c9 10-15 4 247 1 17 1 6 9 15-20 348 7 67 578 22 179 11 74 175 20-25 808 32 229 699 44 316 35 109 289 25-30 789 26 243 648 52 268 38 109 226 30-35 916 17 257 825 74 291 52 123 281 35-40 1118 27 313 1278 87 293 49 134 268 40-45 926 13 250 1273 89 299 53 78 198 45-50 855 9 203 1381 71 347 68 103 190 50-55 684 14 136 1282 87 229 62 63 146 55-60 502 6 77 972 49 151 46 66 77 60-65 516 5 74 1249 83 162 52 92 122 65-70 513 8 31 1360 75 164 56 115 95 70-75 425 5 21 1268 90 121 44 119 82 75-80 266 4 9 866 63 78 30 79 34 80-85 159 2 2 479 39 18 18 46 19 85-90 70 1 259 16 10 9 18 10 90+ 18 1 76 4 2 4 6 2

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Data

Female

Method Age c1 c2 c3 c4 c5 c6 c7 c8 c9 10-15 28 3 20 1 10 6 15-20 353 2 11 81 6 15 2 43 47 20-25 540 4 20 111 24 9 9 78 67 25-30 454 6 27 125 33 26 7 86 75 30-35 530 2 29 178 42 14 20 92 78 35-40 688 5 44 272 64 24 14 98 110 40-45 566 4 24 343 76 18 22 103 86 45-50 716 6 24 447 94 13 21 95 88 50-55 942 7 26 691 184 21 37 129 131 55-60 723 3 14 527 163 14 30 92 92 60-65 820 8 8 702 245 11 35 140 114 65-70 740 8 4 785 271 4 38 156 90 70-75 624 6 4 610 244 1 27 129 46 75-80 495 8 1 420 161 1 29 129 35 80-85 292 3 2 223 78 10 84 23 85-90 113 4 83 14 6 34 2 90+ 24 1 19 4 2 7

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References CA

Two approaches in CA

◮ Peter’s trick:

• rdinary CA of either table

M F

• and/or

M′ F ′

• ◮ Michael’s trick:
• rdinary CA of table

M F F M

• equivalent to

◮ ordinary CA of the ‘average’ table 1

2M + 1 2F

◮ adapted CA of table M (resp. table F)

with respect to 1

2M + 1 2F.

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References CA

Two approaches in CA (Continued)

◮ Implicit in the first stream of approaches are

◮ choice of a log-linear model between C + S ∗ R and R + S ∗ C

where R, C, and S denote row, column, matching factors

◮ ordinary CA of the table formed accordingly

◮ Implicit in the second stream of approaches are

◮ metric choice for the rows (the ages) and the columns (the

causes): metrics attached to each table M, F or (smoothed) metrics attached to the ‘average’ table 1

2M + 1 2F or . . . ?

Metric choice impacts plots and, to a lesser extent, patterns in graphs.

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References CA

Peter’s plot

M F

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References CA

Michael’s trick

M F F M

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References CA

Peter’s trick versus Michael’s trick

dissimilarity similarity

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References CA

Peter’s trick versus Michael’s trick

dissimilarity similarity

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Bird

bird and cage

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Bird

trick

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Bird

bird in cage

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Fish

fish and bowl

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Fish

fish in bowl

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References restricted two-way interaction

Notation for a two-way table

◮ Observed #A × #B two-way table yAB of counts cross

classified by factor A (row) and factor B (column), and margins yAB yA (yB)′ y∅

◮ Profiles:

A-profiles yB|A=a

b

=

1 yA

a yAB

ab

B-profiles yA|B=b

a

=

1 yB

b yAB

ab ◮ Weights: wAB ab = y∅ yA

a

y∅ yB

b

y∅ can be generalized into γ∅γA a γB a

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References restricted two-way interaction

Diet modeling

The yAB

ab

are observed values of independant r.v. Y AB

ab

with

◮ expected value: E[Y AB ab ] = µAB ab = m(ηAB ab ) ◮ bi-linear predictor

ηAB

ab

=

• ffset + [β∅ + βA

a + βB b +]

reduced rank interaction

• k=1,...,r

δkβA

k,aβB k,b

with identification constraints for the βs

◮ variance: Var(Y AB ab ) = V (µAB ab )

= ⇒ allows to replicate most models with rank restricted interaction. = ⇒ has consequences on the distribution of the profiles.

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References restricted two-way interaction

Implementations

Current implementations are Benz´ ecri’s CA and Goodman’s RC. But non-canonical crossovers are possible.

◮ CA: µAB ab = wAB ab

(1 + ηAB

ab ) and V (µAB ab ) = wAB ab

a taste of heteroscedastic Normal distribution with a zest of Poisson

◮ RC: µAB ab = exp (ηAB ab ) and V (µAB ab ) = µAB ab

a definite taste of Poisson distribution

◮ GB: µAB ab = exp(ηAB ab ) and V (µAB ab ) = wAB ab ◮ BG: µAB ab = max{ǫ, wAB ab (1 + ηAB ab )} and V (µAB ab ) = µAB ab

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References restricted two-way interaction

Diet Poisson-Multinomial

Yi (i ∈ {1, . . . , n}) are independent r.v. with E[Yi] = µi and Var(Yi) = σ2

i E[ 1 y [Y1, . . . , Yn]′|Y1 + . . . + Yn = y] = 1 y [µ1, . . . , µn]′ + 1 y

i σ2 i

• (y −
• i

µi)[σ2

1, . . . , σ2 n]′

Var( 1 y [Y1, . . . , Yn]′|Y1 + . . . + Yn = y) = 1 y2    σ2

1

... σ2

n

   − 1 y2

i σ2 i

   σ2

1

. . . σ2

n

   σ2

1,

. . . , σ2

n

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Poisson-Multinomial trick for two matched tables

Poisson-Multinomial trick for two matched tables

Poisson counts for the three way table ySAB = (ySAB

1

, ySAB

2

): log(λSAB

2ab )

= β∅ + βM

2 + βA a + βB b + βSA 2a + βSB 2b + βAB ab +

• k

δkξA

akξB bk

log(λSAB

1ab )

= β∅ + + βA

a + βB b +

+ + βAB

ab +

• k

(−δk)ξA

akξB bk

Binomial model for the two way table ySAB

2

given the table yAB (sum of counts of matched cells): logit(πAB

ab ) = log(λSAB 2ab

λSAB

1ab

) = βS

2 + βSA 2a + βSB 2b + 2

• k

δkξA

akξB bk

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Poisson-Multinomial trick for two matched tables

What if CA is used?

Three way table ySAB = (ySAB

1

, ySAB

2

) and associated weights wAB

ab = y∅ yA a

y∅ yb

b

y∅ CA of table ySAB

2

with respect to table 1

2yAB:

1 wAB

ab

E[Y SAB

2ab ] =

1 wAB

ab

(

• ffset

1 2yAB

ab + k δkξA akξB bk)

Interpretation for the reduced rank interaction: 4wAB

ab

yAB

b

• k

δkξA

akξB bk

≈ logit(πSAB

2ab )

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References Poisson-Multinomial trick for two matched tables

Log-odds

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References

References

◮ Peter van der Heijden and Jan de Leeuw (1985):

Correspondence analysis and complementary to loglinear analysis, Psychometrika, 50(4), 429-447.

◮ Michael Greenacre (2003): Singular value decomposition of

matched matrices, Journal of Applied Statistics, 30, 1101-1113.

◮ Simplice Dossou-Gb´

et´ e (2002): Reduced rank quasi-symmetry and biplots for matched two-way tables, Annales de la Facult´ e des Sciences, vol. XI (4), 469-483.

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net

Plan Key ideas Example Plots Bilinear models Case of two matched tables References

Thank you for your atention

• 1. Universit´

e de Pau et des Pays de l’Adour 2. Universit´ e Paul Sabatier (Toulouse III) * Antoine -at- Falguerolles.net