Decomposing the deviance in GLMMs, with applications in marine - - PowerPoint PPT Presentation

Decomposing the deviance in GLMM’s, with applications in marine ecology

Mariangela SCIANDRA, Gianfranco LOVISON

sciandra@dssm.unipa.it, lovison@unipa.it Dipartimento di Scienze Statistiche e Matematiche ‘S. Vianelli’ Universit` a di Palermo

GRASPA Conference 2008 SIENA - March, 27-28 2008

Outline

Outline

• 1. Partitioning variation: Newton and Spurrel’s regression elements

Outline

• 1. Partitioning variation: Newton and Spurrel’s regression elements
• 2. Partitioning variation: Whittaker’s extension

Outline

• 1. Partitioning variation: Newton and Spurrel’s regression elements
• 2. Partitioning variation: Whittaker’s extension
• 3. Partitioning variation: which extension to mixed models?

Outline

• 1. Partitioning variation: Newton and Spurrel’s regression elements
• 2. Partitioning variation: Whittaker’s extension
• 3. Partitioning variation: which extension to mixed models?
• Partitioning the Penalized Quasi-Likelihood

Outline

• 1. Partitioning variation: Newton and Spurrel’s regression elements
• 2. Partitioning variation: Whittaker’s extension
• 3. Partitioning variation: which extension to mixed models?
• Partitioning the Penalized Quasi-Likelihood
• Partitioning the h-Likelihood

Outline

• 1. Partitioning variation: Newton and Spurrel’s regression elements
• 2. Partitioning variation: Whittaker’s extension
• 3. Partitioning variation: which extension to mixed models?
• Partitioning the Penalized Quasi-Likelihood
• Partitioning the h-Likelihood
• 4. Partitioning the Penalized Quasi-Likelihood for given random effects: an

application in Marine Ecology

Outline

• 1. Partitioning variation: Newton and Spurrel’s regression elements
• 2. Partitioning variation: Whittaker’s extension
• 3. Partitioning variation: which extension to mixed models?
• Partitioning the Penalized Quasi-Likelihood
• Partitioning the h-Likelihood
• 4. Partitioning the Penalized Quasi-Likelihood for given random effects: an

application in Marine Ecology

• 5. Open problems: how to quantify the contribution of random effects?

• 1. Partitioning variation: Newton and Spurrel’s regression elements

• 1. Partitioning variation: Newton and Spurrel’s regression elements

How, and at what extent, can we uniquely attribute variation in the response variable to each explanatory variable in classical linear regression?

• 1. Partitioning variation: Newton and Spurrel’s regression elements

How, and at what extent, can we uniquely attribute variation in the response variable to each explanatory variable in classical linear regression? Newton, Spurrel (1967) = ⇒ regression elements

• 1. Partitioning variation: Newton and Spurrel’s regression elements

How, and at what extent, can we uniquely attribute variation in the response variable to each explanatory variable in classical linear regression? Newton, Spurrel (1967) = ⇒ regression elements Let: y = β0 + ǫ G(:) = rss(∅) y = β0 + β1x1 + ǫ G(: 1) = rss(x1) y = β0 + β2x2 + ǫ G(: 2) = rss(x2) y = β0 + β1x1 + β2x2 + ǫ G(: 21) = rss(x1, x2)

• 1. Partitioning variation: Newton and Spurrel’s regression elements

How, and at what extent, can we uniquely attribute variation in the response variable to each explanatory variable in classical linear regression? Newton, Spurrel (1967) = ⇒ regression elements Let: y = β0 + ǫ G(:) = rss(∅) y = β0 + β1x1 + ǫ G(: 1) = rss(x1) y = β0 + β2x2 + ǫ G(: 2) = rss(x2) y = β0 + β1x1 + β2x2 + ǫ G(: 21) = rss(x1, x2) Then: G(1 :) = ss(x1) = G(:) − G(: 1) variation that can be attributed to x1 ignoring x2 G(1 : 2) = ss(x1|x2) = G(: 2) − G(: 12) variation that can be attributed to x1 adjusting for x2 G(2 :) = ss(x2) = G(:) − G(: 2) variation that can be attributed to x2 ignoring x1 G(2 : 1) = ss(x2|x1) = G(: 1) − G(: 12) variation that can be attributed to x2 adjusting for x1 G(12 :) = G(1 :) − G(1 : 2) = G(2 :) − G(2 : 1) variation that can be equally well be attributed to either x1

• r to x2

G(: 12), G(1 : 2), G(2 : 1), G(12 :) are called regression elements

G(: 12), G(1 : 2), G(2 : 1), G(12 :) are called regression elements Notice: G(:) = G(: 12) + G(1 : 2) + G(2 : 1) + G(12 :) i.e. the regression elements provide an additive decomposition (i.e. a parti- tion) of the total variation in y.

• 2. Partitioning variation: Whittaker’s extension

• 2. Partitioning variation: Whittaker’s extension

Whittaker(1984) gave a general theoretical framework to Newton and Spurrel’s regression elements, showing how to extend them to more complex (fixed ef- fects) models.

• 2. Partitioning variation: Whittaker’s extension

Whittaker(1984) gave a general theoretical framework to Newton and Spurrel’s regression elements, showing how to extend them to more complex (fixed ef- fects) models. Let: K = {1, 2, . . . , k} the index set of K variables L = {∅, 1, 2, . . . , k, 12, . . . , 123, . . . , 12..k} the power set of K (a binary lattice) S(a), for a ∈ L a set function L → ℜ monotone on L, i.e. S(ia) ≤ S(a), where ia contains the integers in a plus i

• 2. Partitioning variation: Whittaker’s extension

Whittaker(1984) gave a general theoretical framework to Newton and Spurrel’s regression elements, showing how to extend them to more complex (fixed ef- fects) models. Let: K = {1, 2, . . . , k} the index set of K variables L = {∅, 1, 2, . . . , k, 12, . . . , 123, . . . , 12..k} the power set of K (a binary lattice) S(a), for a ∈ L a set function L → ℜ monotone on L, i.e. S(ia) ≤ S(a), where ia contains the integers in a plus i Define: G(: a) = S(a) ∀a ∈ L

• 2. Partitioning variation: Whittaker’s extension

Whittaker(1984) gave a general theoretical framework to Newton and Spurrel’s regression elements, showing how to extend them to more complex (fixed ef- fects) models. Let: K = {1, 2, . . . , k} the index set of K variables L = {∅, 1, 2, . . . , k, 12, . . . , 123, . . . , 12..k} the power set of K (a binary lattice) S(a), for a ∈ L a set function L → ℜ monotone on L, i.e. S(ia) ≤ S(a), where ia contains the integers in a plus i Define: G(: a) = S(a) ∀a ∈ L Then, the additive elements of S(·) on L are denoted by G(a : b) where: {a, b} is a partition of the full index set K and are obtained through the recursion: G(ia : b) = G(a : b) − G(a : ib) ∀a, b ∈ L and i ∈ K

Within this general approach, Newton and Spurrel’s regression elements are just a special case of additive elements, with S(a) = rss(a)

Within this general approach, Newton and Spurrel’s regression elements are just a special case of additive elements, with S(a) = rss(a) Whittaker proposed to use S(a) = deviance(a) to attribute portions of variation to explanatory variables in the linear predictor

• f a GLM.

Within this general approach, Newton and Spurrel’s regression elements are just a special case of additive elements, with S(a) = rss(a) Whittaker proposed to use S(a) = deviance(a) to attribute portions of variation to explanatory variables in the linear predictor

• f a GLM.

Notice that, since: deviance(a) = 2ℓ(saturated) − 2ℓ(a) the additive elements result to be differences in −2ℓ(·). E.g.: G(1 : 23) = G(: 23) − G(: 123) = dev(23) − dev(123) = 2ℓ(saturated) − 2ℓ(23) − [2ℓ(saturated) − 2ℓ(123)] = −2ℓ(23) − [−2ℓ(123)] Consequently, Whittaker called them additive likelihood elements.

• 3. Partitioning variation: which extension to mixed models?

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM).

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models.

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models. However ....extension of the idea of additive elements to mixed models is problematic:

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models. However ....extension of the idea of additive elements to mixed models is problematic:

• choice of likelihood to use:

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models. However ....extension of the idea of additive elements to mixed models is problematic:

• choice of likelihood to use:

– marginal (log-)likelihood ℓ(β, φ, D|y)

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models. However ....extension of the idea of additive elements to mixed models is problematic:

• choice of likelihood to use:

– marginal (log-)likelihood ℓ(β, φ, D|y) – conditional (log-)likelihood ℓ(β, φ, D|y, b)

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models. However ....extension of the idea of additive elements to mixed models is problematic:

• choice of likelihood to use:

– marginal (log-)likelihood ℓ(β, φ, D|y) – conditional (log-)likelihood ℓ(β, φ, D|y, b) – joint (log-)likelihood ℓ(β, φ, b, D|y)

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models. However ....extension of the idea of additive elements to mixed models is problematic:

• choice of likelihood to use:

– marginal (log-)likelihood ℓ(β, φ, D|y) – conditional (log-)likelihood ℓ(β, φ, D|y, b) – joint (log-)likelihood ℓ(β, φ, b, D|y)

• choice of method of estimation

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models. However ....extension of the idea of additive elements to mixed models is problematic:

• choice of likelihood to use:

– marginal (log-)likelihood ℓ(β, φ, D|y) – conditional (log-)likelihood ℓ(β, φ, D|y, b) – joint (log-)likelihood ℓ(β, φ, b, D|y)

• choice of method of estimation
• lack of separability of the fixed and random effects

• 3. Partitioning variation: which extension to mixed models?

In ecological applications, often unexplained heterogeneity and between-units dependence are taken into account through random effects within mixed mod- els (LMM, GLMM or HGLM). Our objective is to extend Whittaker additive likelihood elements to mixed models. However ....extension of the idea of additive elements to mixed models is problematic:

• choice of likelihood to use:

– marginal (log-)likelihood ℓ(β, φ, D|y) – conditional (log-)likelihood ℓ(β, φ, D|y, b) – joint (log-)likelihood ℓ(β, φ, b, D|y)

• choice of method of estimation
• lack of separability of the fixed and random effects
• lack of a well defined saturated and null models

We are exploring the potentials of using:

We are exploring the potentials of using:

• an approximation of the joint likelihood ⇒ Penalized Quasi-Likelihood

(Wolfinger, O’Connel,1993; Breslow, Clayton, 1993)

We are exploring the potentials of using:

• an approximation of the joint likelihood ⇒ Penalized Quasi-Likelihood

(Wolfinger, O’Connel,1993; Breslow, Clayton, 1993)

• the exact joint likelihood ⇒ Hierarchical Likelihood (Lee and Nelder,

1996)

3.1 Partitioning the Penalized Quasi-Likelihood PQL = − 1 2φ

m

• i=1

ni

• j=1

dij(yij, µij(β, (b)) − 1 2bTDb (1)

3.1 Partitioning the Penalized Quasi-Likelihood PQL = − 1 2φ

m

• i=1

ni

• j=1

dij(yij, µij(β, (b)) − 1 2bTDb (1) For given random effects, twice the maximized PQL: 2PQL(ˆ

β,ˆ b) = −2 1

2ˆ φ

m

• i=1

ni

• j=1

dij(yij, µij(ˆ

β,ˆ b) − 1

2 ˆ

bT ˆ Dˆ b

(2) can be used to find the ”additive PQL elements” relative to the fixed effects.

3.2 Partitioning the h-Likelihood h = ℓ(β, φ; y|ω(b)) + ℓ(α; ω(b)), (3) MHLE : ∂h/∂β = 0 and ∂h/∂ω = 0.

3.2 Partitioning the h-Likelihood h = ℓ(β, φ; y|ω(b)) + ℓ(α; ω(b)), (3) MHLE : ∂h/∂β = 0 and ∂h/∂ω = 0. MHLEs for β given the random effects are obtained from GLM equations with

ω(b) considered as an offset

⇓ Whittaker’s decomposition can be used on the GLM deviance.

4. Partitioning the Penalized Quasi-Likelihood for given random ef- fects: an application in Marine Ecology

4. Partitioning the Penalized Quasi-Likelihood for given random ef- fects: an application in Marine Ecology In the analysis of the growth performance of Posidonia oceanica, both ”ex-

• genous” and ”endogenous” factors contribute to the explanation of the total

variation in rhizome elongation.

4. Partitioning the Penalized Quasi-Likelihood for given random ef- fects: an application in Marine Ecology In the analysis of the growth performance of Posidonia oceanica, both ”ex-

• genous” and ”endogenous” factors contribute to the explanation of the total

variation in rhizome elongation. Results below are from Gamma-Normal GLMMs on shoots sampled from 14 locations in the Malta area; Location (L) and Calendar Year (CY) have been considered as exogenous factors and rhizome Age (A) as endogenous factor; a random intercept for each shoot is assumed throughout, to take heterogeneity (due to genetic differences, individual features, etc.) into account.

4. Partitioning the Penalized Quasi-Likelihood for given random ef- fects: an application in Marine Ecology In the analysis of the growth performance of Posidonia oceanica, both ”ex-

• genous” and ”endogenous” factors contribute to the explanation of the total

variation in rhizome elongation. Results below are from Gamma-Normal GLMMs on shoots sampled from 14 locations in the Malta area; Location (L) and Calendar Year (CY) have been considered as exogenous factors and rhizome Age (A) as endogenous factor; a random intercept for each shoot is assumed throughout, to take heterogeneity (due to genetic differences, individual features, etc.) into account. Let Location=1, Calendar Year=2 and Age=3; using the PQL decomposition approach, the following total variation partition is obtained:

G(:) = 568.159 G(: 123) = 400.8448 G(1 :) = G(:) − G(: 1) = 568.159 − 517.691 = 50.468 G(1 : 2) = G(: 2) − G(: 12) = 470.7994 − 416.0087 = 54.7907 G(12 :) = G(1 :) − G(1 : 2) = 50.468 − 54.7907 = −4.3227 G(1 : 23) = G(: 23) − G(: 123) = 449.6787 − 400.8448 = 48.8339 G(1 : 3) = G(: 3) − G(: 13) = 474.6905 − 433.6875 = 41.003 G(12 : 3) = G(1 : 3) − G(1 : 23) = 41.003 − 48.8339 = −7.8309 G(123 :) = G(12 :) − G(12 : 3) = −4.3227 + 7.8309 = 3.5082 G(13 : 2) = G(1 : 2) − G(1 : 23) = 54.7907 − 48.8339 = 5.9568 G(2 : 13) = G(: 13) − G(: 123) = 433.6875 − 400.8448 = 32.8427 G(3 : 12) = G(: 12) − G(: 123) = 416.0087 − 400.8448 = 15.1639 G(2 : 1) = G(: 1) − G(: 12) = 517.691 − 416.0087 = 101.6823 G(23 : 1) = G(2 : 1) − G(2 : 13) = 101.6823 − 32.8427 = 68.8396

Then, the additive elements are:

• rder

resid primary secondary tertiary index :123 1:23 2:13 3:12 12:3 13:2 23:1 123: G 400.85 48.83 32.84 15.16

• 7.83

5.97 68.84 3.51

• Additive Elements Order

% of response variation Resid Primary Secondary Tertiary 10 20 30 40 50 60 70 res

1:23 2:13 3:12 12:3 13:2 23:1 123:

Here:

Here: endogenous factor (age) ≈ biological component

Here: endogenous factor (age) ≈ biological component exogenous factors (space and time) ≈ environmental component

Here: endogenous factor (age) ≈ biological component exogenous factors (space and time) ≈ environmental component so it is important to asses the relative role of these two components in explaining growth performance.

Here: endogenous factor (age) ≈ biological component exogenous factors (space and time) ≈ environmental component so it is important to asses the relative role of these two components in explaining growth performance.

Exogenous (CY,L) 13% Residual 70.52% Endogenous (A) 2.7% Confused 13.78%

5. Open problems: how to quantify the contribution of random ef- fects? Aim: extension of the Whittaker’s approach for defining additive likelihood el- ements associated with both the fixed and random components in the model.

5. Open problems: how to quantify the contribution of random ef- fects? Aim: extension of the Whittaker’s approach for defining additive likelihood el- ements associated with both the fixed and random components in the model. Problems:

5. Open problems: how to quantify the contribution of random ef- fects? Aim: extension of the Whittaker’s approach for defining additive likelihood el- ements associated with both the fixed and random components in the model. Problems:

• difficulty in defining a binary lattice including a random component

5. Open problems: how to quantify the contribution of random ef- fects? Aim: extension of the Whittaker’s approach for defining additive likelihood el- ements associated with both the fixed and random components in the model. Problems:

• difficulty in defining a binary lattice including a random component
• difficulty in defining the “null” model: can a fixed model with only an

intercept be interpreted as the “null” model of a mixed model?

5. Open problems: how to quantify the contribution of random ef- fects? Aim: extension of the Whittaker’s approach for defining additive likelihood el- ements associated with both the fixed and random components in the model. Problems:

• difficulty in defining a binary lattice including a random component
• difficulty in defining the “null” model: can a fixed model with only an

intercept be interpreted as the “null” model of a mixed model?

• How to minimize the influence of the fixed part in quantifying the contri-

bution of the random component to the total variation, and vice-versa?

A (possible) h-likelihood approach: h = ℓ(β; y|ω) + ℓ(α; ω) = L + ℓ(β, φ, α; ω|y) (4) where L is the marginal likelihood and ℓ(β, φ, α; ω|y) = logf1(y|ω, β, φ)f2(ω|α)/

• f1(y|ω, β, φ)f2(ω|α)dω

Using the Laplace approximation for the marginal likelihood, the next step would be to decompose the h-likelihood in such a way to have likelihoods for the fixed component only indirectly influenced by the random effects included in the model, and vice-versa.

References

• Lee, Y. and Nelder, J. A., Hierarchical Generalized Linear Models. Journal
• f the Royal Statistical Society, Series B. 1996,58,619-678;
• Breslow, N. E. and Clayton, D. G., Approximate Inference in General-

ized Linear Mixed Models, Journal of the American Statisticial Associa- tion,1993,88,9-25;

• Lee, Y. and Nelder, J. A., Hierarchical Generalized Linear Models: a syn-

thesis of generalized linear models random-effects models and structured dispersion, Biometrika,2001,88,987-1006;

• Whittaker, J., Model Interpretation from the Additive Elements of the

Likelihood Function, Applied Statistics, 1984, 33,52-64