Approach 5: Variance-Stabilizing Transformation For certain - PowerPoint PPT Presentation

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Approach 5: Variance-Stabilizing Transformation For certain distributions of Y , the variance is a known function of the mean, and consequently a known function h ( Y ) has an approximately constant variance. For example: Distribution Mean-variance Transformation h ( Y ) √ σ 2 = µ Poisson Y sin − 1 √ σ 2 ∝ µ (1 − µ ) Binomial Y σ 2 ∝ µ 2 lognormal log Y 1 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Empirically, transformation often was found also to make the dependence linear. So for example counted data Y , where the Poisson distribution might be relevant, might be modeled as �� � � = x T E Y j � x j j β , � � �� � = σ 2 . var Y j � x j � Remarks We may not be so lucky! Even if we are, β is hard to interpret. 2 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Approach 6: Box-Cox Transformation Let the data determine the transformation in a parametric family h ( Y , λ ). Box and Cox used the power family: Y λ − 1  λ � = 0  h ( Y , λ ) = λ log Y λ = 0 .  � j β , σ 2 � Assume linearity and normality: h ( Y j , λ ) ∼ N x T . Estimate λ, β , and σ 2 by ML. 3 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Remarks Same issues as before. One transformation is assumed to achieve linearity, normality, and constant variance. β is hard to interpret. 4 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Approach 7: Transform Both Sides (TBS) Suppose we have scientific reasons for a model f ( x , β ). Assume that, conditionally on x j , h ( Y j , λ ) = h { f ( x j , β ) , λ } + e j The deviations e 1 , e 2 , . . . , e n satisfy the standard assumptions: constant variance σ 2 ; independence and normality. Estimate β , λ , and σ 2 by ML. 5 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Still requires a strong assumption. Still difficult to interpret. But: if h is smooth and invertible and σ is small relative to f ( x , β ), we can use a linear approximation: Y j = h − 1 [ h { f ( x j , β ) , λ } + e j , λ ] 1 ≈ f ( x j , β ) + h y { f ( x j , β ) , λ } × e j where h y ( y , λ ) = ∂ h ( y , λ ) . ∂ y 6 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response So to this order of approximation, E ( Y j | x j ) ≈ f ( x j , β ) and σ 2 var ( Y j | x j ) ≈ h y { f ( x j , β ) , λ } 2 σ 2 ≈ h y { E ( Y j ) , λ } 2 . For the Box-Cox family, h y ( y , λ ) = y λ − 1 , so var ( Y j | x j ) ≈ σ 2 f ( x j , β ) 2(1 − λ ) ≈ σ 2 E ( Y j | x j ) 2(1 − λ ) . 7 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response So: normality and constant variance on the transformed scale are (approximately) equivalent to normality and non-constant variance on the original scale, with the variance given by a certain function of the mean. Henceforth, we do not consider transformations. Instead, we model variances, and sometimes distributions, explicitly. 8 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Approach 8: ML for an Assumed Model E.g. Y is a count, which we could model as Poisson-distributed, with a model for the mean: E( Y | x ) = f ( x , β ) . This implies a model for the variance, var( Y | x ) = E( Y | x ) = f ( x , β ) but we do not need it in likelihood-based inference. 9 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Approach 9: Generalized Least Squares (GLS) Assume E( Y | x ) = f ( x , β ) , var( Y | x ) = σ 2 g ( β , θ , x ) 2 . Here: β is, as before, the vector of parameters in the mean function; θ is a possible additional parameter in the structure of the variance function; σ 2 is a possible over-dispersion parameter. 10 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Ad hoc estimation scheme: Get initial estimate of β using OLS; i Get initial estimate of θ , if needed, and construct initial ii estimated weights 1 w j = ˆ � 2 � β OLS , ˆ ˆ g θ , x iii Re-estimate β using WLS, treating ˆ w j as fixed: solve the estimating equation n � w j { Y j − f ( x j , β ) } f β ( x j , β ) = 0 . ˆ j =1 11 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Remarks In step iii, equivalently, ˆ β WLS minimizes n w j { Y j − f ( x j , β ) } 2 . � S ˆ w ( β ) = ˆ j =1 As noted before, this is a computational alternative to solving the estimating equation, but is not generally preferred. The estimating-equation approach may be extended to cases where no such objective function exists. Could repeat step ii with ˆ β WLS instead of ˆ β OLS , then repeat step iii. We could also iterate, perhaps to convergence. 12 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Approach 10: ML Same moment structure as in Approach 9: E( Y | x ) = f ( x , β ) , var( Y | x ) = σ 2 g ( β , θ , x ) 2 . Add an assumption about the distribution of Y | x : Y = f ( x , β ) + σ g ( β , θ , x ) Z where Z has mean 0 and variance 1, such as N (0 , 1), and use likelihood methods. When g ( · ) involves β , the ML estimating equation for β is not the same as in GLS. 13 / 14

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Comments on Approaches 5–10 Approaches 5–7 require (lucky) transformation, and the parameters can be hard for interpretation. Approaches 8–10 are the most interesting. Depending on the form of the functions f ( · ) and g ( · ), these approaches will sometimes yield the same estimating equations. 14 / 14