BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY Subhashish Chakravarty Advisor: Prof. George G. Woodworth Co-Advisor: Dr. Matthew A. Bognar April 24, 2007 Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Objective of the thesis Smooth spatial data using thin-plate splines. Both Gaussian and count data. In the presence of anisotropy. Under Bayesian paradigm using smoothness priors. Achieve model selection concurrently with parameter updates in MCMC. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Example of spatial data Data from Cressie (1991) Spatial Statistics . From Robena Mine in Greene County, PA. % Coal Ash present at different locations. Data re-oriented so that top of page faces North. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Plot of the data Figure: Scatter plot of the core values in %coal ash. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Some earlier work done Wahba (1978). Improper Priors, Spline smoothing and the problem of guarding against model errors in regression , JRSS, ser.B, 40 , no.3. We use her representation of the smooth surface. Smoothness parameter estimated via cross-validation. Ecker and Gelfand (1999). Bayesian Modeling and Inference for Geometrically Anisotropic Spatial Data , Math. Geol., 31 , no. 1. Uses parametric form of the covariance to model anisotropy. Regression is the objective. No model selection done. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Some earlier work done (contd.) Wood et al (2002) Bayesian mixture of splines for spatially adaptive nonparametric regression , Biometrika, 89 , no. 3. Uses a mixture of splines, each with own smoothing parameter. Uses BIC to decide the number of splines in the mixture. Brezger and Lang (2006) Generalized structured additive regression based on Bayesian P-splines , Comp.Stat. and Data Analysis, 50 . Uses Generalized additive models. Extended to multicategorical regression models. Model choice based on pointwise credible intervals and DIC. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
What is new Using smoothness priors to model surface in the presence of anisotropy. Model selection concurrently with parameter updates. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Model y i = f ( x i ) + ǫ i , i = 1 , 2 , . . . , n . y i is the response at the i th location x i . Locations x i are points in the plane. f ( · ) is assumed to be a smooth function. i . i . d . ǫ i ∼ N (0 , 1 /τ e ) . Smoothness means all partial derivatives of order 2 of f are square integrable. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Penalized least squares i =1 ( y i − f i ) 2 + λ J 2 , 2 ( f ) over the space of smooth Minimize � n functions. λ > 0 is the smoothness parameter. J 2 , 2 ( · ) is the thin-plate penalty for roughness in 2 dimensions for a function of 2 variables. The solution (exists when the locations are unique), has a linear component and non-linear component (arising from thin plate splines). Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Thin Plate Splines and anisotropy Wahba’s (1978) representation of the solution(for a given τ ) is ˆ � f ( x ) = θ 0 + θ 1 x 1 + θ 2 x 2 + (1 /τ ) Z ( x ) . x = ( x 1 , x 2 ) ′ . Θ = ( θ 0 , θ 1 , θ 2 ) ′ are parameters with a diffuse Normal prior. Z ( · ) is a stationary zero mean Gaussian random field. τ is the inverse of the smoothness parameter. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Thin Plate Splines and anisotropy (contd.) At the observed locations x 1 , . . . , x n , we express the covariance of Z by V = P X EP X X is the matrix of locations. First column of X has ones. P X = I − X ( X ′ X ) + X ′ . It is orthogonal to X . E ( x i , x j ) = (1 / 16 π ) h ′ ij Ah ij log( h ′ ij Ah ij ) is the matrix constructed from the thin plate spline basis functions. A is the positive definite anisotropy matrix. h ij = ( x i − x j ). Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Thin Plate Splines and anisotropy (contd.) We can thus rewrite the solution as XΘ + T A Γ columns of T A are the orthonormal eigenvectors of V scaled by square-root of the eigenvalues. Columns arranged according to descending eigenvalue. Γ ∼ N ( 0 , (1 /τ ) I ). We include only a maximum of K max eigenvectors. Substituting the form the solution into the data model we have y = XΘ + T A Γ + ǫ ǫ ∼ N ( 0 , (1 /τ e ) I n ) Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Priors π ( Θ , A , Γ , τ, τ e , α e , β e , K ) = π ( A ) π ( K ) π ( α e ) π ( β e ) π ( τ ) π ( Θ ) .π ( τ e | α e , β e ) π ( Γ | K , τ ) Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Priors (contd.) Θ has a diffuse conjugate N ( 0 , (1 / 0 . 001) I ) prior. A has a W (2 , Σ)) prior. K has a Poisson (5) prior. τ , conjugate Gamma (1 , β τ ) prior. τ e conditional on α e and β e , has a conjugate Gamma ( α e , β e ) prior. Γ conditional on K and τ has a conjugate N ( 0 , (1 /τ ) I ) prior. α e , β e each have a LN (0 , 1) hyperprior. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Partially analytic RJ proposal for model order K We consider a nested model structure. Changing K , changes the dimension of Γ . Consider three types of moves: birth , death , stay . Acceptance probability for birth move is � � k + 1 . min(( k + 1) / 5 , 1) 5 ( τ. 2 π ) 1 / 2 [ τ e δ k +1 + τ ] − 1 / 2 . α = min min(5 / ( k + 1) , 1) , 1 δ k +1 is ( k + 1) th eigenvalue of V . See thesis for details. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Partially analytic RJ proposal for model order (contd.) Acceptance probability for death move is � ( τ. 2 π ) − 1 / 2 [ τ e δ k + τ ] 1 / 2 . k 5 . min(5 / k , 1) � α = min min( k / 5 , 1) , 1 δ k is k th eigenvalue of V . See thesis for details. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Posterior Sampling algorithm Sample K using RJMCMC. Conditional on K , update Γ from posterior. Update Θ , τ , τ e from the posteriors. Update α e and β e using a random walk metropolis algorithm. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Data Transformed Coal ash data from Cressie. Added quadratic surface 0 . 03 x 2 1 + 0 . 01 x 2 2 + 0 . 05 x 1 x 2 . Prior mean of τ = 0.01. K max = 20. Mean of anisotropy matrix � 0 . 04 � 0 . 012 Σ = 0 . 012 0 . 021 Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Observed surface Figure: Surface plot of the observed values. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Fitted Surface Figure: Surface plot of the fitted values (chain 1). Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Trace Plot of K Figure: Trace plot of K (chain 1). Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Model Response y , is distributed as Poisson(exp { η } ). η is the surface to be smoothed. Introduce latent variables z = η + ǫ ǫ has a Normal distribution with precision (1 /τ e )I . Use the earlier representation of the surface η = XΘ + T A Γ The distribution of y conditional on z is independent of the parameters of the surface. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Prior and posterior distributions The surface parameters have the same priors. Conditional on z the posterior distributions will remain the same. Need to consider only the conditional posterior distribution of z . The conditional posterior distribution of z is given by π ( z | y , . . . ) ∝ exp( − � n i =1 exp( z i ) + � n i =1 z i y i ) . . exp {− 1 2 τ e ( z − XΘ − T A Γ ) } Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Posterior simulation of z We have used Langevin-Hastings Metropolis algorithm to propose a new value for z . The new value z ∗ is proposed from a multivariate normal with mean z + h ξ ( z ) = 2 ∇ log( π ( z | . . . )) z + h = 2 { y − exp( z ) − τ e ( z − X θ − T A Γ ) } and variance h I . Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Posterior simulation of z (contd.) Acceptance probability of the move is given by 2 h � z ∗ − ξ ( z ) � 2 ) π ( z | y , . . . ) . exp( − 1 min { 1 , π ( z ∗ | y , . . . ) 2 h � z − ξ ( z ∗ ) � 2 ) } exp( − 1 h is tuned so that the acceptance probability is close to 57%. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Simulated Data Simulated data from Poisson(10 + 0 . 5 N (0 , 1) + 0 . 03 x 2 1 + 0 . 01 x 2 2 + 0 . 005 x 1 x 2 ) N (0 , 1) standard normal error. Prior mean of τ = 0.01. K max = 20. Mean of anisotropy matrix � 0 . 04 0 . 012 � Σ = 0 . 012 0 . 021 Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Scatter plot of observed data Figure: Scatter plot of the observed values. Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Fitted surface plot Figure: Fitted surface plot (chain 1). Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Trace plot of K Figure: Trace plot of K (chain 1). Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY
Recommend
More recommend
