BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY Subhashish Chakravarty - - PowerPoint PPT Presentation

BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Subhashish Chakravarty Advisor: Prof. George G. Woodworth Co-Advisor: Dr. Matthew A. Bognar April 24, 2007

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Objective of the thesis

Smooth spatial data using thin-plate splines. Both Gaussian and count data. In the presence of anisotropy. Under Bayesian paradigm using smoothness priors. Achieve model selection concurrently with parameter updates in MCMC.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Example of spatial data

Data from Cressie (1991) Spatial Statistics. From Robena Mine in Greene County, PA. % Coal Ash present at different locations. Data re-oriented so that top of page faces North.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Plot of the data

Figure: Scatter plot of the core values in %coal ash.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Some earlier work done

Wahba (1978). Improper Priors, Spline smoothing and the problem of guarding against model errors in regression, JRSS, ser.B, 40, no.3.

We use her representation of the smooth surface. Smoothness parameter estimated via cross-validation.

Ecker and Gelfand (1999). Bayesian Modeling and Inference for Geometrically Anisotropic Spatial Data, Math. Geol., 31,

• no. 1.

Uses parametric form of the covariance to model anisotropy. Regression is the objective. No model selection done.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Some earlier work done (contd.)

Wood et al (2002) Bayesian mixture of splines for spatially adaptive nonparametric regression, Biometrika, 89, no. 3.

Uses a mixture of splines, each with own smoothing parameter. Uses BIC to decide the number of splines in the mixture.

Brezger and Lang (2006) Generalized structured additive regression based on Bayesian P-splines, Comp.Stat. and Data Analysis, 50.

Uses Generalized additive models. Extended to multicategorical regression models. Model choice based on pointwise credible intervals and DIC.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

What is new

Using smoothness priors to model surface in the presence of anisotropy. Model selection concurrently with parameter updates.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Model

yi = f (xi) + ǫi, i = 1, 2, . . . , n. yi is the response at the ith location xi. Locations xi are points in the plane. f (·) is assumed to be a smooth function. ǫi

i.i.d.

∼ N(0, 1/τe). Smoothness means all partial derivatives of order 2 of f are square integrable.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Penalized least squares

Minimize n

i=1(yi − fi)2 + λJ2,2(f) over the space of smooth

functions. λ > 0 is the smoothness parameter. J2,2(·) is the thin-plate penalty for roughness in 2 dimensions for a function of 2 variables. The solution (exists when the locations are unique), has a linear component and non-linear component (arising from thin plate splines).

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Thin Plate Splines and anisotropy

Wahba’s (1978) representation of the solution(for a given τ) is ˆ f (x) = θ0 + θ1x1 + θ2x2 +

• (1/τ)Z(x).

x = (x1, x2)′. Θ = (θ0, θ1, θ2)′ are parameters with a diffuse Normal prior. Z(·) is a stationary zero mean Gaussian random field. τ is the inverse of the smoothness parameter.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Thin Plate Splines and anisotropy (contd.)

At the observed locations x1, . . . , xn, we express the covariance of Z by V = PXEPX X is the matrix of locations. First column of X has ones. PX = I − X(X′X)+X′. It is orthogonal to X. E(xi, xj) = (1/16π)h′

ijAhij log(h′ ijAhij) is the matrix

constructed from the thin plate spline basis functions.

A is the positive definite anisotropy matrix. hij = (xi − xj).

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Thin Plate Splines and anisotropy (contd.)

We can thus rewrite the solution as XΘ + TAΓ

columns of TA are the orthonormal eigenvectors of V scaled by square-root of the eigenvalues. Columns arranged according to descending eigenvalue. Γ ∼ N(0, (1/τ)I).

We include only a maximum of Kmax eigenvectors. Substituting the form the solution into the data model we have y = XΘ + TAΓ+ ǫ

ǫ ∼ N(0, (1/τe)In)

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Priors

π(Θ, A, Γ, τ, τe, αe, βe, K) = π(A)π(K)π(αe)π(βe)π(τ)π(Θ) .π(τe|αe, βe)π(Γ|K, τ)

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Priors (contd.)

Θ has a diffuse conjugate N(0, (1/0.001)I) prior. A has a W (2, Σ)) prior. K has a Poisson(5) prior. τ, conjugate Gamma(1, βτ) prior. τe conditional on αe and βe, has a conjugate Gamma(αe, βe) prior. Γ conditional on K and τ has a conjugate N(0, (1/τ)I) prior. αe, βe each have a LN(0, 1) hyperprior.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Partially analytic RJ proposal for model order K

We consider a nested model structure. Changing K, changes the dimension of Γ. Consider three types of moves: birth, death, stay. Acceptance probability for birth move is α = min

• (τ.2π)1/2[τeδk+1 + τ]−1/2.

5 k + 1.min((k + 1)/5, 1) min(5/(k + 1), 1), 1

• δk+1 is (k + 1)th eigenvalue of V.

See thesis for details.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Partially analytic RJ proposal for model order (contd.)

Acceptance probability for death move is α = min

• (τ.2π)−1/2[τeδk + τ]1/2.k

5.min(5/k, 1) min(k/5, 1), 1

• δk is kth eigenvalue of V.

See thesis for details.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Posterior Sampling algorithm

Sample K using RJMCMC. Conditional on K, update Γ from posterior. Update Θ, τ, τe from the posteriors. Update αe and βe using a random walk metropolis algorithm.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Data

Transformed Coal ash data from Cressie. Added quadratic surface 0.03x2

1 + 0.01x2 2 + 0.05x1x2.

Prior mean of τ = 0.01. Kmax = 20. Mean of anisotropy matrix Σ = 0.04 0.012 0.012 0.021

• Subhashish Chakravarty

BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Observed surface

Figure: Surface plot of the observed values.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Fitted Surface

Figure: Surface plot of the fitted values (chain 1).

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Trace Plot of K

Figure: Trace plot of K (chain 1).

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Model

Response y, is distributed as Poisson(exp{η}). η is the surface to be smoothed. Introduce latent variables z = η + ǫ

ǫ has a Normal distribution with precision (1/τe)I.

Use the earlier representation of the surface η = XΘ + TAΓ The distribution of y conditional on z is independent of the parameters of the surface.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Prior and posterior distributions

The surface parameters have the same priors. Conditional on z the posterior distributions will remain the same. Need to consider only the conditional posterior distribution of z. The conditional posterior distribution of z is given by π(z|y, . . .) ∝ exp(− n

i=1 exp(zi) + n i=1 ziyi).

. exp{−1

2τe(z − XΘ − TAΓ)}

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Posterior simulation of z

We have used Langevin-Hastings Metropolis algorithm to propose a new value for z. The new value z∗ is proposed from a multivariate normal with mean ξ(z) = z + h 2∇ log(π(z| . . .)) = z + h 2{y − exp(z) − τe(z − Xθ − TAΓ)} and variance hI.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Posterior simulation of z (contd.)

Acceptance probability of the move is given by min{1, π(z∗|y, . . .) π(z|y, . . .) .exp(− 1

2hz∗ − ξ(z)2)

exp(− 1

2hz − ξ(z∗)2)}

h is tuned so that the acceptance probability is close to 57%.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Simulated Data

Simulated data from Poisson(10 + 0.5N(0, 1) + 0.03x2

1 + 0.01x2 2 + 0.005x1x2)

N(0, 1) standard normal error.

Prior mean of τ = 0.01. Kmax = 20. Mean of anisotropy matrix Σ = 0.04 0.012 0.012 0.021

• Subhashish Chakravarty

BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Scatter plot of observed data

Figure: Scatter plot of the observed values.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Fitted surface plot

Figure: Fitted surface plot (chain 1).

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Trace plot of K

Figure: Trace plot of K (chain 1).

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Future Research

Thorough sensitivity analysis of K. Sampling strategy for A. Prior mean of smoothness parameter τ. Extending to higher dimensions, maybe using other basis functions. Incorporate constraints in addition to smoothness on the surface.

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

. . . hope this is the beginning...

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY

Acknowledgments

Everybody in this room....

Subhashish Chakravarty BAYESIAN SURFACE SMOOTHING UNDER ANISOTROPY