Spatial covariance-robust minimax prediction based on experimental - - PowerPoint PPT Presentation

Spatial covariance-robust minimax prediction based on experimental design ideas

Gunter Spoeck

gunter.spoeck@uni-klu.ac.at

Department of Statistics, University of Klagenfurt, Austria

Overview

Bayesian linear kriging. Covariance-robust minimax kriging

Spatial covariance-robust minimax prediction based on experimental design ideas – p.1

Bayesian linear kriging

0.1 The model

Gaussian model for the random field {Y (x)|x ∈ X}. Covariance function C(x1, x2) assumed to be known.

Y (x) = f(x)T β + ǫ(x)

E(Y (x)|β)

= f(x)T β

cov(Y (x1), Y (x2)|β)

= C(x1, x2)

Prior knowledge about the trend parameter vector β: A priori distribution for β is Gaussian with E(β) = µ and cov(β) = Φ known.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.2

0.2 The predictor

Known from Gaussian distribution theory: predictive a posteriori mean

ˆ YBK(x0) = f(x0)T ˆ βBK + cT

0 K−1(Ydat − Fˆ

βBK).

The a posteriori mean of the trend parameter vector

β: ˆ βBK = (FT K−1F + Φ−1)−1 (FT K−1Ydat + Φ−1µ)

Spatial covariance-robust minimax prediction based on experimental design ideas – p.3

0.3 The quality of prediction

The Total Mean Squared Error of Prediction: TMSEP( ˆ

YBK(x0)) =

E((Y (x0) − ˆ

YBK(x0))2) = C(x0, x0) + f(x0)TΦf(x0)− kT (K + FΦFT )−1k ,

where

k = (c0 + FΦf(x0)).

TMSEP( ˆ

YBK(x0)) ≤ TMSEP(Universal Kriging)

Spatial covariance-robust minimax prediction based on experimental design ideas – p.4

0.4 Problems with kriging

The covariance function cov(Y (x1), Y (x2)|β) = C(x1, x2) is assumed to be known exactly. But generally the covariance function is just an estimate and therefore always is uncertain. Stein (1999): kriging predictor=empirical BLUP . Pilz et. al. (1997): kriging predictor=plug-in predictor. Consequence: Kriging plug-in predictor is non linear. Christensen (1991): Kriging TMSEP underestimates the true unknown TMSEP of the plug-in kriging predictor.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.5

0.5 Solutions

Make use of the Bayesian paradigm and specify prior distributions over a space of covariance functions. How should one get prior knowledge about the covariance function? What are noninformative priors for covariance functions? Make use of the minimax principle, specify a class of plausible covariance functions and calculate a predictor in such a way that the maximum possible TMSEP becomes a minimum (Spöck, 1997, 2005).

Spatial covariance-robust minimax prediction based on experimental design ideas – p.6

1 Covariance Functions

The Matern class of covariance functions:

KM

θ,σ2(h = ||x1 − x2||) = σ2

1 2θ2−1Γ(θ2)( h θ1 )θ2Jθ2( h θ1 ),

where Jθ2 is the modified Bessel function of order θ2.

θ1 > 0 controlling the range of correlation. θ2 > 0 a smoothness parameter (differentiability).

Exponential model (θ2 = 0.5), KE

θ,σ2(h) = σ2exp(− h θ1).

Gaussian model (θ2 → ∞), KG

θ,σ2(h) = σ2exp(−h2 θ2

1 ).

Spatial covariance-robust minimax prediction based on experimental design ideas – p.7

Covariance-robust minimax kriging

2 Model

Y (x) = f(x)Tβ + ǫ(x), x ∈ X ⊂ Rm

E(Y (x)|β, ν)

= f(x)Tβ

cov(Y (x1), Y (x2)|β, ν)

= Cν(x1, x2) ∈ C, ν ∈ Θ , compact

E(β|ν)

= µ

cov(β|ν)

= Φ.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.8

3 Characterization of the CRMKP

Restriction to linear affine predictors

Da = {ˆ Y = wT Ydat + w0; w ∈ Rn, w0 ∈ R}

The TMSEPs for different predictors ˆ

Y ∈ Da:

TMSEP(Cν; ˆ

Y ) = EβEY0|β,ν(( ˆ Y − Y (x0))2) = = Cν(x0, x0) + f(x0)T Φf(x0) − 2wT (c0,ν + FΦf(x0)) + wT(Kν + FΦFT )w + {µT (f(x0) − FT w) − w0}2.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.9

In the squared marginal bias

{µT (f(x0) − FT w) − w0}2

the covariance function does not appear. Consequence: ˆ

Y M(x0) may be sought in the class of

all marginally unbiased predictors

D = {ˆ Y = wT Ydat + µT (f(x0) − FTw)|w ∈ Rn}.

It is characterized by the relationship

sup

Cν∈C

TMSEP(Cν; ˆ

Y M(x0)) = inf

ˆ Y ∈D

sup

Cν∈C

TMSEP(Cν; ˆ

Y ).

Spatial covariance-robust minimax prediction based on experimental design ideas – p.10

The definition of the minimax kriging predictor necessitates that the infima and suprema exist. We therefore suppose that the parametrization of the covariance function is continuous on the compact parameter set Θ, i.e. limν→ν0 Cν(x1, x2) = Cν0(x1, x2) for all x1, x2 ∈ X. Under this assumption TMSEP(Cν; ˆ

Y ), interpreted as

a function of ν, is for every fixed ˆ

Y ∈ D a continuous

function with compact domain Θ. Lemma 6.5 of Pilz (1991) then shows that

sup

Cν∈C

TMSEP(Cν; ˆ

Y ) = sup

ξ∈Ξ

• Θ

TMSEP(Cν; ˆ

Y )ξ(dν),

Spatial covariance-robust minimax prediction based on experimental design ideas – p.11

where Ξ is the set of all probability measures ξ, that are defined on the σ-algebra of the Borel sets of Θ. The equivalent new minimax problem now reads

sup

Cν∈C

TMSEP(Cν; ˆ

Y M(x0)) = inf

ˆ Y ∈D

sup

ξ∈Ξ

• Θ

TMSEP(Cν; ˆ

Y )ξ(dν),

This minimax problem may now be solved by interchanging minimization and maximization.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.12

Reason: Minimax theorem of Sion (1958) The set of so-called average covariance matrices

M(ξ) =

• Θ
• Cν(x0, x0)

cT

0,ν

c0,ν Kν

• ξ(dν) =
• Cξ(x0, x0)

cT

0,ξ

c0,ξ Kξ

• is convex and compact. The convexity and

compactness of this set follows from the continuity in ν of the covariance functions Cν(x1, x2) directly from Lemma 5.1.8 in Bandemer et al. (1978) by interpreting the average covariance matrices as information matrices from experimental design.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.13

The minimax theorem of Sion (1958) applies since

• Θ

TMSEP(Cν; ˆ

Y )ξ(dν) = = Cξ(x0, x0) + f(x0)TΦf(x0) − 2wT (c0,ξ + FΦf(x0)) + wT (Kξ + FΦFT)w

is a continuous and concave function in

M(ξ) ∈ M(Ξ) and a convex function in w ∈ Rn.

The average covariance matrix M(ξ) sought in the minimax problem and weight vector w ∈ Rn are saddle points of the above integrated TMSEP .

Spatial covariance-robust minimax prediction based on experimental design ideas – p.14

Interchanging supremum and infimum we see that the integrated TMSEP becomes minimal if and only if we insert for ˆ

Y the Bayes kriging predictor ˆ Y BK

M(ξ)(x0) = f(x0)T ˆ

βBK

M(ξ) + cT 0,ξK−1 ξ (Ydat − Fˆ

βBK

M(ξ))

ˆ βBK

M(ξ) = (FT K−1 ξ F + Φ−1)−1(FTK−1 ξ Ydat + Φ−1µ).

Thus, the minimax predictor is characterized as that Bayesian kriging predictor ˆ

Y BK

M(ξ0) that maximizes

TMSEP(M(ξ); ˆ

Y BK

M(ξ)) = Cξ(x0, x0) + f(x0)T Φf(x0) −

− (c0,ξ + FΦf(x0))T(Kξ + FΦFT )−1(c0,ξ + FΦf(x0)).

Spatial covariance-robust minimax prediction based on experimental design ideas – p.15

4 The Equivalence of the Minimax Problem to an Experimental Design Problem

Instead of maximizing the Bayes risk in M(ξ) we consider here the equivalent problem of minimization

• f the reciprocal

TMSEP(M(ξ); ˆ

Y BK

M(ξ))−1 → min ξ∈Ξ .

Spatial covariance-robust minimax prediction based on experimental design ideas – p.16

If we apply the block-inversion rule to the regular, blocked matrix, in the following called average total covariance matrix,

Mb(ξ) =

• Cξ(x0, x0) + f(x0)TΦf(x0)

(c0,ξ + FΦf(x0))T c0,ξ + FΦf(x0) Kξ + FΦFT

• ,

then the first block in the inverse Mb(ξ)−1 is given by TMSEP(M(ξ); ˆ

Y BK

M(ξ))−1,

the reciprocal of the integrated Bayes kriging TMSEP .

Spatial covariance-robust minimax prediction based on experimental design ideas – p.17

Multiplication of the matrix Mb(ξ)−1 with the unit vector cb = (1, 0, 0, . . . , 0)T ∈ Rn+1 in the quadratic form cT

b Mb(ξ)−1cb thus results exactly in the

reciprokal TMSEP to be minimized. Defining Mb(Ξ) = {Mb(ξ), ξ ∈ Ξ}, Ub = cbcT

b and

Ψ(ξ) = Ψ(Mb(ξ)) = tr(UbMb(ξ)−1)

with tr(.) the trace operator, the minimax predictor may thus be determined by minimizing the functional

Ψ(.) in Mb(ξ). Ψ(Mb(ξ)) is in form equivalent to a design functional.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.18

Properties of the functional

Ψ(Mb(ξ)) = tr(UbMb(ξ)−1)

to be minimized are analogous to design functionals: From the compactness and convexity of the set

M(Ξ) it follows that also the set Mb(Ξ) is compact

and convex. Analogiously to the proof of Theorem 12.2 in Pilz (1991) it may be shown that the above functional is convex on the set Mb(Ξ). Furthermore it may be shown that it is continuous with respect to the usual Euclidean metric.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.19

Lemma 11.6 and 11.7 in Pilz (1991) show that for all pairs (ξ, ¯

ξ) ∈ Ξ × Ξ the directional derivatives ∆Ψ(ξ, ¯ ξ) = lim

α↓0

Ψ((1 − α)ξ + α¯ ξ) − Ψ(ξ) α

exist, are given by

∆Ψ(ξ, ¯ ξ) = Ψ(ξ) − tr(UbMb(ξ)−1Mb(¯ ξ)Mb(ξ)−1).

and attain their minimum in direction of a probability measure ¯

ξν with one-point support ν ∈ Θ.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.20

5 Calculation of the CRMKP

The directional derivative plays an important role in the construction of algorithms for the minimization of the convex functional Ψ(.). As a consequence of the analogy of this functional to design functionals known from experimental design gradient descent algorithms may be used as they are known from this theory. Inversion of the Matrix Mb(ξ) can thereby be circumvented by means of update formulas given in Pilz (1991) and based only on matrix multiplications.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.21

6 Example

Data set considered: 591 Caesium137 measurements in the region of Gomel, Belarus, 10 years after the Tchernobyl accident. The data set has been log-transformed in order to follow a Gaussian distribution. By means of the empirical variogram estimator and weighted least squares a convex combination of a Gaussian and an Exponential covariance function model has been fit to the data.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.22

In order to get insight into the uncertainty of the covariance function Gaussian random fields have been simulated at exactly the data locations and for every simulation the covariance function has been reestimated (parametric bootstrap). The covariance function used to get the simulations has been exactly the one estimated from the data. After visual inspection of the simulated covariance functions a belt of plausible covariance functions has been defined by means of restricting the parameter space of the convex combination of the two covariance types.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.23

This has been done by restricting a weighted Euclidean distance of the parametrized convex combinations of the two used covariance function types from the “central” estimated covariance function to be bounded. The covariance-robust minimax kriging predictor then has been calculated by means of an algorithm known from experimental design searching for continuous designs ξ ∈ Ξ minimizing the reziprocal of the TMSEP , Ψ(Mb(ξ)).

Spatial covariance-robust minimax prediction based on experimental design ideas – p.24

20 40 60 80 100 120 140 160 180 200 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Figure 1: Semivariogram estimates simulated by means of a parametric bootstrap

Spatial covariance-robust minimax prediction based on experimental design ideas – p.25

−150 −100 −50 50 100 150 −150 −100 −50 50 100 150 −2 −1.5 −1 −0.5 0.5 1 1.5 2 2.5 3

Figure 2: Ordinary minimax kriging predictions for the log-transformed Gomel data

Spatial covariance-robust minimax prediction based on experimental design ideas – p.26

−150 −100 −50 50 100 150 −150 −100 −50 50 100 150 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6

Figure 3: Square root of the minimax risk for the log-transformed Gomel data

Spatial covariance-robust minimax prediction based on experimental design ideas – p.27

−150 −100 −50 50 100 150 −150 −100 −50 50 100 150 −3 −2 −1 1 2 3

Figure 4: Ordinary kriging predictions for the log- transformed Gomel data

Spatial covariance-robust minimax prediction based on experimental design ideas – p.28

−150 −100 −50 50 100 150 −150 −100 −50 50 100 150 0.2 0.4 0.6 0.8 1 1.2 1.4

Figure 5: Ordinary kriging standard deviations for the log-transformed Gomel data

Spatial covariance-robust minimax prediction based on experimental design ideas – p.29

Figure 6: The semivariograms of the minimax pre- dictors (red)

Spatial covariance-robust minimax prediction based on experimental design ideas – p.30

7 Conclusion

The covariance-robust minimax kriging predictor is a worst case Bayesian kriging predictor. For the calculation of the covariance-robust minimax kriging predictor algorithms based on directional derivatives and borrowed from experimental design theory may be used. Prior knowledge on the belt of plausible covariance functions may be gained by means of simulations (bootstrapping) from the sampling distribution of the covariance estimator.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.31

It is visible from Fig. 6 that one seems to be minimax in his decision if one uses kriging with a priori minimum plausible range, maximum plausible nugget effect and maximum plausible sill. Minimum plausible range has been given preference to maximum plausible sill in the presented example.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.32

• Acknowledgement. This work was partially funded by the

European Commission, under the Sixth Framework Programme, by the Contract N. 033811 with DG INFSO, action Line IST-2005-2.5.12 ICT for Environmental Risk

• Management. The views expressed herein are those of

the authors and are not necessarily those of the European Commission.

Spatial covariance-robust minimax prediction based on experimental design ideas – p.33