Stat 598 Presentation on Nonseperable, Stationary Covariance - - PowerPoint PPT Presentation

Stat 598 Presentation on Nonseperable, Stationary Covariance Functions for Space-Time Data

Xiongzhi Chen

Department of Statistics

November 16, 2010

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 1 / 22

Outline

Positive Definite Functions Main Mathematical Results Application to Irish Wind Data Why Such A Correlation Model Other Strategies and Wrong Claims Proofs Acknowledgements

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 2 / 22

Key Component of Spatial-Temporal Modelling

Observations generated from a square-integrable stochastic process Z =

• Z(s; t), (s; t) ∈ Rd × R
• Process Z induces a family of functions

C =

• Cs,t : Rd × R → R| (s; t) ∈ Rd × R
• , where

Cs,t (h; u) = Cov ((Z(s; t) , Z (s + h; t + u)) at each (s; t). Covariance stationarity is equivalent to C = {C (h; u)} Then C (·; ·) is called the covariance function of Z.

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 3 / 22

Positive-Definite (PSD) Functions

A function f : Rd+1 → K is positive definite iff

k

∑

i,j=1

f (wi − wj) aia∗

j ≥ 0

First equivalence f being a covariance function ⇔ f positive definite Second equivalence (Bochner’s Theorem): A continuous function f on Rd+1 is positive definite iff f (w) =

• exp

√ −1 η, w

• dF (η)

(1)

Equivalences and Spectral Representation Proof of Bochner’s Theorem Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 4 / 22

Construction of PSD Functions

Main method: via Laplace transform and Bochner’s theorem. Cressie and Huang (1999) [1] based on closed-form F-inverse of Cω (u) =

• e−ihωC (h; u) dh

(2) where C (h; u) ∈ Cb ∩ L1 and C and Cω both are covariance functions or neither are (3) Obviously C ≡ Cω mod (F) but C might be un-usable.

Completely Monotone Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 5 / 22

Construction of PSD Functions

Main method: via Laplace transform and Bochner’s theorem. Cressie and Huang (1999) [1] based on closed-form F-inverse of Cω (u) =

• e−ihωC (h; u) dh

(2) where C (h; u) ∈ Cb ∩ L1 and C and Cω both are covariance functions or neither are (3) Obviously C ≡ Cω mod (F) but C might be un-usable. Gneiting (2002) [2] (i) avoids explicit F-inverse of Cω and uses L-representation of completely monotone (cm) functions and the Bochner’s theorem; (ii) produces explicit product form of some PSD functions

Completely Monotone Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 5 / 22

Key Mathematical Result

Theorem 2 (Gneiting (2002) [2]): Let k, l ∈ N+, σ2 > 0.Let ϕ (t) , t ≥ 0 be a cm function, and let ψ (t) , t ≥ 0, be a positive function with a cm

• derivative. Then

C (h; u) = σ2 ψ

• u2k/2 ϕ

  h2 ψ

• u2

  , (h; u) ∈ Rk × Rl (4) is a space-time covariance function. The family includes almost all examples given in Cressie and Huang (1999) [1]

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 6 / 22

Key Mathematical Result

Theorem 2 (Gneiting (2002) [2]): Let k, l ∈ N+, σ2 > 0.Let ϕ (t) , t ≥ 0 be a cm function, and let ψ (t) , t ≥ 0, be a positive function with a cm

• derivative. Then

C (h; u) = σ2 ψ

• u2k/2 ϕ

  h2 ψ

• u2

  , (h; u) ∈ Rk × Rl (4) is a space-time covariance function. The family includes almost all examples given in Cressie and Huang (1999) [1] In application, ϕ, ψ can be associated with the data’s spatial and temporal structures, respectively

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 6 / 22

Key Mathematical Result

Theorem 2 (Gneiting (2002) [2]): Let k, l ∈ N+, σ2 > 0.Let ϕ (t) , t ≥ 0 be a cm function, and let ψ (t) , t ≥ 0, be a positive function with a cm

• derivative. Then

C (h; u) = σ2 ψ

• u2k/2 ϕ

  h2 ψ

• u2

  , (h; u) ∈ Rk × Rl (4) is a space-time covariance function. The family includes almost all examples given in Cressie and Huang (1999) [1] In application, ϕ, ψ can be associated with the data’s spatial and temporal structures, respectively Note how the temporal correlation is "scaled" by spatial correlation

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 6 / 22

Examples of Building Functions

ϕ1 (t) = exp (−ctγ) c > 0, γ ∈ (0, 1] ϕ2 (t) =

• 2ν−1Γ (ν)

−1 ct1/2ν Kν

• ct1/2

c > 0, ν > 0 ϕ3 (t) = (1 + ctγ)−ν c > 0, ν > 0, γ ∈ (0, 1] ϕ4 (t) = 2ν exp

• ct1/2 + exp

−ct1/2−ν c > 0, ν > 0 ψ1 (t) = (atα + 1)β a > 0, α ∈ (0, 1], β ∈ [0, 1] ψ1 (t) = ln (atα + b) / ln (b) a > 0, b > 1, α ∈ (0, 1] ψ1 (t) = (atα + b) / (b (atα + 1)) a > 0; b, α ∈ (0, 1] Table 1: Building Functions

Fitted Model Why This Model Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 7 / 22

Correlation Model for Irish Wind Data

For this data set, the estimated correlation model is C (h; u|β) =        ˆ ψ−1

1

(u) =

• .901 |u|1.544 + 1

−1 if h = 0 .968 ˆ ψ1 (u) exp

• −.00134 h

ˆ ψβ/2 (u)

• therwise

(5) where ψ1 and ϕ1 in Table 1 are chosen and h ≤ 450km, |u| ≤ 3 ˆ ψ−1

1

(u) =

• .901 |u|1.544 + 1

−1 , temporal correlation, estimated from data

Table of Functions Why This Model Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 8 / 22

Correlation Model for Irish Wind Data

For this data set, the estimated correlation model is C (h; u|β) =        ˆ ψ−1

1

(u) =

• .901 |u|1.544 + 1

−1 if h = 0 .968 ˆ ψ1 (u) exp

• −.00134 h

ˆ ψβ/2 (u)

• therwise

(5) where ψ1 and ϕ1 in Table 1 are chosen and h ≤ 450km, |u| ≤ 3 ˆ ψ−1

1

(u) =

• .901 |u|1.544 + 1

−1 , temporal correlation, estimated from data ˆ ϕ (h) = .968 exp (−.00134 h) , spatial correlation, borrowed from Haslett and Raftery (1989) ([4])

Table of Functions Why This Model Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 8 / 22

Correlation Model for Irish Wind Data

For this data set, the estimated correlation model is C (h; u|β) =        ˆ ψ−1

1

(u) =

• .901 |u|1.544 + 1

−1 if h = 0 .968 ˆ ψ1 (u) exp

• −.00134 h

ˆ ψβ/2 (u)

• therwise

(5) where ψ1 and ϕ1 in Table 1 are chosen and h ≤ 450km, |u| ≤ 3 ˆ ψ−1

1

(u) =

• .901 |u|1.544 + 1

−1 , temporal correlation, estimated from data ˆ ϕ (h) = .968 exp (−.00134 h) , spatial correlation, borrowed from Haslett and Raftery (1989) ([4]) ˆ β = 0.61, spatial-temporal interaction, estimated from data; γ = 1/2 in ϕ1 was forced

Table of Functions Why This Model Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 8 / 22

Why the Above Covariance Function?

The fitted model is composed of Purely Spatial Correlation with nugget effect C (h; 0|β) = 1 if h = 0; C (h; 0|β) = ˆ ϕ (h) , otherwise. Note ˆ ϕ ∈ C ∞ Some reasons:

Table of Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 9 / 22

Why the Above Covariance Function?

The fitted model is composed of Purely Spatial Correlation with nugget effect C (h; 0|β) = 1 if h = 0; C (h; 0|β) = ˆ ϕ (h) , otherwise. Note ˆ ϕ ∈ C ∞ Purely Temporal Correlation: C (u; 0|β) = ˆ ψ−1

1

(u) . Note ˆ ψ1 / ∈ C ∞ Some reasons:

Table of Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 9 / 22

Why the Above Covariance Function?

The fitted model is composed of Purely Spatial Correlation with nugget effect C (h; 0|β) = 1 if h = 0; C (h; 0|β) = ˆ ϕ (h) , otherwise. Note ˆ ϕ ∈ C ∞ Purely Temporal Correlation: C (u; 0|β) = ˆ ψ−1

1

(u) . Note ˆ ψ1 / ∈ C ∞ Some reasons: Intrinsic: instrument variations and highly irregular wind speeds

Table of Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 9 / 22

Why the Above Covariance Function?

The fitted model is composed of Purely Spatial Correlation with nugget effect C (h; 0|β) = 1 if h = 0; C (h; 0|β) = ˆ ϕ (h) , otherwise. Note ˆ ϕ ∈ C ∞ Purely Temporal Correlation: C (u; 0|β) = ˆ ψ−1

1

(u) . Note ˆ ψ1 / ∈ C ∞ Some reasons: Intrinsic: instrument variations and highly irregular wind speeds Generic: evidence from data (see Graphs)

Table of Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 9 / 22

Why the Above Covariance Function?

The fitted model is composed of Purely Spatial Correlation with nugget effect C (h; 0|β) = 1 if h = 0; C (h; 0|β) = ˆ ϕ (h) , otherwise. Note ˆ ϕ ∈ C ∞ Purely Temporal Correlation: C (u; 0|β) = ˆ ψ−1

1

(u) . Note ˆ ψ1 / ∈ C ∞ Some reasons: Intrinsic: instrument variations and highly irregular wind speeds Generic: evidence from data (see Graphs) Technical: choice of ˆ ϕ fits into the family C (h; u) = σ2

• a u2α + 1

βk/2 exp   − c h2γ

• a u2α + 1

βγ   

Table of Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 9 / 22

Other Strategies and Wrong Claims

All covariance functions in Cressie and Huang (1999) ([1]) and Gneiting (2002) ([2]) are FULLY SYMMETRIC

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 10 / 22

Other Strategies and Wrong Claims

All covariance functions in Cressie and Huang (1999) ([1]) and Gneiting (2002) ([2]) are FULLY SYMMETRIC Lagrangian reference frame by Cox and Isham (1988) ([6]) C (h; u) = EV G (h − Vu) ; (h, u) ∈ R2 × R used to model dynamic environmental and atmospheric processes, where G (r) = λ {D (P1, 1) ∩ D (P2, 1) : P1 − P2 = r} ; P1, P2 ∈ R3 and λ is the Lebesgue measure.

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 10 / 22

Other Strategies and Wrong Claims

All covariance functions in Cressie and Huang (1999) ([1]) and Gneiting (2002) ([2]) are FULLY SYMMETRIC Lagrangian reference frame by Cox and Isham (1988) ([6]) C (h; u) = EV G (h − Vu) ; (h, u) ∈ R2 × R used to model dynamic environmental and atmospheric processes, where G (r) = λ {D (P1, 1) ∩ D (P2, 1) : P1 − P2 = r} ; P1, P2 ∈ R3 and λ is the Lebesgue measure. Perturbation to fully symmetric models

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 10 / 22

Other Strategies and Wrong Claims

All covariance functions in Cressie and Huang (1999) ([1]) and Gneiting (2002) ([2]) are FULLY SYMMETRIC Lagrangian reference frame by Cox and Isham (1988) ([6]) C (h; u) = EV G (h − Vu) ; (h, u) ∈ R2 × R used to model dynamic environmental and atmospheric processes, where G (r) = λ {D (P1, 1) ∩ D (P2, 1) : P1 − P2 = r} ; P1, P2 ∈ R3 and λ is the Lebesgue measure. Perturbation to fully symmetric models False claims of convexity lead to false PSD functions, as in Cressie and Huang 1999 ([1])

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 10 / 22

Generalized Theorem 1 and Proof: I

Theorem 1 (Generalized): A continuous, bounded, symmetric and integrable function C (h; u) ,defined on Rk × Rl,is a covariance function if and only if Cω (u) =

• e−ihωC (h; u) dh,

u ∈ Rl (6) is a covariance function for all most all ω ∈ Rk Digested Proof: The extension is on u ∈ Rl.The arguments are just a combination of Fubini’s, Bochner’s theorems and the Dirichlet integral. (a) C ∈ R1 Rk × Rl ⇒ C ∈ L1 Rk × Rl (whenever k + l ≥ 2) and Cω ∈ L1 Rl ,a.s. ω ∈ Rk. (b) C ∈

• Cb ∩ L1

Rk+l ⇒ Cω ∈ Cac

• Rl

, ∀ω ∈ Rk; (c) C ∈

• Cb ∩ L1

Rk+l and the inversion formula imply that

Inversion Formular Theorem 2 and Proof Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 11 / 22

Generalized Theorem 1 and Proof: II

Digested Proof (continued): C has F-inverse as f (ω, τ) ∝

• e−ihω−iτuC (h; u) dhdu ∝
• e−iτuCω (u) du

(7) where f = dF dλ and F is a signed, bounded, measure on Rk+l such that F << λ and λ is the Lebesgue measure on Rk+l. The last equality is justified by Fubini’s theorem. (d) Now, Bochner’s theorem and Fubini’s together implies the validity of the theorem iff f ≥ 0 on Rk × Rl and f ∈ L1 Rk+l . This completes the proof.

Inversion Formular Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 12 / 22

Theorem 2 and Proof: Multiply Dominating Factor

Theorem 2 (Generalized): Let k and l be nonnegative integers, and let σ2 > 0. Suppose that ϕ (t) , t ≥ 0, is a completely monotone function, and let ψ (t) , t ≥ 0, be a positive function with a completely monotone

• derivative. Then

C (h; u) = σ2 ψ

• u2k/2 ϕ

  h2 ψ

• u2

  , (h; u) ∈ Rk × Rl (8) is a space-time covariance function. Digested Proof: (a) Multiply the target function by dominating integral factor. (Case Restricted) Assume ϕ ∈ R1 Rk and define Ca (h; u) = exp

• −a u2

C (h; u) (9) By Theorem 1 (Generalized), (9) is PSD iff (6) is PSD.

Theorem 1 and Proof Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 13 / 22

Theorem 2 and Proof: Plug in L-Representation of ϕ

(proof continued): (b) Plug the Laplace transform representation of ϕ into the integral. Notice that ϕ (t) =

• [0,∞) exp (−tr) dF (r) = L (F)

(10) and ϕ ∈ R1 Rk , then ϕ is bounded and limt→0 ϕ (t) = 0.Thus F is a bounded and F is right-continuous at 0 (Otherwise, take tn = n, rn = 1/n in (10)). For notational simplicity, let ψu = ψ

• u2

. Then by Fubini theorem, Ca,ω (u) =

• e−ihωCa (h; u) dh

(11) = exp

• −a u2 σ2

ψk/2

u

• (0,∞) dF (r)
• (0,∞) fu,r,ω (h) dh

where fu,r,ω (h) = e−ihω exp

• −
• r h2

/ψu

• Xiongzhi Chen (Department of Statistics)

Stat 598 Presentation November 16, 2010 14 / 22

Theorem 2 and Proof: Obtain L-Representation of Ca,ω

(proof continued): and (c1) that Ca,ω (u) = ϕa,ω

• u2

, u ∈ Rk (12) can be assumed, where ϕa,ω (t) = σ2πk/2 exp (−at)

• (0,∞) exp (−sψ (t)) dGω,a (s) ; t ≥ 0

(13) and Gω,a is a nondecreasing, bounded function. This is just the Laplace transform of the measure induced by composing the measures F by ψ. Since Ca,ω is spherically symmetric and ϕa,ω is cm, Shoenberger’s theorem (Shoenberger 1938 [5]) asserts that Ca,ω (·) is PSD and so is each Ca by Theorem 1. Since lima→0+ Ca = C pointwise and C is continuous at 0, C is PSD.

Theorem 2 and Proof 3 Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 15 / 22

Theorem 2 and Proof: Still by Dominating Factor

(proof continued): (Case General) Given a cm function ϕ (t) , t ≥ 0, then hb (t) = exp (−bt) ϕ (t) , b > 0, t ≥ 0 is still cm (by Tonelli’s theorem and since exp (− (b + r) t) is integrable and F in the L-representation of ϕ is a bounded measure). Further hb is dominated by exp (−bt) , hence hb ∈ L1 (R) and hb ∈ Rk when t = h .Thus Cb (h; u) = σ2 ψ

• u2k/2 exp

 − b h2 ψ

• u2

  ϕ   h2 ψ

• u2

  (14) is PSD by the previous arguments. Again, limb→0+ Cb = C pointwise and C is continuous at 0, C is PSD.

Acknowledgements Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 16 / 22

Details On Equivalences and Spectral Representation

f positive definite ⇒ Gw = (f (wi − wj)) a positive-definite matrix ⇒ a characteristic function (ch.f) hw (t) = exp

• −1

2tGw t

• ⇔

ˆ hw (s) = e−it,shw (t) dt a finite measure. Take a process Z (w) (subtracted its mean function) with ˆ hw as its distribution. when f ∈ L1 Rd+1 , F is absolute continuous and f , F are inverse Fourier transformations (modulo a constant). Based on the inversion formula F(a, b] = lim

n→∞

1 2π

T

−T dt

b

a e−itsds

via Dirichlet integral lima→−∞,b→+∞ b

a

sin x x dx = π Every psd function is equivalent to a double integral with non-negative integrand.

PSD Functions Proof of Theorem1 Proof of Theorem1 (II) Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 17 / 22

Proof of Bochner’s Theorem

Digested Proof (Feller [3], p58): ∑k

i,j=1 f (wi − wj) aia∗ j ≥ 0 ⇒

• f (α − β) ρ (α) ρ (β)dµαdµβ ≥ 0, ∀ρ ∈ L1

Fix x and ε > 0 and put ρ (α) = e−2ε|α|2e2πα−β,x. Do transformation α − β = γ, α + β = δ to obtain e−ε|δ|2dµδ

• e−ε|γ|2f (γ) e2πiγ,xdµγ ≥ 0 =

⇒ fε (γ) = e−ε|γ|2f (γ) ∈ L1 and has non-negative F-inverse. Hence fε has spectral representation and limε→0 fε = f completes the proof by weak and complete convergence.

Covariance Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 18 / 22

Details on Completely Monotone Functions

Theorem(Feller [3], p. 439): ϕ : (0, ∞) → (0, ∞) is said to be completely monotone iff (−1)n ϕn (t) ≥ 0, t > 0, n ≥ 0 iff ϕ (t) =

• [0,∞) exp (−tr) dF (r) = L (F)

with F being a measure on [0, ∞]. Digested Proof (Feller [3], p440). ϕ∗ (s) = ϕ (a − as) = ∑ (−a)n ϕn (a) n! sn = ⇒ ϕa (λ) = ϕ

• a − ae−λ/a =

∑ (−a)n ϕn (a) n! e−nλ/a.And lima→∞ ϕa = ϕ ⇐ ⇒ lima→∞ Fa = F. ϕ is a Laplace transform. Moreover under the above settings, it holds F (x) = lima→∞ ∑n≤ax (−a)n ϕn (a) n! the family of cm functions is closed under multiplication and composition, justified by induction via definition.

Construction of PDS Functions Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 19 / 22

Theorem 2 and Proof: Contour integral of fu,r,ω(h)

(proof continued): (c) Derive the Laplace transform from Cω. Let D = 2diag {r/ψu, · · · , r/ψu}. Notice that D > 0 and fu,r,ω (h) ∝ e−ihω−hDh/2, then rectangle contour integral by Cauchy integral formular shows ˆ fu,r (ω) =

• (0,∞) fu,r,ω (h) dh =

πr ψu −k/2 exp

• −ω2

4r ψu

• and

Cω (u) = σ2 exp

• −a u2

exp

• −ω2

4r ψu

(0,∞)

1 rk/2 dF (r) (15) The continuity of C0 (·) and (15) implies

• (0,∞)

1 rk/2 dF (r) = C0 (0) σ2πk/2 < ∞

Theorem 2 and Proof 4 Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 20 / 22

References

Cressie, N., and Huang, H.-C. (1999), Classes of Nonseparable, Spatiotemporal Stationary Covariance Functions, JASA, 94, 1330-1340 Tilmann Gneiting (2002), Nonseparable, Stationary Covariance Functions for Space-Time Data, JASA, Vol. 97, No. 458 (Jun., 2002), pp. 590-600 Feller, W. (1966), An Introduction to Probability Theory and Its Applications (Vol. II), New York: Wiley John Haslett and Adrian E. Raftery (1989), Space-Time Modelling with Long-Memory Dependence: Assessing Ireland’s Wind Power, JRSS. Ser C (Applied Statistics), Vol. 38, No. 1(1989), pp. 1-50 Schoenberg, I. J. (1938), Metric Spaces and Completely Monotone Functions, Annals of Mathematics, 39, 811-841. Cox, D. R., and Isham, V. (1988), A Simple Spatial-Temporal Model of Rainfall, in Proceedings of the Royal Society of London, Ser. A, 415, 317-328

Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 21 / 22

Acknowledgements

Thank you for being so PATIENT! Irish Wind

Thoerem2 and Proof 5 Xiongzhi Chen (Department of Statistics) Stat 598 Presentation November 16, 2010 22 / 22