Approximation of Stochastic Partial Differential Equations by a - - PowerPoint PPT Presentation

Approximation of Stochastic Partial Differential Equations by a Kernel-based Collocation Method

Qi Ye

Department of Applied Mathematics Illinois Institute of Technology Joint work with Prof. I. Cialenco and Prof. G. E. Fasshauer

February 2012

qye3@iit.edu MCQMC 2012 February 2012

Introduction

Outline

1

Introduction

2

Background

3

Kernel-based Collocation Methods

4

Numerical Examples

5

Acknowledgments

qye3@iit.edu MCQMC 2012 February 2012

Introduction

Meshfree Methods

Stochastic Analysis Statistical Learning

qye3@iit.edu MCQMC 2012 February 2012

Introduction Books

Monographs

j

jpg jpg

g

qye3@iit.edu MCQMC 2012 February 2012

Background

Outline

1

Introduction

2

Background

3

Kernel-based Collocation Methods

4

Numerical Examples

5

Acknowledgments

qye3@iit.edu MCQMC 2012 February 2012

Background The method in a nutshell

Parabolic Stochastic Equations = ⇒ Elliptic Stochastic Equations Here, we only consider the simple high-dimensional elliptic SPDE

• ∆u = f + ξ,

in D ⊂ Rd, u = 0,

• n ∂D,

where ∆ = d

j=1 ∂2 ∂x2

j is the Laplacian operator,

suppose that u ∈ Sobolev space Hm(D) with m > 2 + d/2 a.s., f : D → R is a deterministic function, ξ : D × Ωξ → R is a Gaussian field with mean zero and covariance kernel W : D × D → R defined on a probability space (Ωξ, Fξ, Pξ), i.e., E(ξx) = 0, Cov(ξx, ξy) = W(x, y).

qye3@iit.edu MCQMC 2012 February 2012

Background The method in a nutshell

The proposed numerical method for solving a parabolic SPDE can be described as follows:

1

We choose a reproducing kernel K : D × D → R whose reproducing kernel Hilbert space HK(D) is embedded into Hm(D).

Noise Covariance Kernel W → Smoothness of Exact Solution u ↓ ց ↓ Convergent Rates ← Reproducing Kernel K

qye3@iit.edu MCQMC 2012 February 2012

Background The method in a nutshell 2

We simulate the Gaussian field with covariance structure W at a finite collection of predetermined collocation points XD := {x1, · · · , xN} ⊂ D, X∂D := {xN+1, · · · , xN+M} ⊂ ∂D, i.e., yj := f(xj) + ξxj, j = 1, · · · , N, yN+j := 0, j = 1, · · · , M, and ξ := (ξx1, · · · , ξxN) ∼ N (0, W) , W :=

• W(xj, xk)

N,N

j,k=1 .

We also let the random vector yξ := (y1, · · · , yN+M)T.

qye3@iit.edu MCQMC 2012 February 2012

Background The method in a nutshell 3

We also define its integral-type kernel

∗

K(x, y) :=

• D

K(x, z)K(y, z)dz,

∗

K ∈ Hm,m(D × D).

4

The kernel-based collocation solution is written as u(x) ≈ ˆ u(x) :=

N

• k=1

ck∆2

∗

K(x, xk) +

M

• k=1

cN+k

∗

K(x, xN+k), where the unknown random coefficients c := (c1, · · · , cN+M)T are obtained by solving a random system of linear equations, i.e.,

∗

Kc = yξ.

qye3@iit.edu MCQMC 2012 February 2012

Background Advantages

Advantages

The kernel-based collocation method is a meshfree approximation

• method. It does not require an underlying triangular mesh as the

Galerkin finite element method does. The kernel-based collocation method can be applied to a high-dimensional domain D with complex boundary ∂D. To obtain the truncated Gaussian noise ξn for the finite element method, it is difficult for us to compute the eigenvalues and eigenfunctions of the noise covariance kernel W. For the kernel-based collocation method we need not worry about this issue. Once the reproducing kernel is fixed, the error of the collocation solution only depends on the collocation points.

qye3@iit.edu MCQMC 2012 February 2012

Background Difference for Finite Element Methods

Given a finite element basis φ, we shall compute the right-hand side for the Galerkin finite element methods. Popular Methods:

• D

ξxφ(x)dx ≈

• D

ξn

xφ(x)dx = n

• k=1

ζk

• D
• λkek(x)φ(x)dx,

where the truncated Gaussian noise ξx ≈ ξn

x = n

• k=1

ζk

• λkek(x),

ζ1, . . . , ζn ∼ i.i.d.N(0, 1), and W(x, y) ≈ W n(x, y) =

n

• k=1

λkek(x)ek(y).

qye3@iit.edu MCQMC 2012 February 2012

Background Difference for Finite Element Methods

Monte Carlo Methods: For each fixed sample path ω ∈ Ωξ, ξx(ω) is a function defined on

• D. However, we do not know its exact form. We can only use

Monte Carlo methods to approximate the right-hand side, i.e.,

• D

ξxφ(x)dx ≈

N

• j=1

ξxjφ(xj). Kernel-based Methods: ξx ≈ ˆ ξx := w(x)TW−1ξ, where w(x) := (W(x, x1), · · · , W(x, xN))T, W :=

• W(xj, xk)

N,N

j,k=1 .

qye3@iit.edu MCQMC 2012 February 2012

Kernel-based Collocation Methods

Outline

1

Introduction

2

Background

3

Kernel-based Collocation Methods

4

Numerical Examples

5

Acknowledgments

qye3@iit.edu MCQMC 2012 February 2012

Kernel-based Collocation Methods Gaussian Fields

According to [Cialenco, Fasshauer and Ye 2011 SPDE, Theorem 3.1], for a given µ ∈ HK(D), there exists a probability measure Pµ defined on (ΩK, FK) = (HK(D), B(HK(D))) such that the stochastic fields ∆S, S given by ∆Sx(ω) = ∆S(x, ω) := (∆ω)(x), x ∈ D, ω ∈ ΩK = HK(D), Sx(ω) = S(x, ω) := ω(x), x ∈ D ∪ ∂D, ω ∈ ΩK = HK(D), are Gaussian with means ∆µ, µ and covariance kernels ∆1∆2

∗

K,

∗

K defined on (ΩK, FK, Pµ), respectively. For any fixed z ∈ R, we let Ex(z) := {ω ∈ ΩK : ω(x) = z} = {ω ∈ ΩK : Sx(ω) = z} .

qye3@iit.edu MCQMC 2012 February 2012

Kernel-based Collocation Methods Gaussian Fields

[Cialenco, Fasshauer and Ye 2011 SPDE, Corollary 3.2], shows that the random vector S := (∆Sx1, · · · , ∆SxN, SxN+1, · · · , SxN+M) ∼ N(mµ,

∗

K), where mµ := (∆µ(x1), · · · , ∆µ(xN), µ(xN+1), · · · , µ(xN+M))T

∗

K :=  (∆1∆2

∗

K(xj, xk))N,N

j,k=1,

(∆1

∗

K(xj, xN+k))N,M

j,k=1

(∆2

∗

K(xN+j, xk))M,N

j,k=1,

(

∗

K(xN+j, xN+k))M,M

j,k=1

  . For any given y = (y1, · · · , yN+M)T ∈ RN+M, we let EX(y) := {ω ∈ ΩK : ∆ω(x1) = y1, . . . , ω(xN+M) = yN+M} = {ω ∈ ΩK : S(ω) = y} .

qye3@iit.edu MCQMC 2012 February 2012

Kernel-based Collocation Methods Approximation and Convergence

For each fixed x ∈ D and ω2 ∈ Ωξ, we obtain the "optimal" estimator u(x, ω2) ≈ ˆ u(x, ω2) = argmax

z∈R

sup

µ∈HK (D)

Pµ

ξ

• Ex(z) × Ωξ
• EX
• yξ(ω2)
• ,

= argmax

z∈R

sup

µ∈HK (D)

Pµ

ξ

• Sx = z
• S = yξ(ω2)
• ,

= argmax

z∈R

sup

µ∈HK (D)

pµ

x(z|yξ(ω2)),

= k(x)T ∗ K−1yξ(ω2) where k(x) := (∆2

∗

K(x, x1), · · · ,

∗

K(x, xN+M))T and ΩKξ := ΩK × Ωξ, FKξ := FK ⊗ Fξ, Pµ

ξ := Pµ ⊗ Pξ,

so that ∆S, S and ξ can be extended to the product space while preserving the original probability distributional properties.

qye3@iit.edu MCQMC 2012 February 2012

Kernel-based Collocation Methods Approximation and Convergence

Error Bound Analysis

For any ǫ > 0, we define Eǫ

x :=

• ω1 × ω2 ∈ ΩK × Ωξ : |ω1(x) − ˆ

u(x, ω2)| ≥ ǫ, s.t. ∆ω1(x1) = y1(ω2), . . . , ω1(xN+M) = yN+M(ω2)

• .

Let the fill distance hX := sup

x∈D

min

1≤j≤N+Mx − xj2.

qye3@iit.edu MCQMC 2012 February 2012

Kernel-based Collocation Methods Approximation and Convergence

We can deduce that sup

µ∈HK (D)

Pµ

ξ (Eǫ x) = O

• hm−2−d/2

X

ǫ

• ,

where m is the order of the Sobolev space corresponded to the exact solution of the SPDE. Since |u(x, ω2) − ˆ u(x, ω2)| ≥ ǫ if and only if u ∈ Eǫ

x, we have

sup

µ∈HK (D)

Pµ

ξ

• u − ˆ

uL∞(D) ≥ ǫ

• ≤

sup

µ∈HK (D),x∈D

Pµ

ξ (Eǫ x) → 0,

when hX → 0.

qye3@iit.edu MCQMC 2012 February 2012

Numerical Examples

Outline

1

Introduction

2

Background

3

Kernel-based Collocation Methods

4

Numerical Examples

5

Acknowledgments

qye3@iit.edu MCQMC 2012 February 2012

Numerical Examples Stochastic Laplace’s Equations

Let the domain D := (0, 1)2 ⊂ R2. We choose the deterministic function f(x) := −2π2 sin(πx1) sin(πx2) − 8π2 sin(2πx1) sin(2πx2), and the covariance kernel of the Gaussian noise ξ to be W(x, y) :=4π4 sin(πx1) sin(πx2) sin(πy1) sin(πy2) + 16π4 sin(2πx1) sin(2πx2) sin(2πy1) sin(2πy2). Then the exact solution of the above elliptic SPDE has the form u(x) := sin(πx1) sin(πx2) + sin(2πx1) sin(2πx2) + ζ1 sin(πx1) sin(πx2) + ζ2 2 sin(2πx1) sin(2πx2), where ζ1, ζ2 ∼ i.i.d. N(0, 1).

qye3@iit.edu MCQMC 2012 February 2012

Numerical Examples Stochastic Laplace’s Equations

For the collocation methods, we use the C4-Matérn function with shape parameter θ > 0 gθ(r) := (3 + 3θr + θ2r 2)e−θr, r > 0, to construct the reproducing kernel (Sobolev-spline kernel) Kθ(x, y) := gθ(x − y2). According to [Fasshauer and Ye 2011 Distributional Operators, Fasshauer and Ye 2011 Differential and Boundary Operators], we can deduce that HKθ(D) ∼ = H3+1/2(D) ⊂ C2(D).

qye3@iit.edu MCQMC 2012 February 2012

Numerical Examples Stochastic Laplace’s Equations

0.5 1 0.5 1 −1 1 2

Approximate Mean

Relative Absolute Error 0.02 0.04 0.06 0.08 0.5 1 0.5 1 0.5 1

Approximate Variance

Relative Absolute Error 0.01 0.02 0.03 0.04 0.05 0.06 −4 −2 2 4 0.1 0.2 0.3 0.4 0.5 PDF, x

1 = 0.52632, x2 = 0.52632

Empirical Theoretical 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1

Collocation Points

Figure: N = 65, M = 28 and θ = 0.9

qye3@iit.edu MCQMC 2012 February 2012

Numerical Examples Stochastic Laplace’s Equations 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.05 0.1 0.15 0.2 0.25 0.3

fill distance hX Relative Root−mean−square Error Mean, θ = 0.9 Variance, θ = 0.9 Mean, θ = 1.9 Variance, θ = 1.9 Mean, θ = 2.9 Variance, θ = 2.9

Figure: Convergence of Mean and Variance

qye3@iit.edu MCQMC 2012 February 2012

Acknowledgments

Outline

1

Introduction

2

Background

3

Kernel-based Collocation Methods

4

Numerical Examples

5

Acknowledgments

qye3@iit.edu MCQMC 2012 February 2012

Acknowledgments

THANK YOU for the invitation and the NSF support from Prof. Owen.

qye3@iit.edu MCQMC 2012 February 2012

Appendix References

References I

• R. A. Adams and J. J. F

. Fournier, Sobolev Spaces (2nd Ed.), Pure and Applied Mathematics, Vol. 140, Academic Press, 2003.

• A. Berlinet and C. Thomas-Agnan,

Reproducing Kernel Hilbert Spaces in Probability and Statistics, Kluwer Academic Publishers, 2004.

• M. D. Buhmann,

Radial Basis Functions: Theory and Implementations, Cambridge University Press (Cambridge), 2003.

• G. E. Fasshauer,

Meshfree Approximation Methods with MATLAB, Interdisciplinary Mathematical Sciences, Vol. 6, World Scientific Publishers (Singapore), 2007.

qye3@iit.edu MCQMC 2012 February 2012

Appendix References

References II

• L. Hörmander,

The analysis of linear partial differential operators I, Classics in Mathematics, Springer, 2004. P . E. Kloeden and E. Platen Numerical Solution of Stochastic Differential Equations, Vol. 23, Springer, 2011.

• B. Øksendal

Stochastic Differential Equations: An Introduction with Applications, 6th edition, Springer, 2010.

• I. Steinwart and A. Christmann,

Support Vector Machines, Springer Science Press, 2008.

qye3@iit.edu MCQMC 2012 February 2012

Appendix References

References III

• G. Wahba,

Spline Models for Observational Data, CBMS-NSF Regional Conference Series in Applied Mathematics 59, SIAM (Philadelphia), 1990.

• H. Wendland,

Scattered Data Approximation, Cambridge University Press, 2005.

• G. E. Fasshauer and Q. Ye,

Reproducing Kernels of Generalized Sobolev Spaces via a Green Function Approach with Distributional Operator, Numerische Mathematik, Volume 119, Number 3, Pages 585-611, 2011.

qye3@iit.edu MCQMC 2012 February 2012

Appendix References

References IV

• G. E. Fasshauer and Q. Ye,

Reproducing Kernels of Sobolev Spaces via a Green Function Approach with Differential Operators and Boundary Operators, Advances in Computational Mathematics, 2011, to appear, DOI: 10.1007/s10444-011-9264-6.

• G. E. Fasshauer and Q. Ye,

Kernel-based Collocation Methods versus Galerkin Finite Element Methods for Approximating Elliptic Stochastic Partial Differential Equations, in preparation.

• I. Cialenco, G. E. Fasshauer and Q. Ye,

Approximation of Stochastic Partial Differential Equations by a Kernel-based Collocation Method, submitted.

qye3@iit.edu MCQMC 2012 February 2012

Appendix References

References V

• S. Koutsourelakis and J. Warner

Learning Solutions to Multiscale Elliptic Problems with Gaussian Process Models, Research report at Cornell University, 2009.

• Q. Ye,

Reproducing Kernels of Generalized Sobolev Spaces via a Green Function Approach with Differential Operator, IIT technical report, 2010.

qye3@iit.edu MCQMC 2012 February 2012