Analysis of the stability and accuracy of multivariate polynomial - - PowerPoint PPT Presentation

epfl-mox-logo

Analysis of the stability and accuracy of multivariate polynomial approximation by discrete least squares with evaluations in random

• r low-discrepancy point sets

Giovanni Migliorati

MATHICSE-CSQI, ´ Ecole Polytechnique F´ ed´ erale de Lausanne

Analysis with random points: joint work with Fabio Nobile (EPFL), Raul Tempone (KAUST), Albert Cohen (UPMC), Abdellah Chkifa (UPMC) and Erik von Schwerin (KTH). Analysis with low-discrepancy points: joint work with Fabio Nobile.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 1

epfl-mox-logo

Outline

1

Discrete least squares on multivariate polynomial spaces

2

Stability and accuracy with evaluations in random points

3

Stability and accuracy with evaluations in low-discrepancy point sets

4

Conclusions

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 2

epfl-mox-logo Discrete least squares on multivariate polynomial spaces

1

Discrete least squares on multivariate polynomial spaces

2

Stability and accuracy with evaluations in random points

3

Stability and accuracy with evaluations in low-discrepancy point sets

4

Conclusions

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 3

epfl-mox-logo Discrete least squares on multivariate polynomial spaces

Notation and definitions

For any d ≥ 1, Γ := [−1, 1]d and any real numbers α, β > −1, define ρ(y) := B(α, β)−d

d

• i=1

(1 − yi)α(1 + yi)β, y ∈ Γ, f1, f2L2

ρ(Γ) :=

• Γ

f1(y)f2(y)ρ(y)dy, f1, f2M := 1 M

M

• m=1

f1(ym) f2(ym), · L2

ρ := ·, ·1/2

L2

ρ ,

· M := ·, ·1/2

M ,

with y1, . . . , yM being any points in Γ, either realizations of i.i.d. random variables Y1, . . . , YM

i.i.d.

∼ ρ or deterministically given (e.g. low-discrepancy point sets). Given univariate L2

ρ-orthonormal polynomials (ϕk)k≥0 and a

multi-index set Λ ⊂ Nd

0, for any ν ∈ Λ we define

ψν(y) :=

d

• i=1

ϕνi(yi), y ∈ Γ, PΛ := span {ψν : ν ∈ Λ} .

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 4

epfl-mox-logo Discrete least squares on multivariate polynomial spaces

Markov and Nikolskii inequalities for multivariate polynomials with downward closed multi-index sets

Definition (Downward closed multi-index set) Λ is downward closed if (ν ∈ Λ and ν′ ≤ ν) ⇒ ν′ ∈ Λ. Lemma (M. 2014) In any dimension, for any Λ downward closed and any α, β ∈ N0 it holds u2

L∞(Γ) ≤ (#Λ)2 max{α,β}+2u2 L2

ρ(Γ),

∀u ∈ PΛ(Γ). Lemma (M. 2014) In any dimension and for any Λ downward closed, when α = β = 0 (Legendre polynomials), it holds

• ∂d

∂y1 · · · ∂yd u

• 2

L2

ρ(Γ)

≤ 4−d(#Λ)4u2

L2

ρ(Γ),

∀u ∈ PΛ(Γ).

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 5

epfl-mox-logo Discrete least squares on multivariate polynomial spaces

Discrete least squares on polynomial spaces

For any smooth (analytic) real-valued (or Hilbert-valued) function φ : Γ → R, we define its continuous and discrete L2 projections over PΛ as ΠΛφ := argmin

v∈PΛ

φ − vL2

ρ,

ΠM

Λ φ := argmin v∈PΛ

φ − vM. Algebraic formulation: design matrix [D]ij = ψj(yi), right-hand side [b]i = φ(yi), for any i = 1, . . . , M and j = 1, . . . , #Λ. Normal equations: D⊤D β = D⊤b, with β containing the coefficients of the expansion ΠM

Λ φ = ν∈Λ βνψν.

We define also the matrix G := D⊤D/M.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 6

epfl-mox-logo Discrete least squares on multivariate polynomial spaces

Optimality of discrete least squares in the L2

ρ norm

In any dimension, with any index set Λ and any ρ with bounded support: Proposition (M., Nobile, von Schwerin and Tempone, FoCM 2014) For any (random or deterministic) choice of M points in Γ it holds φ − ΠM

Λ φL2

ρ ≤

• 1 +
• |||G −1|||
• inf

v∈PΛ φ − vL∞.

Proof

Theorem (M., Nobile, von Schwerin and Tempone, FoCM 2014) Given M points in Γ, being realizations of random variables independent and identically distributed w.r.t. ρ, it holds lim

M→+∞ |||G −1||| =

lim

M→+∞ |||G ||| = 1,

almost surely. Proposition (M., Nobile, von Schwerin and Tempone, FoCM 2014) cond (G) = |||G||| |||G −1|||.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 7

epfl-mox-logo Discrete least squares on multivariate polynomial spaces

Norm equivalence on PΛ (case of random points)

Find δ ∈ (0, 1) such that (1 − δ)v2

L2

ρ ≤ v2

M ≤ (1 + δ)v2 L2

ρ,

∀v ∈ PΛ, with high probability. Since v2

M = M−1Dv, Dv2 R#Λ = Gv, v2 R#Λ and v2 L2

ρ = v, v2

R#Λ, the

matrix G satisfies |||G||| = sup

v∈PΛ\{v≡0}

v2

M

v2

L2

ρ

, |||G −1||| = sup

v∈PΛ\{v≡0}

v2

L2

ρ

v2

M

. Hence, norm equivalence on PΛ w.h.p. iff concentration bounds 1 − δ ≤ |||G||| ≤ 1 + δ, 1 1 + δ ≤ |||G −1||| ≤ 1 1 − δ, |||G − I||| ≤ δ, again with high probability.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 8

epfl-mox-logo Stability and accuracy with evaluations in random points

1

Discrete least squares on multivariate polynomial spaces

2

Stability and accuracy with evaluations in random points

3

Stability and accuracy with evaluations in low-discrepancy point sets

4

Conclusions

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 9

epfl-mox-logo Stability and accuracy with evaluations in random points

Given any L2

ρ-orthonormal polynomial basis (ψν)ν∈Λ of PΛ, define

K(Λ) := sup

y∈Γ

• ν∈Λ

|ψν(y)|2

• = sup

v∈PΛ

v2

L∞

v2

L2

ρ

. Lemma (Chkifa, Cohen, M., Nobile and Tempone, 2013) In any dimension and for any downward closed Λ it holds K(Λ) ≤ (#Λ)ln 3/ ln 2, with tensorized Chebyshev 1st kind polynomials. Lemma (M. 2014) In any dimension, for any downward closed Λ and any α, β ∈ N0 it holds K(Λ) ≤ (#Λ)2 max{α,β}+2, with tensorized Jacobi polynomials. These bounds are quite general, and set the ground for adaptive polynomial approximation based on discrete least squares.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 10

epfl-mox-logo Stability and accuracy with evaluations in random points

Assume that |φ| ≤ τ almost surely w.r.t. ρ and define Tτ(t) := sign(t) min{τ, |t|},

• ΠM

Λ := Tτ(ΠM Λ ).

Theorem (Chkifa, Cohen, M., Nobile and Tempone, 2013) For any γ>0 and any downward closed Λ, if M is such that K(Λ) ≤ 0.15 1 + γ M ln M then, for any φ ∈ L∞(Γ) with φL∞ ≤ τ, it holds that Pr (cond(G) ≤ 3) ≥ 1 − 2M−γ, Pr

• φ − ΠM

Λ φL2

ρ ≤ (1 +

√ 2) inf

v∈PΛ φ − vL∞

• ≥ 1 − 2M−γ,

E

• φ −

ΠM

Λ φ2 L2

ρ

• ≤
• 1 +

0.6 (1 + γ) ln M

• φ − ΠΛφ2

L2

ρ + 8τ 2M−γ.

(δ = 1/2 everywhere!)

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 11

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

1

Discrete least squares on multivariate polynomial spaces

2

Stability and accuracy with evaluations in random points

3

Stability and accuracy with evaluations in low-discrepancy point sets

4

Conclusions

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 12

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

Discrete least squares with deterministic points: multivariate case with Chebyshev density in [0, 1]d

Deterministic points introduced by Zhou, Narayan and Xu: yj = cos 2π M (j, . . . , jd)

• ∈ [−1, 1]d,

j = 1, . . . , M, asymptotically distributed according to the Chebyshev density. Theorem (Zhou, Narayan and Xu, arXiv 2014) In any dimension d and with the Chebyshev density, if M is a prime number and M ≥ 4d+1d2(#Λ)2 then it holds that φ − ΠM

Λ φL2

ρ ≤

• 1 +

4 d2#Λ

• inf

v∈PΛ φ − vL∞.

The proof uses arguments from number theory.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 13

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

Discrete least squares with deterministic points: the multivariate case with uniform density in [0, 1]d

Given any set of M points y1, . . . , yM ∈ [0, 1]d and any set ∅ = U ⊆ {1, . . . , d}, we define its local discrepancy ∆U(t, 1) := 1 M

M

• i=1
• q∈U

I[0,tq](yq

i ) −

• q∈U

tq, t ∈ [0, 1]|U|, and its star-discrepancy D∗,U := sup

t∈[0,1]|U| |∆U(t, 1)|.

Values of components in {1, ..., d} \ U are frozen to 1.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 14

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

Example d = 2, U = {2}, {1, 2} \ U = {1}

(picture from J.Dick, F.Pillichshammer: Digital Nets and Sequences, 2010)

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 15

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

Discrete least squares with deterministic points: the multivariate case with uniform density in [0, 1]d

Let t ≥ 0, m ≥ 1, d ≥ 1 and b ≥ 2 be integers with t ≤ m. A (t, m, d)-net in base b is a point set consisting of bm points in [0, 1)d such that every elementary interval of the form

d

• q=1

aj bhj , aj + 1 bhj

• with each hj ≥ 0, 0 ≤ aj < bhj and h1 + . . . + hd = m − t, contains

exactly bt points. Example: (0, 4, 2)-net in base b = 2 (Hammersley points).

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 16

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

Discrete least squares with deterministic points: the multivariate case with uniform density in [0, 1]d

Our analysis uses a type of Koksma-Hlawka inequality and low-discrepancy point sets. Starting point: Lemma (Hlawka-Zaremba’s identity ) Given M points y1, . . . , yM ∈ [0, 1]d, for any f with continuous mixed derivatives it holds

• [0,1]d f (y)dy − 1

M

M

• i=1

f (yi)

• =
• ∅=U⊆{1,...,d}

(−1)|U|

• [0,1]|U| ∆U(yU, 1) ∂|U|

∂yU f (yU, 1) dyU.

Lemma (Standard Koksma-Hlawka inequality)

• [0,1]d f (y)dy − 1

M

M

• i=1

f (yi)

• ≤ D∗,{1,...,d}f HK.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 17

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

Discrete least squares with deterministic points: the multivariate case with uniform density in [0, 1]d

Three main ingredients in our approach: 1) we prove a variant of the standard Koksma-Hlawka inequality starting from the Hlawka-Zaremba’s identity: Lemma (M., Nobile 2014)

• f 2

L2

ρ − f 2

M

• ≤
• ∅=U⊆{1,...,d}

D∗,U

T⊆U

• ∂|T|

∂yT f (yU, 1)

• L2([0,1]|U|)
• ∂|U|−|T|

∂yU\T f (yT , 1)

• L2([0,1]|U|)

.

2) Markov-type and Nikolskii-type multivariate inequalities for polynomials associated with downward closed multi-index sets (M. 2014). 3) upper bounds for the star-discrepancy of (t, m, d)-nets and (t, d)-sequences (e.g. Faure-Kritzer, Monatsh. Math. 2013).

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 18

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

Discrete least squares with deterministic points: the multivariate case with uniform density in [0, 1]d

Consider any (t, m, d)-net in base b ≥ 2 with quality parameter t ≥ 0. Theorem (M., Nobile 2014) In any dimension d, with the uniform density and with anisotropic tensor product spaces PΛ, if 1 > δ >

• 0.7 bt exp

b − 1 2 ln b

• + O (1)
• (#Λ)2 (1 + 2 ln M)d

M then it holds that cond(G) ≤ 1 + δ 1 − δ, φ − ΠM

Λ φL2

ρ ≤

• 1 +
• 1

1 − δ

• inf

v∈PΛ φ − vL∞.

Similar theorem also for (t, d)-sequences (M.,Nobile 2014).

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 19

epfl-mox-logo Stability and accuracy with low-discrepancy point sets

Discrete least squares with deterministic points: the multivariate case with uniform density in [0, 1]d

Given Λ downward closed and U ⊆ {1, . . . , d} we define its sections by ΛU := {ν ∈ N|U| : ∃ µ = (µU, µ{1,...,d}\U) ∈ Λ and ν = µU}. In general, for any downward closed multi-index set the condition becomes 1 > δ > min   (#Λ)4

• ∅=U⊆{1,...,d}

D∗,U,

• ∅=U⊆{1,...,d}

D∗,U #Λ{1,...,d}\U 2

T⊆U

(#ΛT)2 #ΛU\T 2    . Nonoptimal when Λ is more sparse than anisotropic tensor product, compared to M ∝ (#Λ)2 with random points and any Λ downward closed.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 20

epfl-mox-logo Conclusions

1

Discrete least squares on multivariate polynomial spaces

2

Stability and accuracy with evaluations in random points

3

Stability and accuracy with evaluations in low-discrepancy point sets

4

Conclusions

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 21

epfl-mox-logo Conclusions

Conclusions (theoretical analysis)

RANDOM POINTS: analysis w.r.t. M, d, Λ, ρ, smoothness φ: in any dimension d, proven stability and accuracy provided that

M/ ln M ≥ C1(dim(PΛ))

ln 3 ln 2

with Chebyshev density, M/ ln M ≥ C2(dim(PΛ))2 with uniform density, M/ ln M ≥ C3(dim(PΛ))2 max{α,β}+2 with beta(α + 1, β + 1), α, β ≥ 0,

with the constants C1, C2, C3 being independent of d. DETERMINISTIC POINTS: analysis w.r.t. M, d, Λ, smoothness φ: in any dimension d, proven stability and accuracy provided that

M ≥ C1(d)(dim(PΛ))2 with Chebyshev density and any Λ (Zhou et al.), M/(1 + 2 ln M)d ≥ C2(dim(PΛ))2 with uniform density and anisotropic tensor product, M/(1 + 2 ln M)d ≥ C3(dim(PΛ))γ, 2 ≤ γ ≤ 4 with uniform density and any Λ downward closed.

with the constant C1 being dependent on d, and C2, C3 being dependent on the parameters of the (t, m, d)-net or (t, d)-sequence.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 22

epfl-mox-logo Conclusions

Conclusions (experience from numerics)

In high dimensions and with smooth functions, with both random and deterministic points, it seems to be enough M ∝ dim(PΛ) to achieve the optimal convergence rate up to a threshold. A lot

• f numerical evidence, but no formal proof yet.

Deterministic points CAN outperform random points in low

• dimensions. What about high dimensions?

Discrete least squares is a well-promising approximation tool for multivariate aleatory functions and PDEs with stochastic data.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 23

epfl-mox-logo Conclusions

References on discrete least squares with RANDOM points

A.Cohen, M.Davenport, D.Leviatan: On the stability and accuracy of least squares

• approximations. Foundations of Computational Mathematics, 2013.

G.Migliorati, F.Nobile, E.von Schwerin, R.Tempone: Analysis of discrete L2 projection on polynomial spaces with random evaluations. Foundations of Computational Mathematics, 2014. A.Chkifa, A.Cohen, G.Migliorati, F.Nobile, R.Tempone: Discrete least squares polynomial approximation with random evaluations; application to parametric and stochastic elliptic PDEs.

• submitted. Available as MATHICSE report 35-2013.

G.Migliorati, F.Nobile, E.von Schwerin, R.Tempone: Approximation of Quantities of Interest in stochastic PDEs by the random discrete L2 projection on polynomial spaces, SIAM J. Sci. Comput., 2013. G.Migliorati: Multivariate Markov-type and Nikolskii-type inequalities for polynomials associated with downward closed multi-index sets, submitted. Available as MATHICSE report 1-2014. G.Migliorati: Polynomial approximation by the random discrete L2 projection and application to inverse problems for PDEs with stochastic data, PhD thesis, Department of Mathematics at Politecnico di Milano and Centre de Math´ ematiques Appliqu´ ees at ´ Ecole Polytechnique, 2013.

References on discrete least squares with DETERMINISTIC points

T.Zhou, A.Narayan, Z.Xu: Multivariate discrete least-squares approximations with a new type

• f collocation grid, arkiv:1401.0894v1, 2014.

G.Migliorati, F.Nobile: Analysis of discrete least squares on multivariate polynomial spaces with evaluations in low-discrepancy point sets, submitted. Available as MATHICSE report 25-2014.

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 24

epfl-mox-logo Conclusions

Thank you for your attention!

G.Migliorati (EPFL) ICERM - Brown University Providence - September 23th, 2014 25