[PPT] - Motivating Example Compute expectation E ( g ( X ( t 0 ))) for PowerPoint Presentation

SLIDE 1

G. W. Wasilkowski

∞-Variate Linear Problems

G. W. Wasilkowski

Department of Computer Science University of Kentucky

RICAM 2018 1

SLIDE 2

G. W. Wasilkowski

∞-Variate Linear Problems

In this presentation: Information-Based Complexity approach, i.e., study of the worst case complexity with respect to a whole normed space F of functions

RICAM 2018 2

SLIDE 3

G. W. Wasilkowski

∞-Variate Linear Problems

In this presentation: Information-Based Complexity approach, i.e., study of the worst case complexity with respect to a whole normed space F of functions There is the Curse of Dimensionality for isotropic spaces. To brake this curse, Non-isotropic spaces are needed.

RICAM 2018 3

SLIDE 4

G. W. Wasilkowski

∞-Variate Linear Problems

In this presentation: Information-Based Complexity approach, i.e., study of the worst case complexity with respect to a whole normed space F of functions There is the Curse of Dimensionality for isotropic spaces. To brake this curse, Non-isotropic spaces are needed. Weighted Spaces are good candidates. Introduced by [Sloan and Wo´ zniakowski 1998], assign different importance to different variables.

RICAM 2018 4

SLIDE 5

G. W. Wasilkowski

∞-Variate Linear Problems

Motivating Example

Compute expectation E(g(X(t0))) for stochastic process X(t) = ∞

j=1 xj · ξj(t),

where xj i.i.d. random variables w.r.t. µ and ξj(t) → 0 fast.

RICAM 2018 5

SLIDE 6

G. W. Wasilkowski

∞-Variate Linear Problems

Motivating Example

Compute expectation E(g(X(t0))) for stochastic process X(t) = ∞

j=1 xj · ξj(t),

where xj i.i.d. random variables w.r.t. µ and ξj(t) → 0 fast. Equivalent to computing an ∞-variate integral

RN f(x) dµN(x)

for

f(x) = g  

∞

j=1

xj · ξj(t0)  

RICAM 2018 6

SLIDE 7

G. W. Wasilkowski

∞-Variate Linear Problems

In

g  

∞

j=1

xj ξj(t0)  

“importance” of xj is quantized by the size of |ξj(t0)|. The larger |ξj(t0)| the more important xj. This leads to WEIGHTED SPACES

f ∞-VARIATE FUNCTIONS

RICAM 2018 7

SLIDE 8

G. W. Wasilkowski

∞-Variate Linear Problems

Initial complexity attempts: [W. and Wo´ zniakowski 96 and 00], [Plaskota, W. and Wo´ zniakowski 00], and [Hickernell and Wang 01] [Hickernell, M¨ uller-Gronbach, Niu, Ritter 10] Since 2010, many papers have been written by, e.g.,

J. Baldeaux,
A. Cohen,
J. Creutzig,
W. Dahmen,
S. Dereich,
R. DeVore,
J. Dick,
D. D˜

ung,

A. Gilbert,
M. Gnewuch,
M. Griebel,
M. G. Gunzburger,
M. Hefter,
S. Heinrich,
F. J. Hickernell,
A. Hinrichs,
Z. Kacewicz,
P. Kritzer,
F. Y. Kuo,
S. Mayer,
T. M¨

uller-Gronbach,

P. Morkisz,
B. Niu,
J. Nichols,
D. Nuyens,
F. Pillichshammer,
L. Plaskota,
P. Przybylowicz,
K. Ritter,
Ch. Schwab,
I. H. Sloan,
H. Wo´

zniakowski

RICAM 2018 8

SLIDE 9

G. W. Wasilkowski

∞-Variate Linear Problems

However, majority of the papers dealing with: Hilbert spaces, integration problem, and too simplistic cost model In this presentation: Banach or just normed spaces, linear tensor-product problems, and more realistic cost model

RICAM 2018 9

SLIDE 10

G. W. Wasilkowski

∞-Variate Linear Problems

In our model, f(x) can be evaluated for

x with only finitely many xj = 0 f(xw) = g  

j∈w

xj ξj(t0)  

for finite w ⊂ N Cost of obtaining the sample f(xw) should depend on the size of w, i.e.

cost(f(xw)) = $(|w|)

Positive and sharp results for

Ω(k) = $(k) = eO(k)

RICAM 2018 10

SLIDE 11

G. W. Wasilkowski

∞-Variate Linear Problems

Notation: Domain:

DN set of sequences x = (x1, x2, . . . )

with xj ∈ D; e.g., D = [0, 1], R+ or R

w

finite subsets of

N+

listing the “variables in action”, e.g., given x = (x1, x2, . . . ),

xw = (xj : j ∈ w) [xw; 0] = (y1, y2, . . . )

with

yj =    xj

if j ∈ w, if j /

∈ w

For p ∈ [1, ∞] we use p∗ =

p p−1

RICAM 2018 11

SLIDE 12

G. W. Wasilkowski

∞-Variate Linear Problems

Weighted Spaces Fγ

Any f ∈ Fγ has decomposition:

f =

w fw

such that

fw depends only on xw, fw ∈ Fw,

where Fw is a Banach (or just normed) space and Fw ∩ Fv = {0} if w = v It is Anchored if

fw(x) = 0

when

xj = 0 for j ∈ w

RICAM 2018 12

SLIDE 13

G. W. Wasilkowski

∞-Variate Linear Problems

The space Fγ is endowed with

fF =

w

γ−p

w fwp Fw

1/p

Here

p ∈ [1, ∞]

and

γw ≥ 0 are weights

Product Weights [Sloan and Wo´

zniakowski 1998]:

γw = c

j∈w

j−β

Product Order Depedent Weights [Kuo, Schwab, and Sloan 14]:

γw = (|w|!)α

j∈w

j−β β > max(α, 1/p∗)

RICAM 2018 13

SLIDE 14

G. W. Wasilkowski

∞-Variate Linear Problems

Problem to Approximate

Approximate

S(f),

where

S : Fγ → G

is a linear operator into a normed space G Example:

S(f) =

DN f ρN

= lim

d→∞

Dd f(x1, . . . , xd, 0, . . . )

d

j=1

ρ(xj) d(x1, . . . , xd)

where ρ is a probability density on D

RICAM 2018 14

SLIDE 15

G. W. Wasilkowski

∞-Variate Linear Problems

ASSUMPTION 1: For every w,

S|FwFw ≤ C|w|

1

ASSUMPTION 2: Continuity of S

SFγ =

w

γp∗

w S|Fwp∗ Fw

1/p∗ ≤

w

γp∗

w C|w| p∗ 1

1/p∗ < ∞

For tensor product problems: S|FwFw = C|w|

1

with C1 = S|F1F1 and for product weights: SFγ =

∞

j=1

1 + (C1 j−β)p∗1/p∗

RICAM 2018 15

SLIDE 16

G. W. Wasilkowski

∞-Variate Linear Problems

Standard Integration Problem:

Fγ

endowed with the norm

fF =

w

γ−p

w

f (w)([·w; 0])
p

Lp(D|w|)

1/p , f (w) =

j∈w

∂ ∂xj f D = [0, 1] and ρ ≡ 1 (most often Hilbert case p = 2)

and

S = I =

DN

Then

γw ≃

j∈w

1 |ξj(t0)|δ

and

C1 = I|F1F1 = (1 + p∗)−1/p∗ (= 1

if p∗ = ∞)

RICAM 2018 16

SLIDE 17

G. W. Wasilkowski

∞-Variate Linear Problems

How to Cope with So Many Variables?

Truncate the Dimension, i.e., replace

f(x)

by

f(x1, . . . , xk, 0, 0, . . . )

for a “relatively” small k = k(error)

r (even better) use

Multivariate Decomposition Method i.e., approximate a small number of integrals each with a small number of variables

RICAM 2018 17

SLIDE 18

G. W. Wasilkowski

∞-Variate Linear Problems

Low Truncation Dimension

Anchored Decomposition

[Kritzer, Pillichshammer, W.2016]⊂ [Hinrichs, Kritzer, Pillichshammer, W.2018]

Let

fk(x1, . . . , xk) = f(x1, . . . , xk, 0, 0, . . . ) dimtrnc(ε) ε-truncation dimension

is the smallest k such that

S(f) − S(fk)G ≤ ε fFγ

for all f ∈ Fγ Our concept of Truncation Dimension is different than the one in Statistics.

RICAM 2018 18

SLIDE 19

G. W. Wasilkowski

∞-Variate Linear Problems

If S(f) − S(fk)G ≤ ε fF and

S(fk) − Ak(fk)G ≤ ε fF then S(f) − Ak(fk)G ≤ 2 ε fF

Hence the smaller dimtrnc(ε) the better THEOREM 1 For product weights γw =

j∈w j−β

dimtrnc(ε) = O

ε−1/(β−1+1/p)

for p > 1, and

dimtrnc(ε) = O

ε−1/β

for p = 1.

RICAM 2018 19

SLIDE 20

G. W. Wasilkowski

∞-Variate Linear Problems

Values of dimtrnc(ε) for Standard Integration Problem with p = 1 and γw =

j∈w j−β

ε 10−1 10−2 10−3 10−4 10−5 dimtrnc(ε) 2 9 31 99 316 β = 2 dimtrnc(ε) 2 4 9 21 46 β = 3 dimtrnc(ε) 1 3 5 9 17 β = 4

For instance, for the error demand ε = 10−3 with β = 4,

nly five variables instead of ∞-many!

Worst Case Error of QMC or Sparse Grids Methods is:

≤ O ln4 n n

RICAM 2018

20

SLIDE 21

G. W. Wasilkowski

∞-Variate Linear Problems

Arbitrary Decomposition Theorem 1 holds true for “Quasi”-Truncation Dimension defined by

q − dimtrnc(ε) := min   k :

S(f) − S

 

w⊆1:k

fw  

G

≤ ε fFγ   

f interest in Statistics (ANOVA Decomposition)

However for non-Anchored Decomposition

f(x1, . . . , xk, 0, 0, . . . ) =

w⊆1:k

fw(x)

and small q − dimtrnc(ε) need not lead to efficient algorithms. Unless..... we’ll see it later.

RICAM 2018 21

SLIDE 22

G. W. Wasilkowski

∞-Variate Linear Problems

Multivariate Decomposition Method

Introduced by [Kuo, Sloan, W., and Wo´

znakowski 2010]

MDM replaces

ne ∞-variate integral

by

nly few integrals

each with

nly few variables

RICAM 2018 22

SLIDE 23

G. W. Wasilkowski

∞-Variate Linear Problems

More precisely: Given the error demand ε > 0, construct an “active set” Act(ε)

f subsets w such that
S

 

w/

∈Act(ε)

fw  

G

≤ ε 2 fFγ

for all f ∈ F. Do nothing for

S(fw) with w/ ∈Act(ε),

and concentrate on S(fw) with w∈Act(ε).

RICAM 2018 23

SLIDE 24

G. W. Wasilkowski

∞-Variate Linear Problems

For w∈Act(ε), choose nw and algorithms Aw,nw to approximate S(fw) such that

w∈Act(ε)

S(fw) − Aw,nw(fw)G ≤ ε 2 fF

for all f ∈ F. The algorithms Aw,nw could be |w|-variate Sparse Grids Algorithms

RICAM 2018 24

SLIDE 25

G. W. Wasilkowski

∞-Variate Linear Problems

Then the MDM given by

Aε(f) :=

w∈Act(ε)

Aw,nw(fw)

has the “worst case error” bounded by ε, i.e.,

S(f) − Aε(f)G ≤ ε fF

for all f ∈ F.

RICAM 2018 25

SLIDE 26

G. W. Wasilkowski

∞-Variate Linear Problems

Then the MDM given by

Aε(f) :=

w∈Act(ε)

Aw,nw(fw)

has the “worst case error” bounded by ε, i.e.,

S(f) − Aε(f)G ≤ ε fF

for all f ∈ F.

How about the COST?

RICAM 2018 26

SLIDE 27

G. W. Wasilkowski

∞-Variate Linear Problems

The number of S(fw) to approximate is small:

card(ε) := |Act(ε)| = O

ε−1/r

,

where r is regularity degree (later). Each fw depends on only |w| variables. The largest number of variables is also small:

dim(Act(ε)) := max {|w| : w ∈ Act(ε)} = O(???)

RICAM 2018 27

SLIDE 28

G. W. Wasilkowski

∞-Variate Linear Problems

The number of S(fw) to approximate is small:

card(ε) := |Act(ε)| = O

ε−1/r

,

where r is regularity degree (later). Each fw depends on only |w| variables. The largest number of variables is also small:

dim(Act(ε)) := max {|w| : w ∈ Act(ε)} = O(???)

Theorem 2 For product order dependent weights, there is Act(ε) with

dim(Act(ε)) = O

ln(1/ε)

ln(ln(1/ε))

Proof like in [Plaskota and W. 2011]

RICAM 2018 28

SLIDE 29

G. W. Wasilkowski

∞-Variate Linear Problems

This leads to Worst Case ε-Superposition Dimension

dimsprp(ε) := inf

Act(ε) dim(Act(ε))

Theorem 2 provides upper bounds on dimsprp(ε)

[Gilbert, Kuo, Nuyens, and W. 2018] Efficient implementation of MDM

for Product Order Dependent Weights

[Gilbert and W. 2017] Very efficient algorithm

to construct optimal Act(ε) for Product Weights This yields exact value of dimsprp(ε)

RICAM 2018 29

SLIDE 30

G. W. Wasilkowski

∞-Variate Linear Problems

Illustration for Standard Integration Problem: Values of

dimsprp(ε)

and

card(ε)

for p = 1 and γw =

j∈w j−β

ε 10−1 10−2 10−3 10−4 10−5 2 , 6 3 , 22 4 , 113 4 , 534 5 , 2424 β = 2 2 , 6 2 , 8 3 , 22 3 , 68 4 , 192 β = 3 1 , 2 2 , 6 2 , 10 3 , 26 3 , 50 β = 4

For instance, for ε = 10−3 with β = 4 it is sufficient to approximate 10 integrals with at most 2 variables!

RICAM 2018 30

SLIDE 31

G. W. Wasilkowski

∞-Variate Linear Problems

Active Set Act(10−3) For β = 4

∅, {1}, {2}, {3}, {4}, {5}, {1, 2}, {1, 3}, {1, 4}, {1, 5}

For β = 3

∅, {1}, {2}, {3}, {4}, {5}, {6}, {7}, {8}, {9}, {1, 2}, {1, 3}, {1, 4}, {1, 5}, {1, 6}, {1, 7}, {1, 8}, {1, 9}, {2, 3}, {2, 4}, {1, 2, 3}, {1, 2, 4}

RICAM 2018 31

SLIDE 32

G. W. Wasilkowski

∞-Variate Linear Problems

For β = 2

31 of integrals with 1 variable 54 of integrals with 2 variables 26 of integrals with 3 variables 2 of integrals with 4 variables ∅, {1}, . . . , {31}, {1, 2}, . . . , {1, 31}, {2, 3}, . . . , {2, 15}, {3, 4}, . . . , {3, 10}, {4, 5}, {4, 6}, {4, 7}, {5, 6}, {1, 2, 3}, . . . , {1, 2, 15}, {1, 3, 4}, . . ., {1, 3, 10}, {1, 4, 5}, {1, 4, 6}, {1, 4, 7}, {1, 5, 6}, {2, 3, 4}, {2, 3, 5}, {1, 2, 3, 4}, {1, 2, 3, 5}

RICAM 2018 32

SLIDE 33

G. W. Wasilkowski

∞-Variate Linear Problems

REMARK: We do NOT know fw terms. However, we can sample them: due to [Kuo, Sloan, W., and Wo´

zniakowski 2010b]

fw(xw) =

v⊆w

(−1)|w|−|v| f([xv; 0])

requires

2|w|

samples of f

RICAM 2018 33

SLIDE 34

G. W. Wasilkowski

∞-Variate Linear Problems

REMARK: We do NOT know fw terms. However, we can sample them: due to [Kuo, Sloan, W., and Wo´

zniakowski 2010b]

fw(xw) =

v⊆w

(−1)|w|−|v| f([xv; 0])

requires

2|w|

samples of f but from [Plaskota and W. 2011]

2|w| = O

ε

−1 ln(ln(1/ε))

= o
ε−δ

for any δ > 0.

RICAM 2018 34

SLIDE 35

G. W. Wasilkowski

∞-Variate Linear Problems

Theorem 3: Suppose Fw are |w|-fold tensor product of F1 and for d = 1 there are algorithms An with

error(An; F1) = O(n−r) r − regularity degree.

Then for POD weights γw = (|w|!)α

j∈w C/j−β, there are

MDMethods Aε such that

error(A; Fγ) ≤ ε

and

cost(Aε) ≤ cδ ε−κ−δ

for and δ > 0, where

κ = max 1 r , 1 β − 1/p∗

.

RICAM 2018 35

SLIDE 36

G. W. Wasilkowski

∞-Variate Linear Problems

Theorem is sharp (modulo arbitrarily small δ > 0): Using a proof technique from

[Kuo, Sloan, W., and Wo´ zniakowski 2010]

complexity(ε; F) = Ω

ε−κ

Hence

Ω

ε−κ

= complexity(ε; Fγ) = O

ε−κ−δ

∀δ > 0.

Recall that complexity(ε; Fγ) is the smallest cost among all algorithms with errors ≤ ε

RICAM 2018 36

SLIDE 37

G. W. Wasilkowski

∞-Variate Linear Problems

Efficient implementation of MDM in

[Gilbert, Kuo, Nuyens, and W. [18]]

The same value f(xv) might be needed for approximating integrals of fw for v ⊆ w ∈ Act(ε). In the implementation, we avoid such repeated samplings.

RICAM 2018 37

SLIDE 38

G. W. Wasilkowski

∞-Variate Linear Problems

Application to Problems on Unbounded Domains

Univariate Functions:

F1 = W1,p,ψ(R+) is the Banach space of f(x) =

R+

h(t) (x − t)0

+dt

for

h ∈ Lp,ψ(R+)

with the norm

fF1 = f ′ ψLp(R+)

Clearly

f(0) = 0

and

f ′ = h,

Here

ψ : D → R+

is measurable and positive. Property: The faster the decay ψ(x) → 0 as x → ∞, the larger the space F1

RICAM 2018 38

SLIDE 39

G. W. Wasilkowski

∞-Variate Linear Problems

Univariate Integration:

S|F1(f) = I1(f) =

R+

f(x) ρ(x)dx

for a probability density function

ρ : R+ → R+

[Kuo, Plaskota, and W. 2016] necessary and sufficient condition for

I1F1 = C1 < ∞

and for a special quadratures to have errors ≃ n−1, i.e., r = 1

RICAM 2018 39

SLIDE 40

G. W. Wasilkowski

∞-Variate Linear Problems

Multivariate Case:

Fw is |w|-fold tensor product of F1, i.e fw(x) =

R|w|

+

h(tw)

j∈w

(xj − tj)0

+dtw,

h ψw ∈ Lp(R|w|

+ )

That is

fwFw =

ψw

j∈w

∂ ∂xj fw

Lp(R|w|

+ )

= h ψwLp(R|w|

+ )

Integration

S|Fw(fw) = Iw(fw) =

R|w|

+

fw(xw) ρw(xw)dxw.

Here

ρw(xw) =

j∈w

ρ(xj)

and

ψw(xw) =

j∈w

ψ(xj)

RICAM 2018 40

SLIDE 41

G. W. Wasilkowski

∞-Variate Linear Problems

∞-variate Integration

As in General Setting

F ∋ f

iff

f =

w

fw

and

fF =

w

γ−p

w fwp Fw

1/p < ∞.

Equivalently

fF =

w

γ−p

w

ψw f (w)([·w; 0])
p

Lp(Rw)

1/p∗ .

Integration

S(f) = I(f) =

RN

+

f ρN =

w

Iw(fw)

RICAM 2018 41

SLIDE 42

G. W. Wasilkowski

∞-Variate Linear Problems

REMARK: For unbounded domains D and some spaces F, function evaluation could be discontinuous functional. That is, for some f ∈ F and some x ∈ DN we do not have point-wise convergence of

w

fw(x)

REMEDY 1: Due to [Gnewuch, Mayer, and Ritter 2014]: Suppose that Fw are |w|-fold tensor products of a reproducing kernel Hilbert space F1 = H(k) with kernel k. Then under suitable assumption on the kernel k, there is a subset DN ⊂ DN such that

DN has measure 1, and function sampling f(x) is continuous for

every x ∈ DN.

DN has a complicated structure

and the restriction to Hilbert spaces is essential.

RICAM 2018 42

SLIDE 43

G. W. Wasilkowski

∞-Variate Linear Problems

REMEDY 2: We treat F as a Banach space of sequences

f = (fw)w⊂N

with finite

fF =

w

γ−p

w fwp Fw

1/p

We only need to assume continuity of I:

I < ∞

Sampling at x with only finitely many nonzero xj and cubatures that use only such points are well defined and continuous. It follows from Theorem 3 that the corresponding MDM have

error(Aε; F) ≤ ε

and

cost(Aε) = O

ε

− max

1 ,

1 β−1/p∗

−δ
RICAM 2018

43

SLIDE 44

G. W. Wasilkowski

∞-Variate Linear Problems

How About ANOVA Spaces?

f ∈ FAVOVA

iff

f(x) =

w fw,A(xw)

with

D fw,A(xw) dxj = 0

for any j ∈ w and, as before,

fFANOVA =

w

γ−p

w f (w) w,Ap Lp

1/p < ∞

RICAM 2018 44

SLIDE 45

G. W. Wasilkowski

∞-Variate Linear Problems

ANOVA decomposition terms fw,A cannot be sampled, i.e., low truncation dimension and MDM might not be applicable. Even worse: the ‘easiest’ (constant) term is NOT known; and it is the integral we want to approximate:

f∅,A = I(f)

RICAM 2018 45

SLIDE 46

G. W. Wasilkowski

∞-Variate Linear Problems

HOWEVER If the spaces are EQUIVALENT, then efficient algorithms for anchored spaces are also efficient for ANOVA spaces Small truncation/superposition dimension for anchored spaces are also small for ANOVA spaces. This motivated the study of

RICAM 2018 46

SLIDE 47

G. W. Wasilkowski

∞-Variate Linear Problems

Equivalence of anchored and ANOVA Spaces

For Product Weights γw =

j∈w j−β

F = FANOVA

as sets. For the imbedding ı : F ֒

→ FANOVA we have ı = ı−1 ≤

∞

j=1
1 + j−β

EQUIVALENCE iff β > 1

RICAM 2018 47

SLIDE 48

G. W. Wasilkowski

∞-Variate Linear Problems

Research direction initiated in [Hefter and Ritter 2014], Hilbert spaces setting p = 2 and product weights [Hefter, Ritter and W. 2016]

p ∈ {1, ∞} and general weights,

[Hinrichs and Schneider 2016]

p ∈ (1, ∞),

[Gnewuch, Hefter, Hinrichs, Ritter, and W. 2016] more general spaces, [Kritzer, Pillichshammer, and W. 2017] sharp lower bounds, [Hinrichs, Kritzer, Pillichshammer, and W. 2017] most general

RICAM 2018 48

SLIDE 49

G. W. Wasilkowski

∞-Variate Linear Problems

GENERALIZATIONS

General Information about f:

L1(f), L2(f), . . . , Ln(f), Lj ∈ F∗

Bayesian Approach: Endowing F with Gaussian measure PROB and studying average case errors:

F

S(f) − Alg(L1(f), . . . , Ln(f))2

G PROB(df)

r in Probabilistic Case Setting

introduced in [Wasilkowski 1986]

RICAM 2018 49

SLIDE 50

G. W. Wasilkowski

∞-Variate Linear Problems

THANK YOU FOR THE ATTENTION

RICAM 2018 50