Approximation in cosine space using tent transformed lattice rules - - PowerPoint PPT Presentation

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

Approximation in cosine space using tent transformed lattice rules

Dirk Nuyens

Department of Computer Science, KU Leuven, Belgium Joint work with Gowri Suryanarayana, Ronald Cools and Frances Kuo (UNSW). ICERM Semester Program on High-dimensional Approximation Information-Based Complexity and Stochastic Computation workshop September 15–19, 2014 Brown University, USA

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 1

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

Outline

Problem setting Rank-1 lattice & tent transformation Approximation Numerical results Conclusion

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 2

Problem setting Lattices & tent transform Approximation Numerical results Conclusion IBC setup

Problem setting

◮ High dimensional multivariate approximation

APPd : Hd → L2([0, 1]d) Hd the weighted cosine space of smoothness α > 1/2,

◮ by linear algorithm using standard information

AN,d(f )(x) =

N

• i=1

f (ti) ai(x)

◮ using a deterministic point set {ti}N i=1

tent transformed lattice points, constructive,

◮ and study the L2 worst case error.

This is a space of non-periodic functions. (Periodic case studied by Kuo, Sloan, Wo´ zniakowski (2006) + lots...) For α = 1 we get the unanchored Sobolev space of smoothness 1.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 3

Problem setting Lattices & tent transform Approximation Numerical results Conclusion Series expansion

Cosine series

◮ Orthonormal basis of L2([0, 1])

φk(x) := √ 2

|k|0 cos(πkx),

k ∈ Z+ := {0, 1, 2, . . .}.

◮ For d-dimensions, use tensor product

φk(x) := √ 2

|k|0 d

• j=1

cos(πkjxj) where |k|0 is the number of non-zero elements of k. Express f as series: f (x) =

k∈Zd

+

ˆ f (k) √ 2

|k|0 d j=1 cos(πkjxj).

For d = 1 in Iserles and Nørsett (2008), for multivariate integration studied in Dick, N. and Pillichshammer (2013). → Laplace operator, talk by Art Werschulz on Helmholtz equation, ...

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 4

Problem setting Lattices & tent transform Approximation Numerical results Conclusion Series expansion

Function expansion using cosine series

A multivariate non-periodic function f ∈ L2([0, 1]d) that is continuously differentiable can be expressed as f (x) =

• k∈Zd

+

ˆ f (k) φk(x), where ˆ f is the cosine transform of f ˆ f (k) =

• [0,1]d f (x) φk(x) dx.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 5

Problem setting Lattices & tent transform Approximation Numerical results Conclusion The function space

The weighted cosine space

We set f 2

d,α,γ :=

• k∈Zd

+

|ˆ f (k)|2 rα,γ(k) and define the (half-period) cosine space by Cd,α,γ :=

• f ∈ L2([0, 1]d) : f d,α,γ < ∞
• ,

with α > 1/2, weights 1 ≥ γ1 ≥ γ2 ≥ · · · > 0 and rα,γj(kj) :=

• 1,

if kj = 0, k2α

j /γj,

if kj > 0, rα,γ(k) :=

d

• j=1

rα,γj(kj).

(For numerical integration: Dick, N. and Pillichshammer (2013).) The weights control tractability, see Sloan & Wo´ zniakowski, Hickernell, Novak & Wo´ zniakowski, ...

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 6

Problem setting Lattices & tent transform Approximation Numerical results Conclusion The function space

Reproducing kernel

It is a reproducing kernel Hilbert space with reproducing kernel Kd(x, y) = d

j=1 Kj(xj, yj), where

Kj(xj, yj) = 1 + 2

• k≥1

r−1

α,γj(k) cos(πkxj) cos(πkyj).

For α = 1 and γj → π−2γj we obtain the unanchored Sobolev space of dominating mixed smoothness 1, i.e., for d = 1 1+2

• k≥1

γ π2k2 cos(πkx) cos(πky) = 1+γB1(x)B1(y)+γ B2(|x − y|) 2 (see DNP2013) with inner product given by f , g = 1 f (x) dx 1 g(x) dx + 1 γ 1 f ′(x)g′(x) dx.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 7

Problem setting Lattices & tent transform Approximation Numerical results Conclusion The algorithm

The approximation algorithm

Truncate cosine series expansion to k ∈ H d,α,γ

T

, then approximate those cosine coefficients by an N-point tent transformed rank-1 lattice rule with points Λϕ(z, N): AN,d,T(f )(x) :=

• k∈H d,α,γ

T

1 N

• t∈Λϕ(z,N)

f (t) φk(t)

• φk(x).

Here H d,α,γ

T

is a weighted hyperbolic cross, for T ≥ 0, H d,α,γ

T

:=

• k ∈ Zd

+ : rα,γ(k) ≤ T

• ,

and, for α > 1/2 and positive weights γj, and as before rα,γj(kj) =

• 1,

if kj = 0, k2α

j /γj,

if kj > 0, rα,γ(k) =

d

• j=1

rα,γj(kj).

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 8

Problem setting Lattices & tent transform Approximation Numerical results Conclusion Worst case error

Worst case error

The approximation error at a point x is then (f − AN,d,T(f ))(x) =

• k∈H d,α,γ

T

ˆ f (k) φk(x) +

• k∈H d,α,γ

T

• ˆ

f (k) − 1 N

• t∈Λϕ(z,N)

f (t) φk(t)

• φk(x).

The worst case error of the algorithm AN,d,T is then given as ewor

n,d (AN,d,T; Cd,α,γ) :=

sup

f ∈Cd,α,γ f d,α,γ≤1

f − AN,d,T(f )L2([0,1]d).

Many references from people here present... Too many to list! Same setup as in Kuo, Sloan, Wo´ zniakowski (2006) for Fourier series.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 9

Problem setting Lattices & tent transform Approximation Numerical results Conclusion Rank-1 lattice

Lattice points

For given number of points N ≥ 1 and z ∈ Zd

N a rank-1 lattice is

defined as Λ(z, N) := zk N mod 1 : k = 0, 1, . . . , N − 1

• ,

and z is called the generating vector.

(Sloan & Joe, Korobov, Temlyakov, ... Mostly for integration.)

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Rank-1 lattice points with N = 34, z = [1, 21] Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 10

Problem setting Lattices & tent transform Approximation Numerical results Conclusion Tent transform

Tent transformation

Tent transformed lattice points Λϕ(z, N) (a multiset):

◮ Tent transformation, ϕ : [0, 1] → [0, 1] is given by

ϕ(x) := 1 − |2x − 1|.

◮ Applied to each coordinate separately. ◮ Brings symmetry to the sampling points. ◮ Then for any k ∈ Zd +:

√ 2

|k|0 d j=1 cos(πkjϕ(xj)) =

√ 2

|k|0 d j=1 cos(2πkjxj).

This is the “baker’s transform” in Hickernell (2000).

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 11

Problem setting Lattices & tent transform Approximation Numerical results Conclusion Tent transform

Tent transformation

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Rank-1 lattice points with N = 34, z = [1, 21] 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Tent transformed lattice points (18 points)

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 12

Problem setting Lattices & tent transform Approximation Numerical results Conclusion Two cases

Approximation for two cases

We consider two different cases:

• 1. Find generating vector which “approximates exactly” for f

with frequency support on a hyperbolic cross of “degree” T.

• 2. Find generating vector which minimises L2 worst case

error.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 13

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 1. Hyperbolic cross support

Functions with support on a hyperbolic cross

Suppose f has frequency support limited to a hyperbolic cross H d,β

T

= H d,1/2,β

T

⊆ Zd

+ then

f (x) =

• k∈Zd

+

ˆ f (k) φk(x) =

• k∈H d,β

T

ˆ f (k) φk(x).

5 10 15 20 5 10 15 20 H 2,{1,1}

20

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 14

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 1. Hyperbolic cross support

Functions with support on a hyperbolic cross

We have a weighted hyperbolic cross given by H d,β

T

= H d,1/2,β

T

:=

• k ∈ Zd

+ : d

• j=1

max

• 1, kj

βj

• ≤ T
• with T ≥ 0, α = 1/2 (which is allowed as we now have finite

sums), and weights β1 ≥ β2 ≥ · · · > 0.

5 10 15 20 2 4 6 8 10 H

2,{1, 1

2}

20

“Degrees of exactness” for integration from Cools, Kuo, N. (2000). Picked up by Kämmerer et al for approximation on hyperbolic cross.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 15

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 1. Hyperbolic cross support

What do we need?

We need to exactly calculate the cosine coefficients ∀k ∈ H d,β

T

ˆ f (k) =

• [0,1]d f (x) φk(x) dx

using our tent transformed lattice rule Λϕ(z, N), thus ˆ f a(k) = 1 N

• t∈Λϕ(z,N)

f (t) φk(t) = 1 N

• t∈Λϕ(z,N)

ℓ∈H d,β

T

ˆ f (ℓ) φℓ(t)

• φk(t)

=

• ℓ∈H d,β

T

ˆ f (ℓ) 1 N

• t∈Λϕ(z,N)

φℓ(t)φk(t)

• =δk,ℓ

≡ ˆ f (k).

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 16

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 1. Hyperbolic cross support

Discrete orthogonality

Obviously we need the discrete orthogonality of the basis functions w.r.t. our cubature rule. Good news:

Lemma

We have ∀k, ℓ ∈ H d,β

T

δk,ℓ = 1 N

• t∈Λ(z,N)

( √ 2)|ℓ|0+|k|0 22d

• σ,σ′∈{±1}d

exp(2πi (σ(ℓ) − σ′(k)) · t). We have the original lattice points acting on the Fourier basis! Kämmerer (2013) has the similar condition ∀k, ℓ ∈ H d,β

T

: 1 N

• t∈Λ(z,N)

exp(2πi (ℓ − k) · t) = δk,ℓ.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 17

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 1. Hyperbolic cross support

The difference sets

Define the difference sets Dd,β

T

:=

• σ(ℓ) − σ′(k) ∈ Zd : ℓ, k ∈ H d,β

T

⊂ Zd

+, ∀σ, σ′ ∈ {±1}d

• ,
• Dd,β

T

:=

• ℓ − k ∈ Zd : ℓ, k ∈

H d,β

T

⊂ Zd

• .

It now follows that discrete orthogonality in the cosine space is equivalent as discrete orthogonality in the periodic space for the original lattice points since Dd,β

T

= Dd,β

T

.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 18

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 1. Hyperbolic cross support

The difference sets

2 4 6 8 10 12 14 16 18 20 1 2 3 4 5 6 7 8 9 10

H

2,{ 1

2 , 1 4 }

40

−40 −30 −20 −10 10 20 30 40 −20 −15 −10 −5 5 10 15 20

D

2,{ 1

2 , 1 4 }

40

−20 −15 −10 −5 5 10 15 20 −10 −8 −6 −4 −2 2 4 6 8 10

• H

2,{ 1

2 , 1 4 }

40

−40 −30 −20 −10 10 20 30 40 −20 −15 −10 −5 5 10 15 20

• D

2,{ 1

2 , 1 4 }

40

Figure: Hyperbolic crosses and the difference sets for the cosine and Fourier series.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 19

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 1. Hyperbolic cross support

Component-By-Component search

Theorem

Given d ≥ 2, T ≥ 1, β and prime N s.t. N ≥ | Dd,β

T

| − | Dd−1,β

T

| − 4⌊βdT⌋ + 4 2 and assume there exists a rank-1 lattice Λ(z∗, N), z∗ ∈ Zd−1 and k · z∗ ≡ 0 (mod N) for all k ∈ Dd−1,β

T

\{0} , then there exists a zd ∈ {1, . . . , N − 1} such that k · z∗ + kd · zd ≡ 0 (mod N) ∀k ∈ Dd,β

T

\{0}.

Theorem from Kämmerer (2013), similar to the bound for integration in Cools, Kuo, and N. (2010).

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 20

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 1. Hyperbolic cross support

Fast approximation using 1D FFT

Algorithm 1 Input: null N prime, z ∈ Zd, d ≥ 1, H d,β

T

, f = N −1 f (ϕ( jz

N mod 1))

N−1

j=0

ϕ is the tent transform function

ˆ g = FFT_1D(f ) for each k ∈ H d,β

T

do ˜ fk = ˆ gk·z mod N

(Aliased) Fourier coefficients

for each k ∈ H d,β

T

do ˆ f k =

√ 2

|k|0

2|k|0

• σ∈{±1}2|k|0

˜ fσ(k)

(Aliased) cosine coefficients

Output: ˆ f → Now on to unlimited support...

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 21

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 2. Approximation in Cd,α,γ

Worst case error for general f ∈ Cd,α,γ

Remember, the approximation error (f − AN,d,T(f ))(x) =

• k∈H d,α,γ

T

ˆ f (k) φk(x) +

• k∈H d,α,γ

T

• ˆ

f (k) − 1 N

• t∈Λϕ(z,N)

f (t) φk(t)

• φk(x),

and the worst case error of the algorithm AN,d,T ewor

n,d (AN,d,T; Cd,α,γ) =

sup

f ∈Cd,α,γ f d,α,γ≤1

f − AN,d,T(f )L2([0,1]d).

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 22

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 2. Approximation in Cd,α,γ

Worst case error

The L2 error in approximating f is f − AN,d,T(f )2

L2([0,1]d) =

• k∈H d,α,γ

T

|ˆ f (k)|2 +

• k∈H d,α,γ

T

• 0=h∈Λ(z,N)⊥
• σ∈{±1}d

ˆ f (|σ(h)+k|)( √ 2)−|σ(h)+k|0+|k|0 2d

• 2

, where Λ(z, N)⊥ is the dual set of the lattice point set Λ(z, N). The truncation error is bounded as follows

• k∈H d,α,γ

T

|ˆ f (k)|2 < 1 T f 2

d,α,γ where f 2 d,α,γ =

• k∈Zd

+

|ˆ f (k)|2 rα,γ(k).

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 23

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 2. Approximation in Cd,α,γ

Worst case error

The error from aliasing is

• k∈H d,α,γ

T

• 0=h∈Λ(z,N)⊥
• σ∈{±1}d

ˆ f (|σ(h) + k|)( √ 2)−|σ(h)+k|0+|k|0 2d

• 2

≤ f 2

d,α,γ

• k∈H d,α,γ

T

• 0=h∈Λ(z,N)⊥
• σ∈{±1}d

2|k|0 2drα,γ(σ(h) + k). This can further be simplified as follows

• k∈H d,α,γ

T

• 0=h∈Λ(z,N)⊥
• σ∈{±1}d

2|k|0 2drα,γ(σ(h) + k) =

• k∈

H d,α,γ

T

• 0=h∈Λ(z,N)⊥

1 rα,γ(h + k), which is the worst case error in the periodic space. Hurray! → KSW2006

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 24

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 2. Approximation in Cd,α,γ

Component-by-Component search

From Kuo, Sloan & Wo´ zniakowski (2006): Let N be prime and T ≥ 1

◮ Set z1 = 1. ◮ For s = 2, 3, . . . , d, find zs in {1, 2, . . . , N − 1} to minimize

EN,s,T(zs; z1, . . . , zs−1) =

• k∈

H s,α,γ

T

• 0=h∈Λ(z,N)⊥

1 rα,γ(h + k).

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 25

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

• 2. Approximation in Cd,α,γ

Tractability results

Let p∗ = 2 max

• 1

2α, 1 α0

• where

α0 = decay(γ) := inf{s > 0 : ∞

j=1 γ1/s j

< ∞}. Then strong tractability holds

◮ if ∞ j=1 γj < ∞, α0 ≤ 1, and ◮ z∗ CBC constructed for appropriately chosen N and T(N).

Then the worst case error ewor

n,d (AN,d,T; Cd,α,γ) ≤ ǫ, for any ǫ > 0,

using N = O(ǫ−p) function values where p can be arbitrarily close to 2p∗.

This is as in Kuo, Sloan and Wo´ zniakowski (2006). For polynomial tractability the condition on the weights is a := lim supd→∞

∞

j=1 γj

log(d+1) < ∞ with N = O(ǫ−4dq) function values

where q can be arbitrarily close to 4ζ(2α)a.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 26

Problem setting Lattices & tent transform Approximation Numerical results Conclusion Cosine versus Fourier (using rules for degree T)

Approximation: cosine series versus Fourier series

103 104 10−7 10−6 10−5 10−4 10−3 10−2 10−1 100 N f − faL2 / f L2 d = 3 with Fourier series: O(N −0.2) with cosine series: O(N −1.8) 104 105 10−6 10−5 10−4 10−3 10−2 10−1 100 N f − faL2 / f L2 d = 4 with Fourier series: O(N −0.2) with cosine series: O(N −1.6)

Figure: Convergence plots for the relative L2 approximation errors in 3 and 4 dimensions for f (x) = d

j=1

• x3

j /3 − x2 j /2

• .

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 27

Problem setting Lattices & tent transform Approximation Numerical results Conclusion A collocation problem (using rules for degree T)

Collocation

102.5 103 103.5 10−2 10−1 N d = 3, γj = 1, βj = 1 truncationL2 / uL2 aliasingL2 / uL2 102 103 104 10−2 10−1 N d = 5, γj = 81−j, βj = 8

1−j 6

truncationL2 / uL2 aliasingL2 / uL2

Figure: Convergence plots for the relative L2 approximation errors for d = 3: O(N −1.3), and d = 5: O(N −1.1) for ∇2u(x) = d

j=1 γj(12x2 j −12xj +2) d j=i=1

1

630 +γi

• x2

i (1−xi)2 − 1 630

• .

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 28

Problem setting Lattices & tent transform Approximation Numerical results Conclusion

Conclusions

◮ Tent transformed rank-1 lattice points are suitable as

sampling points in the approximation of non-periodic functions (when expanded in terms of cosine series).

◮ We presented two constructive methods to construct such

lattice point sets.

◮ What about error analysis for the cosine space in the

average case setting?

◮ What about approximation of functions of higher

smoothness (as in derivatives)?

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 29

References

◮ L. Kämmerer, S. Kunis, and D. Potts. Interpolation lattices for hyperbolic cross trigonometric polynomials. Journal of Complexity, 2012. ◮ D. Li and F . J. Hickernell. Trigonometric spectral collocation methods on

• lattices. Contemporary Mathematics, 2003.

◮ R. Cools, F . Y. Kuo, and D. Nuyens. Constructing lattice rules based on weighted degree of exactness and worst case error. Computing, 2010. ◮ B. J. S. Adcock. Modified Fourier expansions: Theory, construction and

• applications. PhD thesis, University of Cambridge, 2010.

◮ J. Dick, D. Nuyens, and F . Pillichshammer. Lattice rules for nonperiodic smooth integrands, Numerische Mathematik, 2013. ◮ A. Iserles and S. P . Nørsett. From high oscillation to rapid approximation I: Modified Fourier expansions. IMA Journal of Numerical Analysis, 2008. ◮ L. Kämmerer. Reconstructing hyperbolic cross trigonometric polynomials by sampling along rank-1 lattices, SIAM Journal on Numerical Analysis, 2013. ◮ F . J. Hickernell. Obtaining O(n−2+ǫ) convergence for lattice quadrature rules. Monte Carlo and Quasi-Monte Carlo Methods 2000, 2002. ◮ I. H. Sloan and H. Wo´

• zniakowski. When are quasi-Monte Carlo algorithms

efficient for high dimensional integrals? Journal of Complexity, 1998. ◮ G. Suryanarayana, D. Nuyens and R. Cools. Reconstruction and collocation of a class of non-periodic functions by sampling along tent-transformed rank-1

• lattices. Submitted.

Approximation in cosine space using tent transformed lattice rules / Dirk Nuyens 30