Computation in High Dimensions Ronald DeVore Collaborators: Peter - - PowerPoint PPT Presentation

Computation in High Dimensions

Ronald DeVore

Collaborators: Peter Binev, Andrea Bonito, Albert Cohen, Wolfgang Dahmen, Bojan Popov, Guergana Petrova, Przemek Wojtaszczyk

Toulouse – p. 1/24

High Dimensional Numerics

Querying/Approximating/Computing functions of many variables is ubiquitous and difficult

Toulouse – p. 2/24

High Dimensional Numerics

Querying/Approximating/Computing functions of many variables is ubiquitous and difficult Quering Large Data

Toulouse – p. 2/24

High Dimensional Numerics

Querying/Approximating/Computing functions of many variables is ubiquitous and difficult Quering Large Data Stochastic Learning

Toulouse – p. 2/24

High Dimensional Numerics

Querying/Approximating/Computing functions of many variables is ubiquitous and difficult Quering Large Data Stochastic Learning Parametric/Stochastic PDEs

Toulouse – p. 2/24

High Dimensional Numerics

Querying/Approximating/Computing functions of many variables is ubiquitous and difficult Quering Large Data Stochastic Learning Parametric/Stochastic PDEs Complex physical modeling

Toulouse – p. 2/24

High Dimensional Numerics

Querying/Approximating/Computing functions of many variables is ubiquitous and difficult Quering Large Data Stochastic Learning Parametric/Stochastic PDEs Complex physical modeling Obstacle: Curse of Dimensionality

Toulouse – p. 2/24

High Dimensional Numerics

Querying/Approximating/Computing functions of many variables is ubiquitous and difficult Quering Large Data Stochastic Learning Parametric/Stochastic PDEs Complex physical modeling Obstacle: Curse of Dimensionality Traditional methods based solely on smoothness cannot work

Toulouse – p. 2/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation

Toulouse – p. 3/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation Sparsity

Toulouse – p. 3/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation Sparsity There is a basis (ψλ)λ∈Λ

Toulouse – p. 3/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation Sparsity There is a basis (ψλ)λ∈Λ

u =

λ∈Λ cλψλ

Toulouse – p. 3/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation Sparsity There is a basis (ψλ)λ∈Λ

u =

λ∈Λ cλψλ

(cλ)λ∈Λ, when rearranged in decreasing order,

decays fast, e.g. in ℓp for some small value of p.

Toulouse – p. 3/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation Sparsity There is a basis (ψλ)λ∈Λ

u =

λ∈Λ cλψλ

(cλ)λ∈Λ, when rearranged in decreasing order,

decays fast, e.g. in ℓp for some small value of p. Variable reduction

Toulouse – p. 3/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation Sparsity There is a basis (ψλ)λ∈Λ

u =

λ∈Λ cλψλ

(cλ)λ∈Λ, when rearranged in decreasing order,

decays fast, e.g. in ℓp for some small value of p. Variable reduction Variables not democratic: some are more important than others

Toulouse – p. 3/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation Sparsity There is a basis (ψλ)λ∈Λ

u =

λ∈Λ cλψλ

(cλ)λ∈Λ, when rearranged in decreasing order,

decays fast, e.g. in ℓp for some small value of p. Variable reduction Variables not democratic: some are more important than others Anisotropy

Toulouse – p. 3/24

How to Avoid the Curse

The functions u we want to capture must have other properties that make them amenable to computation Sparsity There is a basis (ψλ)λ∈Λ

u =

λ∈Λ cλψλ

(cλ)λ∈Λ, when rearranged in decreasing order,

decays fast, e.g. in ℓp for some small value of p. Variable reduction Variables not democratic: some are more important than others Anisotropy I want to bring out a few general principles that have had some success in breaking the curse. Given time constraints and the interest of this audience I will restrtict my discussion to Parametric/Stochastic PDES

Toulouse – p. 3/24

The Problem

D ⊂ I Rd is a physical domain and A is a collection of

diffusion coefficients a that satisfy the Uniform Ellipticity Assumption

UEA : 0 < r ≤ a(x) ≤ R, x ∈ D, for all a ∈ A

Toulouse – p. 4/24

The Problem

D ⊂ I Rd is a physical domain and A is a collection of

diffusion coefficients a that satisfy the Uniform Ellipticity Assumption

UEA : 0 < r ≤ a(x) ≤ R, x ∈ D, for all a ∈ A

For each a ∈ A we are interested in the solution ua to the elliptic problem

(∗) − div(a(x)∇ua(x)) = f(x), x ∈ D, ua(x) = 0, x ∈ ∂D

Toulouse – p. 4/24

The Problem

D ⊂ I Rd is a physical domain and A is a collection of

diffusion coefficients a that satisfy the Uniform Ellipticity Assumption

UEA : 0 < r ≤ a(x) ≤ R, x ∈ D, for all a ∈ A

For each a ∈ A we are interested in the solution ua to the elliptic problem

(∗) − div(a(x)∇ua(x)) = f(x), x ∈ D, ua(x) = 0, x ∈ ∂D

Note all energy spaces H1

0(a) are equivalent to H1 0(1)

Toulouse – p. 4/24

The Problem

D ⊂ I Rd is a physical domain and A is a collection of

diffusion coefficients a that satisfy the Uniform Ellipticity Assumption

UEA : 0 < r ≤ a(x) ≤ R, x ∈ D, for all a ∈ A

For each a ∈ A we are interested in the solution ua to the elliptic problem

(∗) − div(a(x)∇ua(x)) = f(x), x ∈ D, ua(x) = 0, x ∈ ∂D

Note all energy spaces H1

0(a) are equivalent to H1 0(1)

Let

K := K(f, A) = {ua : a ∈ A}

Toulouse – p. 4/24

The Problem

D ⊂ I Rd is a physical domain and A is a collection of

diffusion coefficients a that satisfy the Uniform Ellipticity Assumption

UEA : 0 < r ≤ a(x) ≤ R, x ∈ D, for all a ∈ A

For each a ∈ A we are interested in the solution ua to the elliptic problem

(∗) − div(a(x)∇ua(x)) = f(x), x ∈ D, ua(x) = 0, x ∈ ∂D

Note all energy spaces H1

0(a) are equivalent to H1 0(1)

Let

K := K(f, A) = {ua : a ∈ A}

We are interested in a fast on line method that will compute ua given any query a ∈ A

Toulouse – p. 4/24

The AFFINE MODEL

a(x, y) = ¯ a(x) + ∞

j=1 yjψj(x), yj ∈ [−1, 1], j = 1, 2, . . .

Toulouse – p. 5/24

The AFFINE MODEL

a(x, y) = ¯ a(x) + ∞

j=1 yjψj(x), yj ∈ [−1, 1], j = 1, 2, . . .

Here we put the normalization into the yj and so behavior of ψjL∞(D) will be crucial. We assume that we reorder indices so that (ψjL∞(D)) is a decreasing sequence

Toulouse – p. 5/24

The AFFINE MODEL

a(x, y) = ¯ a(x) + ∞

j=1 yjψj(x), yj ∈ [−1, 1], j = 1, 2, . . .

Here we put the normalization into the yj and so behavior of ψjL∞(D) will be crucial. We assume that we reorder indices so that (ψjL∞(D)) is a decreasing sequence Write u(x, y) := ua(x)

Toulouse – p. 5/24

The AFFINE MODEL

a(x, y) = ¯ a(x) + ∞

j=1 yjψj(x), yj ∈ [−1, 1], j = 1, 2, . . .

Here we put the normalization into the yj and so behavior of ψjL∞(D) will be crucial. We assume that we reorder indices so that (ψjL∞(D)) is a decreasing sequence Write u(x, y) := ua(x) Stochastic setting

Toulouse – p. 5/24

The AFFINE MODEL

a(x, y) = ¯ a(x) + ∞

j=1 yjψj(x), yj ∈ [−1, 1], j = 1, 2, . . .

Here we put the normalization into the yj and so behavior of ψjL∞(D) will be crucial. We assume that we reorder indices so that (ψjL∞(D)) is a decreasing sequence Write u(x, y) := ua(x) Stochastic setting

a(·, ω) is an L∞(Ω) valued random variable on (Ω, ρ)

Toulouse – p. 5/24

The AFFINE MODEL

a(x, y) = ¯ a(x) + ∞

j=1 yjψj(x), yj ∈ [−1, 1], j = 1, 2, . . .

Here we put the normalization into the yj and so behavior of ψjL∞(D) will be crucial. We assume that we reorder indices so that (ψjL∞(D)) is a decreasing sequence Write u(x, y) := ua(x) Stochastic setting

a(·, ω) is an L∞(Ω) valued random variable on (Ω, ρ)

Wiener chaos: Choose basis (ψj)

a(x, ω) = ¯ a(x) + ∞

k=1 yk(ω)ψk(x)

Toulouse – p. 5/24

The AFFINE MODEL

a(x, y) = ¯ a(x) + ∞

j=1 yjψj(x), yj ∈ [−1, 1], j = 1, 2, . . .

Here we put the normalization into the yj and so behavior of ψjL∞(D) will be crucial. We assume that we reorder indices so that (ψjL∞(D)) is a decreasing sequence Write u(x, y) := ua(x) Stochastic setting

a(·, ω) is an L∞(Ω) valued random variable on (Ω, ρ)

Wiener chaos: Choose basis (ψj)

a(x, ω) = ¯ a(x) + ∞

k=1 yk(ω)ψk(x)

Normalized so that ykL∞(Ω) = 1

Toulouse – p. 5/24

The AFFINE MODEL

a(x, y) = ¯ a(x) + ∞

j=1 yjψj(x), yj ∈ [−1, 1], j = 1, 2, . . .

Here we put the normalization into the yj and so behavior of ψjL∞(D) will be crucial. We assume that we reorder indices so that (ψjL∞(D)) is a decreasing sequence Write u(x, y) := ua(x) Stochastic setting

a(·, ω) is an L∞(Ω) valued random variable on (Ω, ρ)

Wiener chaos: Choose basis (ψj)

a(x, ω) = ¯ a(x) + ∞

k=1 yk(ω)ψk(x)

Normalized so that ykL∞(Ω) = 1 This is embedded in the affine representation: However the role of the probability measure is lost

Toulouse – p. 5/24

NUMERICAL GOALS

PARAMETRIC GOAL: Given a query a ∈ A, quickly compute a good H1

0(D, a) approximation to ua.

Toulouse – p. 6/24

NUMERICAL GOALS

PARAMETRIC GOAL: Given a query a ∈ A, quickly compute a good H1

0(D, a) approximation to ua.

We want an approximation to the solution map

F : a → ua or F : y → u(·, y)

Toulouse – p. 6/24

NUMERICAL GOALS

PARAMETRIC GOAL: Given a query a ∈ A, quickly compute a good H1

0(D, a) approximation to ua.

We want an approximation to the solution map

F : a → ua or F : y → u(·, y)

Given a query a ∈ A quickly evaluate F(a) (F(y))

Toulouse – p. 6/24

NUMERICAL GOALS

PARAMETRIC GOAL: Given a query a ∈ A, quickly compute a good H1

0(D, a) approximation to ua.

We want an approximation to the solution map

F : a → ua or F : y → u(·, y)

Given a query a ∈ A quickly evaluate F(a) (F(y))

F is Banach space valued function of many (possibly

infinitely many) variables (parameters)

Toulouse – p. 6/24

NUMERICAL GOALS

PARAMETRIC GOAL: Given a query a ∈ A, quickly compute a good H1

0(D, a) approximation to ua.

We want an approximation to the solution map

F : a → ua or F : y → u(·, y)

Given a query a ∈ A quickly evaluate F(a) (F(y))

F is Banach space valued function of many (possibly

infinitely many) variables (parameters) STOCHASTIC GOAL: F is a Banach space valued random variable

Toulouse – p. 6/24

NUMERICAL GOALS

PARAMETRIC GOAL: Given a query a ∈ A, quickly compute a good H1

0(D, a) approximation to ua.

We want an approximation to the solution map

F : a → ua or F : y → u(·, y)

Given a query a ∈ A quickly evaluate F(a) (F(y))

F is Banach space valued function of many (possibly

infinitely many) variables (parameters) STOCHASTIC GOAL: F is a Banach space valued random variable Compute stochastic properties of F such as mean, variance, higher moments

Toulouse – p. 6/24

NUMERICAL GOALS

PARAMETRIC GOAL: Given a query a ∈ A, quickly compute a good H1

0(D, a) approximation to ua.

We want an approximation to the solution map

F : a → ua or F : y → u(·, y)

Given a query a ∈ A quickly evaluate F(a) (F(y))

F is Banach space valued function of many (possibly

infinitely many) variables (parameters) STOCHASTIC GOAL: F is a Banach space valued random variable Compute stochastic properties of F such as mean, variance, higher moments These are referred to as Quantities of Interest

Toulouse – p. 6/24

Two Strategies

If we do nothing but call on a standard (adaptive) FEM solver then for n computations we would get acuracy

O(n−α) where α is related to the Besov smoothness of ua with respect to the physical domain variable x

Toulouse – p. 7/24

Two Strategies

If we do nothing but call on a standard (adaptive) FEM solver then for n computations we would get acuracy

O(n−α) where α is related to the Besov smoothness of ua with respect to the physical domain variable x

We hope to improve this rate by another method by taking advantage of the smoothness of the manifold K.

Toulouse – p. 7/24

Two Strategies

If we do nothing but call on a standard (adaptive) FEM solver then for n computations we would get acuracy

O(n−α) where α is related to the Besov smoothness of ua with respect to the physical domain variable x

We hope to improve this rate by another method by taking advantage of the smoothness of the manifold K. For example, can we take a few snapshots uai,

i = 1, . . . , m of the manifold K and use these in some

sort of interpolation formula to compute ua for any given query a?

Toulouse – p. 7/24

Two Strategies

If we do nothing but call on a standard (adaptive) FEM solver then for n computations we would get acuracy

O(n−α) where α is related to the Besov smoothness of ua with respect to the physical domain variable x

We hope to improve this rate by another method by taking advantage of the smoothness of the manifold K. For example, can we take a few snapshots uai,

i = 1, . . . , m of the manifold K and use these in some

sort of interpolation formula to compute ua for any given query a? If so, where should we take the snapshots ?

Toulouse – p. 7/24

Two Strategies

If we do nothing but call on a standard (adaptive) FEM solver then for n computations we would get acuracy

O(n−α) where α is related to the Besov smoothness of ua with respect to the physical domain variable x

We hope to improve this rate by another method by taking advantage of the smoothness of the manifold K. For example, can we take a few snapshots uai,

i = 1, . . . , m of the manifold K and use these in some

sort of interpolation formula to compute ua for any given query a? If so, where should we take the snapshots ? What is the improved accuracy?

Toulouse – p. 7/24

Smoothness of K

For reduced modeling to work, the smoothness of K is critical

Toulouse – p. 8/24

Smoothness of K

For reduced modeling to work, the smoothness of K is critical Classical perturbation theorem gives

ua − u˜

aH1

0 ≤ C0fH−1a − ˜

aL∞

Toulouse – p. 8/24

Smoothness of K

For reduced modeling to work, the smoothness of K is critical Classical perturbation theorem gives

ua − u˜

aH1

0 ≤ C0fH−1a − ˜

aL∞

Actually much more is true

Toulouse – p. 8/24

Smoothness of K

For reduced modeling to work, the smoothness of K is critical Classical perturbation theorem gives

ua − u˜

aH1

0 ≤ C0fH−1a − ˜

aL∞

Actually much more is true In the affine case, the function F is an analytic (infinitely differentiable) function of its parameters

Toulouse – p. 8/24

Smoothness of K

For reduced modeling to work, the smoothness of K is critical Classical perturbation theorem gives

ua − u˜

aH1

0 ≤ C0fH−1a − ˜

aL∞

Actually much more is true In the affine case, the function F is an analytic (infinitely differentiable) function of its parameters Even in the non-affine case, there are principles of analyticity

Toulouse – p. 8/24

Smoothness of K

For reduced modeling to work, the smoothness of K is critical Classical perturbation theorem gives

ua − u˜

aH1

0 ≤ C0fH−1a − ˜

aL∞

Actually much more is true In the affine case, the function F is an analytic (infinitely differentiable) function of its parameters Even in the non-affine case, there are principles of analyticity But let us emphasize that infinite differentiability, in and of itself, is not enough in high dimensions, as we now discuss

Toulouse – p. 8/24

Curse of Dimensionality

Since we are wanting to approximate a function (F) of many variables we have to be wary of the “Curse”

Toulouse – p. 9/24

Curse of Dimensionality

Since we are wanting to approximate a function (F) of many variables we have to be wary of the “Curse” “Curse of Dimensionality” says classical approaches will not work

Toulouse – p. 9/24

Curse of Dimensionality

Since we are wanting to approximate a function (F) of many variables we have to be wary of the “Curse” “Curse of Dimensionality” says classical approaches will not work Classical methods of approximation are based on the assumption that F has smoothness (of order s)

Toulouse – p. 9/24

Curse of Dimensionality

Since we are wanting to approximate a function (F) of many variables we have to be wary of the “Curse” “Curse of Dimensionality” says classical approaches will not work Classical methods of approximation are based on the assumption that F has smoothness (of order s) then with n computations we can only capture F to accuracy Cmn−s/m where m is the number of variables

Toulouse – p. 9/24

Curse of Dimensionality

Since we are wanting to approximate a function (F) of many variables we have to be wary of the “Curse” “Curse of Dimensionality” says classical approaches will not work Classical methods of approximation are based on the assumption that F has smoothness (of order s) then with n computations we can only capture F to accuracy Cmn−s/m where m is the number of variables When m is large than Cm is exponentialy large

Toulouse – p. 9/24

Curse of Dimensionality

Since we are wanting to approximate a function (F) of many variables we have to be wary of the “Curse” “Curse of Dimensionality” says classical approaches will not work Classical methods of approximation are based on the assumption that F has smoothness (of order s) then with n computations we can only capture F to accuracy Cmn−s/m where m is the number of variables When m is large than Cm is exponentialy large Even when F is analytic the curse rears its ugly head

Toulouse – p. 9/24

Example (Nowak-Wozniakowski)

To drive home the debilitating effect of this curse consider the following example of scalar valued functions

Toulouse – p. 10/24

Example (Nowak-Wozniakowski)

To drive home the debilitating effect of this curse consider the following example of scalar valued functions

Ω := [0, 1]m, W := {w : DνwL∞ ≤ 1, ∀ν}

Toulouse – p. 10/24

Example (Nowak-Wozniakowski)

To drive home the debilitating effect of this curse consider the following example of scalar valued functions

Ω := [0, 1]m, W := {w : DνwL∞ ≤ 1, ∀ν}

For any subspace V of dimension 2m/2 we have

dist(W, V )L∞(Ω) ≥ 1/2

Toulouse – p. 10/24

Example (Nowak-Wozniakowski)

To drive home the debilitating effect of this curse consider the following example of scalar valued functions

Ω := [0, 1]m, W := {w : DνwL∞ ≤ 1, ∀ν}

For any subspace V of dimension 2m/2 we have

dist(W, V )L∞(Ω) ≥ 1/2

So if m = 100 we would need 250 ≈ 1015 queries or computations to approximate a general w ∈ W with even the crudest of accuracy

Toulouse – p. 10/24

Example (Nowak-Wozniakowski)

To drive home the debilitating effect of this curse consider the following example of scalar valued functions

Ω := [0, 1]m, W := {w : DνwL∞ ≤ 1, ∀ν}

For any subspace V of dimension 2m/2 we have

dist(W, V )L∞(Ω) ≥ 1/2

So if m = 100 we would need 250 ≈ 1015 queries or computations to approximate a general w ∈ W with even the crudest of accuracy We will see that for reduced modeling the key is that not

• nly is our function F very smooth but in addition the

parameters are not democratic

Toulouse – p. 10/24

Example (Nowak-Wozniakowski)

To drive home the debilitating effect of this curse consider the following example of scalar valued functions

Ω := [0, 1]m, W := {w : DνwL∞ ≤ 1, ∀ν}

For any subspace V of dimension 2m/2 we have

dist(W, V )L∞(Ω) ≥ 1/2

So if m = 100 we would need 250 ≈ 1015 queries or computations to approximate a general w ∈ W with even the crudest of accuracy We will see that for reduced modeling the key is that not

• nly is our function F very smooth but in addition the

parameters are not democratic This gets reflected in a certain decay for derivatives

Toulouse – p. 10/24

Reduced Basis Methods

Basic Steps

Toulouse – p. 11/24

Reduced Basis Methods

Basic Steps Find a good low dimensional linear space

Vn ⊂ H1

0(D) to approximate the elements of K

Toulouse – p. 11/24

Reduced Basis Methods

Basic Steps Find a good low dimensional linear space

Vn ⊂ H1

0(D) to approximate the elements of K

Recall K is the range of F

Toulouse – p. 11/24

Reduced Basis Methods

Basic Steps Find a good low dimensional linear space

Vn ⊂ H1

0(D) to approximate the elements of K

Recall K is the range of F Given a query a find a numerical method to approximate ua from Vn

Toulouse – p. 11/24

Reduced Basis Methods

Basic Steps Find a good low dimensional linear space

Vn ⊂ H1

0(D) to approximate the elements of K

Recall K is the range of F Given a query a find a numerical method to approximate ua from Vn The latter is done by some projection onto Vn

Toulouse – p. 11/24

Implementation

Finding a good space Vn is an off line computation

Toulouse – p. 12/24

Implementation

Finding a good space Vn is an off line computation In practice, this space is usually generated by snapshots uai of K

Toulouse – p. 12/24

Implementation

Finding a good space Vn is an off line computation In practice, this space is usually generated by snapshots uai of K Computing the approximation of ua from Vn

Toulouse – p. 12/24

Implementation

Finding a good space Vn is an off line computation In practice, this space is usually generated by snapshots uai of K Computing the approximation of ua from Vn This is an online computation and must be fast

Toulouse – p. 12/24

Implementation

Finding a good space Vn is an off line computation In practice, this space is usually generated by snapshots uai of K Computing the approximation of ua from Vn This is an online computation and must be fast The Galerkin projection is error optimal but has the bottleneck in the assembly of the stiffness matrix M and the resulting linear algebra

Toulouse – p. 12/24

Implementation

Finding a good space Vn is an off line computation In practice, this space is usually generated by snapshots uai of K Computing the approximation of ua from Vn This is an online computation and must be fast The Galerkin projection is error optimal but has the bottleneck in the assembly of the stiffness matrix M and the resulting linear algebra In the affine case M = m

i=1 yiMi and this helps

Toulouse – p. 12/24

Implementation

Finding a good space Vn is an off line computation In practice, this space is usually generated by snapshots uai of K Computing the approximation of ua from Vn This is an online computation and must be fast The Galerkin projection is error optimal but has the bottleneck in the assembly of the stiffness matrix M and the resulting linear algebra In the affine case M = m

i=1 yiMi and this helps

An alternative is to use a form of interpolation of the snapshots, e.g. polynomial interpolation

Toulouse – p. 12/24

Implementation

Finding a good space Vn is an off line computation In practice, this space is usually generated by snapshots uai of K Computing the approximation of ua from Vn This is an online computation and must be fast The Galerkin projection is error optimal but has the bottleneck in the assembly of the stiffness matrix M and the resulting linear algebra In the affine case M = m

i=1 yiMi and this helps

An alternative is to use a form of interpolation of the snapshots, e.g. polynomial interpolation This would be very fast on line but the size of the Lebesgue constant is now a critical issue

Toulouse – p. 12/24

Kolmogorov widths

It is useful to know the optimal performance we can expect from reduced basis methods

Toulouse – p. 13/24

Kolmogorov widths

It is useful to know the optimal performance we can expect from reduced basis methods Since they use approximation from linear subspace, the

• ptimal performance is governed by the Kolmogorov

width

Toulouse – p. 13/24

Kolmogorov widths

It is useful to know the optimal performance we can expect from reduced basis methods Since they use approximation from linear subspace, the

• ptimal performance is governed by the Kolmogorov

width Kolmogorov widths measure how accurately we can approximation K with n dimensional linear spaces

dn(K)X := inf

dim(Xn)=n sup u∈K

inf

v∈Xn u − vX

Toulouse – p. 13/24

Kolmogorov width of K

Distance of K to Xn

Toulouse – p. 14/24

Kolmogorov widths

It is useful to know the optimal performance we can expect from reduced basis methods Since they use approximation from linear subspace, the

• ptimal performance is governred by the Kolmogorov

width Kolmogorov widths measure how accurately we can approximation K with n dimensional linear spaces

dn(K)X := inf

dim(Xn)=n sup w∈K

inf

v∈Xn w − vX

Let us recall that the Reduced Basis Methods construct an n dimensional linear space (usually spanned by snapshots of the manifold) to be used with a Galerkin projection

Toulouse – p. 15/24

Kolmogorov widths

It is useful to know the optimal performance we can expect from reduced basis methods Since they use approximation from linear subspace, the

• ptimal performance is governred by the Kolmogorov

width Kolmogorov widths measure how accurately we can approximation K with n dimensional linear spaces

dn(K)X := inf

dim(Xn)=n sup w∈K

inf

v∈Xn w − vX

Let us recall that the Reduced Basis Methods construct an n dimensional linear space (usually spanned by snapshots of the manifold) to be used with a Galerkin projection So dn(K)H1

0(D) tells us the best error we can expect

Toulouse – p. 15/24

How well can we do?

So the main issues to understand are

Toulouse – p. 16/24

How well can we do?

So the main issues to understand are How fast does dn(K)H1

0(D) tend to 0?

Toulouse – p. 16/24

How well can we do?

So the main issues to understand are How fast does dn(K)H1

0(D) tend to 0?

Does the RBM selection of snapshots give close to this optimal rate of performance?

Toulouse – p. 16/24

How well can we do?

So the main issues to understand are How fast does dn(K)H1

0(D) tend to 0?

Does the RBM selection of snapshots give close to this optimal rate of performance? Can we give a numerical implementation with online costs reflecting the optimal rate?

Toulouse – p. 16/24

What is the n width of K ?

Let us start with the first question: what do we know about the n width of K?

Toulouse – p. 17/24

What is the n width of K ?

Let us start with the first question: what do we know about the n width of K? The following discussion is for the affine model

Toulouse – p. 17/24

What is the n width of K ?

Let us start with the first question: what do we know about the n width of K? The following discussion is for the affine model Using the analyticity of F, it is easy to show dn(K)H1

0(D)

decays exponentially for the AFFINE MODEL when the number of parameters is finite.

Toulouse – p. 17/24

What is the n width of K ?

Let us start with the first question: what do we know about the n width of K? The following discussion is for the affine model Using the analyticity of F, it is easy to show dn(K)H1

0(D)

decays exponentially for the AFFINE MODEL when the number of parameters is finite. However there are constants in these estimates which grow exponentially in the number of parameters

Toulouse – p. 17/24

What is the n width of K ?

Let us start with the first question: what do we know about the n width of K? The following discussion is for the affine model Using the analyticity of F, it is easy to show dn(K)H1

0(D)

decays exponentially for the AFFINE MODEL when the number of parameters is finite. However there are constants in these estimates which grow exponentially in the number of parameters So, we need a finer analysis of the effect of the number

• f parameters

Toulouse – p. 17/24

What is the n width of K ?

Let us start with the first question: what do we know about the n width of K? The following discussion is for the affine model Using the analyticity of F, it is easy to show dn(K)H1

0(D)

decays exponentially for the AFFINE MODEL when the number of parameters is finite. However there are constants in these estimates which grow exponentially in the number of parameters So, we need a finer analysis of the effect of the number

• f parameters

Cohen-DeVore-Schwab prove results for the AFFINE MODEL with infinitely many parameters and thereby avoid dependence on the number of parameters

Toulouse – p. 17/24

CDS Results

For the Affine Model a(x, y) = ¯

a(x) + ∞

j=1 yjψj(x)

Toulouse – p. 18/24

CDS Results

For the Affine Model a(x, y) = ¯

a(x) + ∞

j=1 yjψj(x)

Theorem (ψjL∞)j≥1 ∈ ℓp −

→ dn(K)H1

0 ≤ C0n−1/p+1

Toulouse – p. 18/24

CDS Results

For the Affine Model a(x, y) = ¯

a(x) + ∞

j=1 yjψj(x)

Theorem (ψjL∞)j≥1 ∈ ℓp −

→ dn(K)H1

0 ≤ C0n−1/p+1

Note that this result is dimension independent

Toulouse – p. 18/24

CDS Results

For the Affine Model a(x, y) = ¯

a(x) + ∞

j=1 yjψj(x)

Theorem (ψjL∞)j≥1 ∈ ℓp −

→ dn(K)H1

0 ≤ C0n−1/p+1

Note that this result is dimension independent This is proved by establishing the analytic expansion

u(x, y) =

ν∈Λ φν(x)yν with (φνH1

0) ∈ ℓp

Toulouse – p. 18/24

CDS Results

For the Affine Model a(x, y) = ¯

a(x) + ∞

j=1 yjψj(x)

Theorem (ψjL∞)j≥1 ∈ ℓp −

→ dn(K)H1

0 ≤ C0n−1/p+1

Note that this result is dimension independent This is proved by establishing the analytic expansion

u(x, y) =

ν∈Λ φν(x)yν with (φνH1

0) ∈ ℓp

φν = DνF(0)

ν!

Toulouse – p. 18/24

CDS Results

For the Affine Model a(x, y) = ¯

a(x) + ∞

j=1 yjψj(x)

Theorem (ψjL∞)j≥1 ∈ ℓp −

→ dn(K)H1

0 ≤ C0n−1/p+1

Note that this result is dimension independent This is proved by establishing the analytic expansion

u(x, y) =

ν∈Λ φν(x)yν with (φνH1

0) ∈ ℓp

φν = DνF(0)

ν!

A good n dimensional subspace corresponds to choosing the n terms with the largest coefficients:

φνH1

0(D) are largest

Toulouse – p. 18/24

CDS Results

For the Affine Model a(x, y) = ¯

a(x) + ∞

j=1 yjψj(x)

Theorem (ψjL∞)j≥1 ∈ ℓp −

→ dn(K)H1

0 ≤ C0n−1/p+1

Note that this result is dimension independent This is proved by establishing the analytic expansion

u(x, y) =

ν∈Λ φν(x)yν with (φνH1

0) ∈ ℓp

φν = DνF(0)

ν!

A good n dimensional subspace corresponds to choosing the n terms with the largest coefficients:

φνH1

0(D) are largest

So snapshots are all taken at y = 0: these are not necessarily on the manifold

Toulouse – p. 18/24

CDS Results

For the Affine Model a(x, y) = ¯

a(x) + ∞

j=1 yjψj(x)

Theorem (ψjL∞)j≥1 ∈ ℓp −

→ dn(K)H1

0 ≤ C0n−1/p+1

Note that this result is dimension independent This is proved by establishing the analytic expansion

u(x, y) =

ν∈Λ φν(x)yν with (φνH1

0) ∈ ℓp

φν = DνF(0)

ν!

A good n dimensional subspace corresponds to choosing the n terms with the largest coefficients:

φνH1

0(D) are largest

So snapshots are all taken at y = 0: these are not necessarily on the manifold Coefficients of basis expansion are simple monomials

Toulouse – p. 18/24

Numerical Recipe

The above leads to a numerical recipe Chifka-Cohen-DeVore-Schwab based on the following:

Toulouse – p. 19/24

Numerical Recipe

The above leads to a numerical recipe Chifka-Cohen-DeVore-Schwab based on the following: The set of Λn of monomial indices can be chosen with the property Lower Set Property: If ν ∈ Λn and µ < ν then µ ∈ Λn

Toulouse – p. 19/24

Numerical Recipe

The above leads to a numerical recipe Chifka-Cohen-DeVore-Schwab based on the following: The set of Λn of monomial indices can be chosen with the property Lower Set Property: If ν ∈ Λn and µ < ν then µ ∈ Λn If Λn is the current set of indices, it is expanded at next iteration

Toulouse – p. 19/24

Numerical Recipe

The above leads to a numerical recipe Chifka-Cohen-DeVore-Schwab based on the following: The set of Λn of monomial indices can be chosen with the property Lower Set Property: If ν ∈ Λn and µ < ν then µ ∈ Λn If Λn is the current set of indices, it is expanded at next iteration The search is limited because of a priori bounds

• n DνFH1

0 in terms of the ψjL∞

Toulouse – p. 19/24

Numerical Recipe

The above leads to a numerical recipe Chifka-Cohen-DeVore-Schwab based on the following: The set of Λn of monomial indices can be chosen with the property Lower Set Property: If ν ∈ Λn and µ < ν then µ ∈ Λn If Λn is the current set of indices, it is expanded at next iteration The search is limited because of a priori bounds

• n DνFH1

0 in terms of the ψjL∞

For the limited ν, one computes DνF recursively from current derivatives

Toulouse – p. 19/24

Numerical Recipe

The above leads to a numerical recipe Chifka-Cohen-DeVore-Schwab based on the following: The set of Λn of monomial indices can be chosen with the property Lower Set Property: If ν ∈ Λn and µ < ν then µ ∈ Λn If Λn is the current set of indices, it is expanded at next iteration The search is limited because of a priori bounds

• n DνFH1

0 in terms of the ψjL∞

For the limited ν, one computes DνF recursively from current derivatives In place of computing DνF(0), one would like to implement interpolation

Toulouse – p. 19/24

Numerical Recipe

The above leads to a numerical recipe Chifka-Cohen-DeVore-Schwab based on the following: The set of Λn of monomial indices can be chosen with the property Lower Set Property: If ν ∈ Λn and µ < ν then µ ∈ Λn If Λn is the current set of indices, it is expanded at next iteration The search is limited because of a priori bounds

• n DνFH1

0 in terms of the ψjL∞

For the limited ν, one computes DνF recursively from current derivatives In place of computing DνF(0), one would like to implement interpolation This leads to interesting quesions on interpolation

Toulouse – p. 19/24

General Strategies

Is there a general strategy for taking snapshots and finding a good subspace?

Toulouse – p. 20/24

General Strategies

Is there a general strategy for taking snapshots and finding a good subspace? Buffa, Maday, Patera, Prud´ homme and Turinici introduced and analyzed a Greedy Procedure for choosing the n snapshots: ua1, ua2, · · · , uan

Toulouse – p. 20/24

General Strategies

Is there a general strategy for taking snapshots and finding a good subspace? Buffa, Maday, Patera, Prud´ homme and Turinici introduced and analyzed a Greedy Procedure for choosing the n snapshots: ua1, ua2, · · · , uan This procedure can be stated for any Banach space X and any compact set K ⊂ X

Toulouse – p. 20/24

Greedy Algorithms

Pure (γ = 1) and Weak (0 < γ < 1) Greedy Algorithms

Toulouse – p. 21/24

Greedy Algorithms

Pure (γ = 1) and Weak (0 < γ < 1) Greedy Algorithms Let 0 < µ ≤ 1

Toulouse – p. 21/24

Greedy Algorithms

Pure (γ = 1) and Weak (0 < γ < 1) Greedy Algorithms Let 0 < µ ≤ 1 Choose g1 ∈ K such that g1X ≥ µ sup

g∈K

gX

Toulouse – p. 21/24

Greedy Algorithms

Pure (γ = 1) and Weak (0 < γ < 1) Greedy Algorithms Let 0 < µ ≤ 1 Choose g1 ∈ K such that g1X ≥ µ sup

g∈K

gX

If g1, . . . , gk have been chosen, let

Vk := span{g1, . . . , gk} then choose gk+1 ∈ K so that dist(gk+1, Vk) ≥ µ supg∈K dist(g, Vk)X

Toulouse – p. 21/24

Greedy Algorithms

Pure (γ = 1) and Weak (0 < γ < 1) Greedy Algorithms Let 0 < µ ≤ 1 Choose g1 ∈ K such that g1X ≥ µ sup

g∈K

gX

If g1, . . . , gk have been chosen, let

Vk := span{g1, . . . , gk} then choose gk+1 ∈ K so that dist(gk+1, Vk) ≥ µ supg∈K dist(g, Vk)X

Thus, add an element of K which is approximated poorly

Toulouse – p. 21/24

Greedy Algorithms

Pure (γ = 1) and Weak (0 < γ < 1) Greedy Algorithms Let 0 < µ ≤ 1 Choose g1 ∈ K such that g1X ≥ µ sup

g∈K

gX

If g1, . . . , gk have been chosen, let

Vk := span{g1, . . . , gk} then choose gk+1 ∈ K so that dist(gk+1, Vk) ≥ µ supg∈K dist(g, Vk)X

Thus, add an element of K which is approximated poorly In going further let

σn(K)X := dist(K, Vn)X := supw∈K infg∈Vn w − gX

Toulouse – p. 21/24

Greedy Algorithms

Pure (γ = 1) and Weak (0 < γ < 1) Greedy Algorithms Let 0 < µ ≤ 1 Choose g1 ∈ K such that g1X ≥ µ sup

g∈K

gX

If g1, . . . , gk have been chosen, let

Vk := span{g1, . . . , gk} then choose gk+1 ∈ K so that dist(gk+1, Vk) ≥ µ supg∈K dist(g, Vk)X

Thus, add an element of K which is approximated poorly In going further let

σn(K)X := dist(K, Vn)X := supw∈K infg∈Vn w − gX

This is the performance of the weak greedy algorithm in approximating the elements in K

Toulouse – p. 21/24

Performance of Greedy Algorithms

Buffa-Maday-Patera-Prud’homme-Turnici

σn(K)H ≤ n2ndn(K)H

Toulouse – p. 22/24

Performance of Greedy Algorithms

Buffa-Maday-Patera-Prud’homme-Turnici

σn(K)H ≤ n2ndn(K)H

Binev-Cohen-Dahmen-DeVore-Petrova-Wojtaszsczyk have given various estimates for the performance of this Weak Greedy Algorithm in a Hilbert space.

Toulouse – p. 22/24

Performance of Greedy Algorithms

Buffa-Maday-Patera-Prud’homme-Turnici

σn(K)H ≤ n2ndn(K)H

Binev-Cohen-Dahmen-DeVore-Petrova-Wojtaszsczyk have given various estimates for the performance of this Weak Greedy Algorithm in a Hilbert space. DeVore, Petrova, Wojtaszsczyk prove results for a general Banach space

Toulouse – p. 22/24

Performance of Greedy Algorithms

Buffa-Maday-Patera-Prud’homme-Turnici

σn(K)H ≤ n2ndn(K)H

Binev-Cohen-Dahmen-DeVore-Petrova-Wojtaszsczyk have given various estimates for the performance of this Weak Greedy Algorithm in a Hilbert space. DeVore, Petrova, Wojtaszsczyk prove results for a general Banach space Here are two fundamental estimates

Toulouse – p. 22/24

Performance of Greedy Algorithms

Buffa-Maday-Patera-Prud’homme-Turnici

σn(K)H ≤ n2ndn(K)H

Binev-Cohen-Dahmen-DeVore-Petrova-Wojtaszsczyk have given various estimates for the performance of this Weak Greedy Algorithm in a Hilbert space. DeVore, Petrova, Wojtaszsczyk prove results for a general Banach space Here are two fundamental estimates BCDDPW In a Hilbert space, if dn(K)H = O(n−α) then the spaces generated by the greedy algorithm give the same performance

Toulouse – p. 22/24

Performance of Greedy Algorithms

Buffa-Maday-Patera-Prud’homme-Turnici

σn(K)H ≤ n2ndn(K)H

Binev-Cohen-Dahmen-DeVore-Petrova-Wojtaszsczyk have given various estimates for the performance of this Weak Greedy Algorithm in a Hilbert space. DeVore, Petrova, Wojtaszsczyk prove results for a general Banach space Here are two fundamental estimates BCDDPW In a Hilbert space, if dn(K)H = O(n−α) then the spaces generated by the greedy algorithm give the same performance DPW In a Hilbert space, if dn(K)H = O(e−c1nα), then there is a c2 such that the greedy algorithm gives the performance O(e−c2nα)

Toulouse – p. 22/24

General Banach Space X

One loses a little when going to a general Banach space

Toulouse – p. 23/24

General Banach Space X

One loses a little when going to a general Banach space If dn(K)X = O(n−α)

Toulouse – p. 23/24

General Banach Space X

One loses a little when going to a general Banach space If dn(K)X = O(n−α) Then the greedy algorithm performance is guaranteed to be O(n−r+1/2)

Toulouse – p. 23/24

General Banach Space X

One loses a little when going to a general Banach space If dn(K)X = O(n−α) Then the greedy algorithm performance is guaranteed to be O(n−r+1/2) Probably in the case X = Lp or more generally for spaces of p type this can be improved to

O(n−r+|1/2−1/p|)

Toulouse – p. 23/24

General Banach Space X

One loses a little when going to a general Banach space If dn(K)X = O(n−α) Then the greedy algorithm performance is guaranteed to be O(n−r+1/2) Probably in the case X = Lp or more generally for spaces of p type this can be improved to

O(n−r+|1/2−1/p|)

If dn(K)X = O(e−c0nα)

Toulouse – p. 23/24

General Banach Space X

One loses a little when going to a general Banach space If dn(K)X = O(n−α) Then the greedy algorithm performance is guaranteed to be O(n−r+1/2) Probably in the case X = Lp or more generally for spaces of p type this can be improved to

O(n−r+|1/2−1/p|)

If dn(K)X = O(e−c0nα) Then the greedy algorithm performance is guaranteed to be O(e−c1nα)

Toulouse – p. 23/24

General Theory?

We have restricted our discussion to the Affine Model

Toulouse – p. 24/24

General Theory?

We have restricted our discussion to the Affine Model It would be good to have a theory that applies to general A

Toulouse – p. 24/24

General Theory?

We have restricted our discussion to the Affine Model It would be good to have a theory that applies to general A Generally speaking, it is easy to determine the n width

• f A

Toulouse – p. 24/24

General Theory?

We have restricted our discussion to the Affine Model It would be good to have a theory that applies to general A Generally speaking, it is easy to determine the n width

• f A

For example in the affine case dn(A)L∞(D) ≤ Cn−1/p+1 when (ψjL∞(D)) ∈ ℓp beginitemstep

Toulouse – p. 24/24

General Theory?

We have restricted our discussion to the Affine Model It would be good to have a theory that applies to general A Generally speaking, it is easy to determine the n width

• f A

For example in the affine case dn(A)L∞(D) ≤ Cn−1/p+1 when (ψjL∞(D)) ∈ ℓp beginitemstep Cohen-DeVore If dn(A)L∞ = O(n−α), n ≥ 1 then

dn(K)H1

0 = O(n−α+1), n ≥ 1

Toulouse – p. 24/24

General Theory?

We have restricted our discussion to the Affine Model It would be good to have a theory that applies to general A Generally speaking, it is easy to determine the n width

• f A

For example in the affine case dn(A)L∞(D) ≤ Cn−1/p+1 when (ψjL∞(D)) ∈ ℓp beginitemstep Cohen-DeVore If dn(A)L∞ = O(n−α), n ≥ 1 then

dn(K)H1

0 = O(n−α+1), n ≥ 1

Do not know if loss of one is necessary in general

Toulouse – p. 24/24