Adaptive Stochastic Collocation for PDE-Constrained Optimization - - PowerPoint PPT Presentation

▶
adaptive stochastic collocation for pde constrained
SMART_READER_LITE
LIVE PREVIEW

Adaptive Stochastic Collocation for PDE-Constrained Optimization - - PowerPoint PPT Presentation

Jan 08, 2024 146 likes •614 views

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Adaptive Stochastic Collocation for PDE-Constrained Optimization under Uncertainty using Sparse Grids and Model Reduction Matthew J. Zahr Advisor: Charbel

slide-1
SLIDE 1

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Adaptive Stochastic Collocation for PDE-Constrained Optimization under Uncertainty using Sparse Grids and Model Reduction

Matthew J. Zahr

Advisor: Charbel Farhat Computational and Mathematical Engineering Stanford University

Joint work with: Kevin Carlberg (Sandia CA), Drew Kouri (Sandia NM)

SIAM Conference on Uncertainty Quantification MS104: Reduced-Order Modeling in Uncertainty Quantification Lausanne, Switzerland April 7, 2016

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-2
SLIDE 2

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

PDE-Constrained Optimization under Uncertainty

Goal: Efficiently solve stochastic PDE-constrained optimization problems minimize

µ∈Rnµ

E[J (u(µ, · ), µ, · )] subject to r(u(µ, ξ), µ, ξ) = 0 ∀ξ ∈ Ξ r : Rnu × Rnµ × Rnξ → Rnu is the discretized stochastic PDE J : Rnu × Rnµ × Rnξ → R is a quantity of interest u ∈ Rnu is the PDE state vector µ ∈ Rnµ is the vector of (deterministic) optimization parameters ξ ∈ Rnξ is the vector of stochastic parameters

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-3
SLIDE 3

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Literature Review: Stochastic Optimal Control

Stochastic collocation

Dimension-adaptive sparse grids – globalized with trust-region method [Kouri et al., 2013, Kouri et al., 2014] Generalized polynomial chaos – sequential quadratic programming [Tiesler et al., 2012] + Orders of magnitude improvement over isotropic sparse grids

  • Still requires many PDE solves for even moderate dimensional problems

Model order reduction

Goal-oriented, dimension-adaptive, weighted greedy algorithm for training stochastic reduced-order model [Chen and Quarteroni, 2015] Extension to optimal control [Chen and Quarteroni, 2014] + Reduction in number of PDE solves at cost of large number of ROM solves

  • Restriction to offline-online framework may lead to unnecessay PDE solves

and large reduced bases

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-4
SLIDE 4

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Proposed Approach

Introduce two levels of inexactness to obtain an inexpensive, approximate version

  • f the stochastic optimization problem; manage inexactness with trust-region-like

method Anisotropic sparse grids used for inexact integration of risk measures Reduced-order models used for inexact evaluations at collocation nodes Error indicators introduced to account for both sources of inexactness Refinement of integral approximation and reduced-order model via dimension-adaptive sparse grids and a greedy method over collocation nodes Embedded in globally convergent trust-region-like algorithm with a strong connection to error indicators and refinement mechanism

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-5
SLIDE 5

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Sparse Grids Model Reduction

Anisotropic Sparse Grids [Gerstner and Griebel, 2003]

1D Quadrature Rules: Define the difference operator ∆j

k ≡ Ej k − Ej−1 k

where E0

k ≡ 0 and Ej k as the level-j 1d quadrature rule for dimension k

Anisotropic Sparse Grid: EI ≡

  • i∈I

∆i1

1 ⊗ · · · ⊗ ∆ inξ nξ

Forward Neighbors: N(I) = {k + ej | k ∈ I} \ I Truncation Error: [Gerstner and Griebel, 2003, Kouri et al., 2013] E − EI =

  • i/

∈I

∆i1

1 ⊗ · · · ⊗ ∆ inξ nξ ≈

  • i∈N (I)

∆i1

1 ⊗ · · · ⊗ ∆ inξ nξ = EN (I)

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-6
SLIDE 6

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Sparse Grids Model Reduction

Stochastic Collocation via Anisotropic Sparse Grids

Stochastic collocation using anisotropic sparse grid nodes used to approximate integral with summation minimize

u∈Rnu, µ∈Rnµ

E[J (u, µ, · )] subject to r(u, µ, ξ) = 0 ∀ξ ∈ Ξ

⇓

minimize

u∈Rnu, µ∈Rnµ

EI[J (u, µ, · )] subject to r(u, µ, ξ) = 0 ∀ξ ∈ ΞI

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-7
SLIDE 7

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Sparse Grids Model Reduction

Projection-Based Model Reduction to Reduce PDE Size

Model Order Reduction (MOR) assumption: state vector lies in low-dimensional subspace u ≈ Φy ∂u ∂µ ≈ Φ ∂y ∂µ where

Φ =

  • φ1

u

· · · φku

u

  • ∈ Rnu×ku is the reduced basis

y ∈ Rku are the reduced coordinates of u nu ≫ ku

Substitute assumption into High-Dimensional Model (HDM), r(u, µ, ξ) = 0, and use a Galerkin projection to obtain the square system ΦT r(Φy, µ, ξ) = 0

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-8
SLIDE 8

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Sparse Grids Model Reduction

Definition of Φ: Data-Driven Reduction

State-Sensitivity Proper Orthogonal Decomposition (SSPOD) Collect state and sensitivity snapshots by sampling HDM X =

  • u(µ1, ξ1)

u(µ2, ξ2) · · · u(µn, ξn)

  • Y =
  • ∂u

∂µ(µ1, ξ1) ∂u ∂µ(µ2, ξ2)

· · ·

∂u ∂µ(µn, ξn)

  • Use Proper Orthogonal Decomposition to generate reduced basis for each

individually ΦX = POD(X) ΦY = POD(Y ) Concatenate and orthogonalize to get reduced-order basis Φ = QR ΦX ΦY

  • Zahr, Carlberg, Kouri

Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-9
SLIDE 9

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Sparse Grids Model Reduction

Reduced-Order Stochastic Collocation via Anisotropic Sparse Grids

Stochastic collocation of the reduced-order model over anisotropic sparse grid nodes used to approximate integral with cheap summation minimize

u∈Rnu, µ∈Rnµ

E[J (u, µ, · )] subject to r(u, µ, ξ) = 0 ∀ξ ∈ Ξ

⇓

minimize

u∈Rnu, µ∈Rnµ

EI[J (u, µ, · )] subject to r(u, µ, ξ) = 0 ∀ξ ∈ ΞI

⇓

minimize

y∈Rku, µ∈Rnµ

EI[J (Φy, µ, · )] subject to ΦT r(Φy, µ, ξ) = 0 ∀ξ ∈ ΞI

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-10
SLIDE 10

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Schematic µ-space Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-11
SLIDE 11

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Compress HDM HDM ROB Φ

Schematic µ-space

HDM ROB

Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-12
SLIDE 12

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Compress HDM HDM ROB Φ ROM Optimizer

Schematic µ-space

HDM ROB ROM ROM ROM · · · ROM ROM

Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-13
SLIDE 13

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Compress HDM HDM HDM ROB Φ ROM Optimizer

Schematic µ-space

HDM ROB ROM ROM ROM · · · ROM ROM HDM ROB

Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-14
SLIDE 14

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Compress HDM HDM HDM ROB Φ ROM Optimizer

Schematic µ-space

HDM ROB ROM ROM ROM · · · ROM ROM HDM ROB ROM ROM ROM · · · ROM ROM

Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-15
SLIDE 15

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Compress HDM HDM HDM ROB Φ ROM Optimizer

Schematic µ-space

HDM ROB ROM ROM ROM · · · ROM ROM HDM ROB ROM ROM ROM · · · ROM ROM · · ·

Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-16
SLIDE 16

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Compress HDM HDM HDM ROB Φ ROM Optimizer

Schematic µ-space

HDM ROB ROM ROM ROM · · · ROM ROM HDM ROB ROM ROM ROM · · · ROM ROM · · ·

Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-17
SLIDE 17

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Compress HDM HDM HDM ROB Φ ROM Optimizer

Schematic µ-space

HDM ROB ROM ROM ROM · · · ROM ROM HDM ROB ROM ROM ROM · · · ROM ROM · · ·

Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-18
SLIDE 18

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

ROM-Based Trust-Region Framework for Optimization

Compress HDM HDM HDM ROB Φ ROM Optimizer

Schematic µ-space

HDM ROB ROM ROM ROM · · · ROM ROM HDM ROB ROM ROM ROM · · · ROM ROM · · ·

Breakdown of Computational Effort

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-19
SLIDE 19

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

Trust-Regions Based on Error Indicators: Motivation

Let ˆ ϑk(µ) = ϑk(Φy(µ), µ) be a vector-valued ROM error indicator and Ak ≡ ∂ ˆ ϑk ∂µ (µk)T ∂ ˆ ϑk ∂µ (µk) = QkΛ2

kQT k

Then, to first order1, ϑk(µ) ≡

  • ˆ

ϑk(µ)

  • 2 =
  • ∂ ˆ

ϑk ∂µ (µk)(µ − µk)

  • 2

= ||µ − µk||Ak ≤ ∆k

∆k λ1 q1 ∆k λ2 q2

µk

Annotated schematic of trust-region: qi = Qkei and λi = eT

i Λkei

1assuming ˆ

ϑk(µk) = 0, i.e., ROM exact at trust-region center

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-20
SLIDE 20

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

A trust-region method with a strong connection to error indicators

Propose a trust-region method to solve the unconstrained stochastic PDE

  • ptimization problem

minimize

µ∈Rnµ

F(µ) ≡ E

  • ˆ

J (µ, · )

  • that leverages trust-region subproblems of the form

minimize

µ∈Rnµ

mk(x) subject to ϑk(x) ≤ ∆k, where ϑk(µ) is an error indicator, i.e., there exists a constant2 ζ > 0 such that |F(µ) − mk(µ)| ≤ ζϑk(µ)

2arbitrary, i.e., not tied to algorithmic parameters, and does not need to be computed

  • r estimated

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-21
SLIDE 21

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

Error indicator for function values must account for ROM inaccuracy and sparse grid truncation error

Error indicator for function values are based on collocation of the residual norm to account for ROM error and forward neighbors to account for truncation error Define ˆ J (µ, ξ) ≡ J (u(µ, ξ), µ, ξ) ˆ Jr(µ, ξ) ≡ J (Φy(µ, ξ), µ, ξ) ˆ r(µ, ξ) ≡ r(Φy(µ, ξ), µ, ξ)

  • E
  • ˆ

J (µ, · )

  • − EI
  • ˆ

Jr(µ, · )

  • ≤ E
  • ˆ

J (µ, · ) − ˆ Jr(µ, · )

  • +
  • EIc
  • ˆ

Jr(µ, · )

  • ζ1EI ∪ N (I) [||ˆ

r(µ, ξ)||] +

  • EN (I)
  • ˆ

Jr(µ, · )

  • Zahr, Carlberg, Kouri

Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-22
SLIDE 22

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

Error indicator for function gradients must account for primal and dual ROM inaccuracy and sparse grid truncation error

Error indicator for function gradients are based on collocation of the primal and dual residual norm to account for ROM error and forward neighbors to account for truncation error Define ˆ r∂(µ, ξ) ≡ ∂r ∂u(Φy(µ, ξ))Φ ∂y ∂µ(µ, ξ) + ∂r ∂µ(Φy(µ, ξ), µ, ξ)

  • E
  • ∇µ ˆ

J (µ, · )

  • − EI
  • ∇µ ˆ

J (µ, · )

  • ≤ E
  • ∇µ ˆ

J (µ, · ) − ∇µ ˆ Jr(µ, · )

  • +
  • EIc
  • ∇µ ˆ

Jr(µ, · )

  • ζ2EI ∪ N (I) [||ˆ

r(µ, · )||] + ζ3EI ∪ N (I)

  • ˆ

r∂(µ, · )

  • +
  • EN (I)
  • ∇µ ˆ

Jr(µ, · )

  • Zahr, Carlberg, Kouri

Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-23
SLIDE 23

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

Model problem is the two-level approximation of the objective using reduced-order models and anisotropic sparse grids

The trust-region subproblem minimize

µ∈Rnµ

mk(x) subject to ϑk(x) ≤ ∆k, is defined3 as mk(µ) = EIk

  • ˆ

J r

k (µ, · )

  • ϑk(µ) = α1EIk ∪ N (Ik) [||ˆ

rk(µ, · )||] + α2

  • EN (Ik)
  • ˆ

J r

k (µ, · )

  • An error indicator for the model gradient is also required and chosen as

ϕk = β1EIk ∪ N (Ik) [||ˆ r(µk, · )||] + β2EIk ∪ N (Ik)

  • ˆ

r∂(µk, · )

  • + β3
  • EN (Ik)
  • ∇µ ˆ

Jr(µk, · )

  • 3 ˆ

J r

k , ˆ

rk, and ˆ r∂

k or defined exactly as ˆ

J r, ˆ r, and ˆ r∂ with Φk in place of Φ.

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-24
SLIDE 24

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

Trust-Region Method for Managing Two-Level Inexactness

1: Model update: Choose mk, i.e., an index set, Ik, and reduced basis, Φk,

such that ϑk(xk) ≤ κucv∆k ϕk ≤ κubp ||∇mk(xk)||

2: Step computation: Approximately solve the trust-region subproblem

ˆ xk = arg min

x∈RN

mk(x) subject to ϑk(x) ≤ ∆k

3: Step acceptance: Compute

ρk = F(xk) − F(ˆ xk) mk(xk) − mk(ˆ xk) if ρk ≥ η1 then xk+1 = ˆ xk else xk+1 = xk end if

4: Trust-region update:

if ρk ≤ η1 then ∆k+1 ∈ (0, γϑk(ˆ xk)] end if if ρk ∈ (η1, η2) then ∆k+1 ∈ [γϑk(ˆ xk), ∆k] end if if ρk ≥ η2 then ∆k+1 ∈ [∆k, ∆max] end if

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-25
SLIDE 25

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

Dimension-Adaptive Sparse Grids and Greedy Sampling to Control Two-Level Inexactness

while EN (Ik) [J r

k (µ, · )] >

1 2α2 κucv∆k do Refine index set: Let j = arg max

j∈N (Ik)

|Ej [J r

k (µ, · )]|

Ik ← Ik ∪ {j} while EIk∪N (Ik) [||ˆ rk(µk, · )||] > 1 2α1 κucv∆k do Evaluate error indicator: For each ξj ∈ ΞIk∪N (Ik), compute rj = ||ˆ rk(µk, ξj )|| where ωj is the quadrature weight associated with node ξj Sample high-dimensional model: Let j∗ = arg max rj and compute w(µk, ξj∗), ∂w ∂µ (µk, ξj∗) and update reduced-order basis, Φk, using SSPOD end while end while

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-26
SLIDE 26

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Primal and Sensitivity Error Indicators Trust-Region Algorithm Two-Level Model Refinement

Trust-Region Convergence Theory

The trust-region method guarantees lim inf

k→∞ ||∇µE [Jk(µk, · )]|| = 0

provided there exists constants ζ, τ > 0 such that |E [Jk(µ, · )] − mk(µ)| ≤ ζϑk(µ) ||∇µE [Jk(µk, · )] − ∇µmk(µk)|| ≤ τϕk

Compress HDM HDM HDM ROB Φ ROM Optimizer

Schematic µ-space

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-27
SLIDE 27

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Optimal control of steady Burgers’ equation

Optimization problem: minimize

µ∈Rnµ

1 2

  • Ξ

ρ(ξ) 1 (u(ξ, x) − ¯ w)2 dx dξ + α 2 1 z(µ, x)2 dx where u(ξ, x) solves −ν(ξ)∂xxu(ξ, x) + u(ξ, x)∂xu(ξ, x) = z(µ, x) x ∈ (0, 1), ξ ∈ Ξ u(ξ, 0) = d0(ξ) u(ξ, 1) = d1(ξ) ξ ∈ Ξ Desired state: ¯ w ≡ 1 Stochastic Space: Ξ = [−1, 1]3, ρ(ξ)dξ = 2−3dξ ν(ξ) = 10ξ1−2 d0(ξ) = 1 + ξ2 1000 d1(ξ) = ξ3 1000 Parametrization: z(µ, x) – cubic splines with 9 knots, nµ = 11

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-28
SLIDE 28

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Recovers Optimal Control

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1 1 2 3

z

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.5 0.5 1 1.5

x u

( ) Determinstic (ξ = 0)

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-29
SLIDE 29

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Recovers Optimal Control

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1 1 2 3

z

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.5 0.5 1 1.5

x u

( ) Determinstic (ξ = 0) ( ) HDM, iso-SG (l = 4)

( ) E[u]

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-30
SLIDE 30

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Recovers Optimal Control

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1 1 2 3

z

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.5 0.5 1 1.5

x u

( ) Determinstic (ξ = 0) ( ) HDM, iso-SG (l = 4)

( ) E[u] ( ) E[u] ± σ[u]

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-31
SLIDE 31

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Recovers Optimal Control

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1 1 2 3

z

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.5 0.5 1 1.5

x u

( ) Determinstic (ξ = 0) ( ) HDM, iso-SG (l = 4)

( ) E[u] ( ) E[u] ± σ[u] ( ) E[u] ± 2σ[u]

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-32
SLIDE 32

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Recovers Optimal Control

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1 1 2 3

z

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.5 0.5 1 1.5

x u

( ) Determinstic (ξ = 0) ( ) HDM, iso-SG (l = 4)

( ) E[u] ( ) E[u] ± σ[u] ( ) E[u] ± 2σ[u]

( ) ROM, aniso-SG

( ) E[Φy]

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-33
SLIDE 33

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Recovers Optimal Control

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1 1 2 3

z

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.5 0.5 1 1.5

x u

( ) Determinstic (ξ = 0) ( ) HDM, iso-SG (l = 4)

( ) E[u] ( ) E[u] ± σ[u] ( ) E[u] ± 2σ[u]

( ) ROM, aniso-SG

( ) E[Φy] ( ) E[Φy] ± σ[Φy]

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-34
SLIDE 34

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Recovers Optimal Control

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1 1 2 3

z

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.5 0.5 1 1.5

x u

( ) Determinstic (ξ = 0) ( ) HDM, iso-SG (l = 4)

( ) E[u] ( ) E[u] ± σ[u] ( ) E[u] ± 2σ[u]

( ) ROM, aniso-SG

( ) E[Φy] ( ) E[Φy] ± σ[Φy] ( ) E[Φy] ± 2σ[Φy]

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-35
SLIDE 35

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Requires Very Few HDM Queries to Converges to Optimal Control

Prior to each trust-region subproblem, the model (sparse grid, Ik, and basis, Φk) must be constructed such that error indicators are below a tolerance Iteration 1 1 2 3 4 5 1 2 3 4 5 i1 i2 −1 −0.5 0.5 1 −1 −0.5 0.5 1 ξ1 ξ2 I – N(I) – HDM Sample –

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-36
SLIDE 36

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Requires Very Few HDM Queries to Converges to Optimal Control

Prior to each trust-region subproblem, the model (sparse grid, Ik, and basis, Φk) must be constructed such that error indicators are below a tolerance Iteration 2 1 2 3 4 5 1 2 3 4 5 i1 i2 −1 −0.5 0.5 1 −1 −0.5 0.5 1 ξ1 ξ2 I – N(I) – HDM Sample –

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-37
SLIDE 37

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Requires Very Few HDM Queries to Converges to Optimal Control

Prior to each trust-region subproblem, the model (sparse grid, Ik, and basis, Φk) must be constructed such that error indicators are below a tolerance Iteration 3 1 2 3 4 5 1 2 3 4 5 i1 i2 −1 −0.5 0.5 1 −1 −0.5 0.5 1 ξ1 ξ2 I – N(I) – HDM Sample –

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-38
SLIDE 38

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Requires Very Few HDM Queries to Converges to Optimal Control

Prior to each trust-region subproblem, the model (sparse grid, Ik, and basis, Φk) must be constructed such that error indicators are below a tolerance Iteration 4 1 2 3 4 5 1 2 3 4 5 i1 i2 −1 −0.5 0.5 1 −1 −0.5 0.5 1 ξ1 ξ2 I – N(I) – HDM Sample –

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-39
SLIDE 39

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Requires Very Few HDM Queries to Converges to Optimal Control

Prior to each trust-region subproblem, the model (sparse grid, Ik, and basis, Φk) must be constructed such that error indicators are below a tolerance Iteration 5 1 2 3 4 5 1 2 3 4 5 i1 i2 −1 −0.5 0.5 1 −1 −0.5 0.5 1 ξ1 ξ2 I – N(I) – HDM Sample –

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-40
SLIDE 40

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Error-Based Trust-Region Method Requires Very Few HDM Queries to Converges to Optimal Control

Prior to each trust-region subproblem, the model (sparse grid, Ik, and basis, Φk) must be constructed such that error indicators are below a tolerance Iteration 6 1 2 3 4 5 1 2 3 4 5 i1 i2 −1 −0.5 0.5 1 −1 −0.5 0.5 1 ξ1 ξ2 I – N(I) – HDM Sample –

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-41
SLIDE 41

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Global Convergence of Trust-Region Method

The trust-region method finds a sequence of parameters µk such that the gradient

  • f HDM (||∇J (µ)||) converges to 0 from arbitrary starting point – global

convergence – while incurring few HDM queries

mk(ˆ µk) ||∇mk(ˆ µk)|| J (ˆ µk) ||∇J (ˆ µk)|| ρk ∆k Success? 3.8783e-03 3.3779e-03 8.3351e-03 6.8542e-03

  • 3.1121e-03

2.0393e-04 7.2687e-03 7.0676e-03 1.3918e+00 1.0000e+02 True 3.0474e-03 7.7900e-05 6.8352e-03 3.3518e-03 3.3943e-01 2.0000e+02 True 1.1910e-02 3.7019e-04 9.7269e-03 3.5655e-03

  • 2.6141e-01

1.0000e+02 False 6.3680e-03 9.6334e-06 6.3591e-03 8.6182e-05 1.0070e+00 2.8202e-03 True 6.3587e-03 7.2419e-07 6.3589e-03 7.2665e-07 1.0018e+00 5.6404e-03 True HDM Queries ROM Queries (max size) 4 3720 (48)

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-42
SLIDE 42

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Comparison to Stochastic Optimization with Collocation on 4-Level Isotropic Sparse Grid: 1000-Fold Reduction in HDM Queries

1 2 3 4 1 2 3 4 i1 i2 −1 −0.5 0.5 1 −1 −0.5 0.5 1 ξ1 ξ2 Iterations (L-BFGS) HDM Queries ||∇J || 34 6372 6.6064e-07

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-43
SLIDE 43

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Leveraging and Managing Two-Levels of Inexactness for Efficient Stochastic PDE-Constrained Optimization

Conclusions Trust-region method with strong connection to model error indicators Two-level approximation of moments of quantities of interest of SPDE

Anisotropic sparse grids - inexact integration Reduced-order models - inexact evaluations

Two-level inexactness managed through trust-region method Future work Comparison with traditional trust-region method (TRPOD) Comparison with offline/online approaches Incorporate nonlinear constraints Local reduced-order models for improved efficiency Less expensive error indicators for cheaper trust-region subproblems [Drohmann and Carlberg, 2014]

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-44
SLIDE 44

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

Acknowledgement

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-45
SLIDE 45

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

References I

Chen, P. and Quarteroni, A. (2014). Weighted reduced basis method for stochastic optimal control problems with elliptic pde constraint. SIAM/ASA Journal on Uncertainty Quantification, 2(1):364–396. Chen, P. and Quarteroni, A. (2015). A new algorithm for high-dimensional uncertainty quantification based on dimension-adaptive sparse grid approximation and reduced basis methods. Journal of Computational Physics, 298:176–193. Drohmann, M. and Carlberg, K. (2014). The romes method for statistical modeling of reduced-order-model error. SIAM Journal on Uncertainty Quantification. Gerstner, T. and Griebel, M. (2003). Dimension–adaptive tensor–product quadrature. Computing, 71(1):65–87. Kouri, D. P., Heinkenschloss, M., Ridzal, D., and van Bloemen Waanders, B. G. (2013). A trust-region algorithm with adaptive stochastic collocation for pde optimization under uncertainty. SIAM Journal on Scientific Computing, 35(4):A1847–A1879.

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs

slide-46
SLIDE 46

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments

References II

Kouri, D. P., Heinkenschloss, M., Ridzal, D., and van Bloemen Waanders, B. G. (2014). Inexact objective function evaluations in a trust-region algorithm for pde-constrained

  • ptimization under uncertainty.

SIAM Journal on Scientific Computing, 36(6):A3011–A3029. Tiesler, H., Kirby, R. M., Xiu, D., and Preusser, T. (2012). Stochastic collocation for optimal control problems with stochastic pde constraints. SIAM Journal on Control and Optimization, 50(5):2659–2682.

Zahr, Carlberg, Kouri Stochastic PDE-Optimization with Adaptive SG/ROMs