Proper Orthogonal Decomposition in Optimization Bret Kragel and - - PowerPoint PPT Presentation

▶

Mar 02, 2023 180 likes •1.48k views

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Proper Orthogonal Decomposition in Optimization Bret Kragel and Ekkehard W. Sachs Surrogate Modelling and Space Mapping for Engineering Optimization

SLIDE 1

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion

Proper Orthogonal Decomposition in Optimization

Bret Kragel and Ekkehard W. Sachs Surrogate Modelling and Space Mapping for Engineering Optimization Organizers: K. Madsen and J. W. Bandler Technical University of Denmark, Copenhagen, November 10, 2006

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 2

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion

1

The Standard POD Method The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

2

Streamline Diffusion POD Models Sources POD Failure The SDPOD Model

3

Optimization with TRPOD Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

4

Conclusion

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 3

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Velocity Tracking Problem

Objective Minimize a cost functional: J (u, g) = α 2 T

(u − ud)2 dx dt + β 2 T

(|g|2 + β1 |gt|2 + β2 |gx|2) dx dt System Nonhomogeneous incompressible Navier-Stokes problem: ut − ν∆u + u · ∇u + ∇p = 0 in Ω × (0, T], ∇ · u = 0 in Ω × (0, T], u = g

n Γc × [0, T],

u = 0

n (Γ \ Γc) × [0, T],

u(0, x) = u0(x) in Ω

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 4

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Discretized Formulation of Navier-Stokes Problem

Semi-discretization in time using fractional-step-θ-scheme leads to generalized stationary Navier-Stokes problem for each time step. Spatial discretization using mixed Galerkin finite element method: Find uh ∈ Vh and ph ∈ Qh ⊆ L2

0(Ω), such that

(uh, vh) + νa(uh, vh) + n(uh, uh, vh) + b(vh, ph) = 0 ∀ vh ∈ Vh b(uh, qh) = 0 ∀ qh ∈ Qh Smaller mesh parameter h for finer mesh and larger finite element basis set Finite element functions have highly local character Finite element discretization of the spatial domain leads to large nonlinear algebraic systems for each step of the temporal discretization. Each evaluation of cost functional requires numerical solution of the time-dependent Navier-Stokes problem.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 5

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Discretized Formulation of Navier-Stokes Problem

Semi-discretization in time using fractional-step-θ-scheme leads to generalized stationary Navier-Stokes problem for each time step. Spatial discretization using mixed Galerkin finite element method: Find uh ∈ Vh and ph ∈ Qh ⊆ L2

0(Ω), such that

(uh, vh) + νa(uh, vh) + n(uh, uh, vh) + b(vh, ph) = 0 ∀ vh ∈ Vh b(uh, qh) = 0 ∀ qh ∈ Qh Smaller mesh parameter h for finer mesh and larger finite element basis set Finite element functions have highly local character Finite element discretization of the spatial domain leads to large nonlinear algebraic systems for each step of the temporal discretization. Each evaluation of cost functional requires numerical solution of the time-dependent Navier-Stokes problem.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 6

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Proper Orthogonal Decomposition

Reduced-order models based on proper orthogonal decomposition (POD) have the potential to simulate the Navier-Stokes equations with much less computational effort. Elements (snapshots) ui ∈ H, i = 1, . . . , n (p = dim{u1, . . . , un} ≥ 1) taken from some Hilbert space H. Find orthonormal basis {Ψi}p

i=1 for space spanned by snapshots, such that

min

Ψ1,...,Ψm n

ωi

ui −

m

(ui, Ψj)HΨj

s.t. (Ψi, Ψj) = δij ∀ 1 ≤ i, j ≤ m, 1 ≤ m ≤ p, ωi > 0, i = 1, . . . , n. POD basis functions have global character Typical: m << p captures most of system energy ⇒ low-order model

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 7

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Proper Orthogonal Decomposition

Reduced-order models based on proper orthogonal decomposition (POD) have the potential to simulate the Navier-Stokes equations with much less computational effort. Elements (snapshots) ui ∈ H, i = 1, . . . , n (p = dim{u1, . . . , un} ≥ 1) taken from some Hilbert space H. Find orthonormal basis {Ψi}p

i=1 for space spanned by snapshots, such that

min

Ψ1,...,Ψm n

ωi

ui −

m

(ui, Ψj)HΨj

s.t. (Ψi, Ψj) = δij ∀ 1 ≤ i, j ≤ m, 1 ≤ m ≤ p, ωi > 0, i = 1, . . . , n. POD basis functions have global character Typical: m << p captures most of system energy ⇒ low-order model

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 8

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Proper Orthogonal Decomposition

Reduced-order models based on proper orthogonal decomposition (POD) have the potential to simulate the Navier-Stokes equations with much less computational effort. Elements (snapshots) ui ∈ H, i = 1, . . . , n (p = dim{u1, . . . , un} ≥ 1) taken from some Hilbert space H. Find orthonormal basis {Ψi}p

i=1 for space spanned by snapshots, such that

min

Ψ1,...,Ψm n

ωi

ui −

m

(ui, Ψj)HΨj

s.t. (Ψi, Ψj) = δij ∀ 1 ≤ i, j ≤ m, 1 ≤ m ≤ p, ωi > 0, i = 1, . . . , n. POD basis functions have global character Typical: m << p captures most of system energy ⇒ low-order model

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 9

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Proper Orthogonal Decomposition

Reduced-order models based on proper orthogonal decomposition (POD) have the potential to simulate the Navier-Stokes equations with much less computational effort. Elements (snapshots) ui ∈ H, i = 1, . . . , n (p = dim{u1, . . . , un} ≥ 1) taken from some Hilbert space H. Find orthonormal basis {Ψi}p

i=1 for space spanned by snapshots, such that

min

Ψ1,...,Ψm n

ωi

ui −

m

(ui, Ψj)HΨj

s.t. (Ψi, Ψj) = δij ∀ 1 ≤ i, j ≤ m, 1 ≤ m ≤ p, ωi > 0, i = 1, . . . , n. POD basis functions have global character Typical: m << p captures most of system energy ⇒ low-order model

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 10

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Driven cavity flow, 8 POD basis functions

σ1 = .2312, σ2 = .0827, σ3 = .0353, σ4 = .0139 σ5 = .0055, σ6 = .0021, σ7 = .0007, σ8 = .0003

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 11

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Galerkin POD-Based Model

Assume boundary velocity separable: g(t, x) = γ(t)h(x) Expand velocity in terms of POD basis: Note separation of variables t and x ˜ u(t, x) = un(x) + γ(t)uc(x) +

m

yi(t)Ψi(x). Project momentum equation onto basis for POD modes: ˙ yj(t) = −ν (∇˜ u, ∇Ψj) − ˙ γ(t) (uc, Ψj) − (˜ u · ∇˜ u, Ψj) , j = 1, . . . , m. Expansion of bilinear terms leads to ODE system for modes: ˙ y(t) = M0 + γ(t)M1 + γ2(t)M2 + ˙ γ(t)Mc + M3y + γ(t)M4y + M5(y, y) with M0, M1, M2, Mc ∈ Rm, M3, M4 ∈ Rm,m, M5(y, y) : Rm,m → Rm

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 12

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Galerkin POD-Based Model

Assume boundary velocity separable: g(t, x) = γ(t)h(x) Expand velocity in terms of POD basis: Note separation of variables t and x ˜ u(t, x) = un(x) + γ(t)uc(x) +

m

yi(t)Ψi(x). Project momentum equation onto basis for POD modes: ˙ yj(t) = −ν (∇˜ u, ∇Ψj) − ˙ γ(t) (uc, Ψj) − (˜ u · ∇˜ u, Ψj) , j = 1, . . . , m. Expansion of bilinear terms leads to ODE system for modes: ˙ y(t) = M0 + γ(t)M1 + γ2(t)M2 + ˙ γ(t)Mc + M3y + γ(t)M4y + M5(y, y) with M0, M1, M2, Mc ∈ Rm, M3, M4 ∈ Rm,m, M5(y, y) : Rm,m → Rm

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 13

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Galerkin POD-Based Model

Assume boundary velocity separable: g(t, x) = γ(t)h(x) Expand velocity in terms of POD basis: Note separation of variables t and x ˜ u(t, x) = un(x) + γ(t)uc(x) +

m

yi(t)Ψi(x). Project momentum equation onto basis for POD modes: ˙ yj(t) = −ν (∇˜ u, ∇Ψj) − ˙ γ(t) (uc, Ψj) − (˜ u · ∇˜ u, Ψj) , j = 1, . . . , m. Expansion of bilinear terms leads to ODE system for modes: ˙ y(t) = M0 + γ(t)M1 + γ2(t)M2 + ˙ γ(t)Mc + M3y + γ(t)M4y + M5(y, y) with M0, M1, M2, Mc ∈ Rm, M3, M4 ∈ Rm,m, M5(y, y) : Rm,m → Rm

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 14

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Driven Cavity Velocity Profile

γ(t) = 1

2 + 3 π [sin (πt/20) + 1 3 sin (3πt/10) + 1 5 sin (5πt/10) + 1 7 sin (7πt/10) + 1 9 sin (9πt/10)]

Reynolds number: Re = ν−1 = 400 Simulation time: T = 20 sec. 100 snapshots Mesh: 49 × 49 Truncation: 99.9% of system energy Direct projection of snapshots onto POD basis: ˆ yi(t) = (u − un(x) − γ(t)uc(x), Ψi) Projected velocity model: ˆ u(t, x) = un(x) + γ(t)uc(x) +

m

ˆ yi(t)Ψi(x).

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 15

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Projected and Predicted Modes at Re = 400

2 4 6 8 10 12 14 16 18 20 0.2 0.4 0.6 0.8 1 1.2 1.4 x 10

−4

Absolute Simulation Error (n=100 snapshots; m=9 basis functions) Time Error Prediction Projection

9 basis functions for 99.9% of energy (9 modes) Compare projection error u − ˆ u2 and prediction error u − ˜ u2 Predicted error slightly greater as expected, but both ≤ 10−4 Excellent approximation at Re=400

Limits of POD in optimization POD basis derived from numerical data. Model fidelity dependent on problem data (boundary conditions, Reynolds number, etc.) Model must be reset repeatedly during optimization process. Computationally expensive!

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 16

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Projected and Predicted Modes at Re = 400

2 4 6 8 10 12 14 16 18 20 0.2 0.4 0.6 0.8 1 1.2 1.4 x 10

−4

Absolute Simulation Error (n=100 snapshots; m=9 basis functions) Time Error Prediction Projection

9 basis functions for 99.9% of energy (9 modes) Compare projection error u − ˆ u2 and prediction error u − ˜ u2 Predicted error slightly greater as expected, but both ≤ 10−4 Excellent approximation at Re=400

Limits of POD in optimization POD basis derived from numerical data. Model fidelity dependent on problem data (boundary conditions, Reynolds number, etc.) Model must be reset repeatedly during optimization process. Computationally expensive!

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 17

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Projected and Predicted Modes at Re = 400

2 4 6 8 10 12 14 16 18 20 0.2 0.4 0.6 0.8 1 1.2 1.4 x 10

−4

Absolute Simulation Error (n=100 snapshots; m=9 basis functions) Time Error Prediction Projection

9 basis functions for 99.9% of energy (9 modes) Compare projection error u − ˆ u2 and prediction error u − ˜ u2 Predicted error slightly greater as expected, but both ≤ 10−4 Excellent approximation at Re=400

Limits of POD in optimization POD basis derived from numerical data. Model fidelity dependent on problem data (boundary conditions, Reynolds number, etc.) Model must be reset repeatedly during optimization process. Computationally expensive!

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 18

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Projected and Predicted Modes at Re = 400

2 4 6 8 10 12 14 16 18 20 0.2 0.4 0.6 0.8 1 1.2 1.4 x 10

−4

Absolute Simulation Error (n=100 snapshots; m=9 basis functions) Time Error Prediction Projection

9 basis functions for 99.9% of energy (9 modes) Compare projection error u − ˆ u2 and prediction error u − ˜ u2 Predicted error slightly greater as expected, but both ≤ 10−4 Excellent approximation at Re=400

Limits of POD in optimization POD basis derived from numerical data. Model fidelity dependent on problem data (boundary conditions, Reynolds number, etc.) Model must be reset repeatedly during optimization process. Computationally expensive!

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 19

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Projected and Predicted Modes at Re = 400

2 4 6 8 10 12 14 16 18 20 0.2 0.4 0.6 0.8 1 1.2 1.4 x 10

−4

Absolute Simulation Error (n=100 snapshots; m=9 basis functions) Time Error Prediction Projection

9 basis functions for 99.9% of energy (9 modes) Compare projection error u − ˆ u2 and prediction error u − ˜ u2 Predicted error slightly greater as expected, but both ≤ 10−4 Excellent approximation at Re=400

Limits of POD in optimization POD basis derived from numerical data. Model fidelity dependent on problem data (boundary conditions, Reynolds number, etc.) Model must be reset repeatedly during optimization process. Computationally expensive!

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 20

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Standard POD on Rough Mesh at Re = 10, 000

Suggestion: Combine POD with coarser grids Questions: Coarser grids and higher Reynolds numbers? Try Re = 10, 000 on 13 × 13 mesh with γ(t) ≡ 1.0 ODE solver fails to converge Standard POD method useless Question: What to do? Distinguish between POD for physical model and POD for numerical model. POD should conform to numerical model. Stability problems well-known from standard numerical approximation methods for Navier-Stokes equations

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 21

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Standard POD on Rough Mesh at Re = 10, 000

Suggestion: Combine POD with coarser grids Questions: Coarser grids and higher Reynolds numbers? Try Re = 10, 000 on 13 × 13 mesh with γ(t) ≡ 1.0

2 4 6 8 10 12 14 16 18 20 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25 0.3 Mode Amplitudes (n=100 snapshots; m=13 basis functions) Time (seconds) Mode amplitude Direct Projection POD Modes

ODE solver fails to converge Standard POD method useless Question: What to do? Distinguish between POD for physical model and POD for numerical model. POD should conform to numerical model. Stability problems well-known from standard numerical approximation methods for Navier-Stokes equations

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 22

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Standard POD on Rough Mesh at Re = 10, 000

Suggestion: Combine POD with coarser grids Questions: Coarser grids and higher Reynolds numbers? Try Re = 10, 000 on 13 × 13 mesh with γ(t) ≡ 1.0

2 4 6 8 10 12 14 16 18 20 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25 0.3 Mode Amplitudes (n=100 snapshots; m=13 basis functions) Time (seconds) Mode amplitude Direct Projection POD Modes

ODE solver fails to converge Standard POD method useless Question: What to do? Distinguish between POD for physical model and POD for numerical model. POD should conform to numerical model. Stability problems well-known from standard numerical approximation methods for Navier-Stokes equations

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 23

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion The Standard POD Method An Example Problem Failure of the Standard POD-Based Model

Use of POD Concept

Problematic Approach Generate snap shots using a numerical scheme with stabilization Generate POD from Galerkin approximation based on physical model Suggested Approach Generate snap shots using a numerical scheme with stabilization Generate POD from Galerkin approximation based on numerical model Upwinding not suitable in this context.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 24

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

Sources of Instabilities in Navier-Stokes Solvers

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −0.5 0.5 1 Exact solution h=1/10 h=ν

Dominant convective terms Example: −νu′′ + u′ = 0 in Ω = (0, 1) u(0) = 0, u(1) = 1, ν = 1/100 FE discretization leads to central difference scheme for convective term u′ Common solutions: Upwinding Streamline diffusion

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 25

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

Streamline diffusion

Find uh ∈ Vh and ph ∈ Qh, such that (uh, vh) + νa(uh, vh) + ˜ n(uh, uh, vh) + b(vh, ph) = 0 ∀ vh ∈ Vh b(uh, qh) = 0 ∀ qh ∈ Qh, Modified convective term ˜ n(uh, vh, wh) = n(uh, vh, wh) +

T∈T h

δT(uh · ∇vh, uh · ∇wh)|T Local damping parameter: δT = δ · hT u ∞ · 2ReT 1 + ReT , ReT = uT · hT/ν Only δT = δT(u, h) is nonlinear in u

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 26

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

The Streamline Diffusion POD Method

Harmonize POD-based model and POD basis functions by adding streamline diffusion regularization to the Galerkin POD projection: ˙ yj(t) = −ν (∇u, ∇Ψj) − ˙ γ(t) (uc, Ψj) − (u · ∇u, Ψj) − δT (u · ∇u, u · ∇Ψj) For large Reynolds numbers δT ≈ δm

T = δ ·

2hT u ∞ SDPOD ODE System

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 27

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

The Streamline Diffusion POD Method

Harmonize POD-based model and POD basis functions by adding streamline diffusion regularization to the Galerkin POD projection: ˙ yj(t) = −ν (∇u, ∇Ψj) − ˙ γ(t) (uc, Ψj) − (u · ∇u, Ψj) − δT (u · ∇u, u · ∇Ψj) For large Reynolds numbers δT ≈ δm

T = δ ·

2hT u ∞ SDPOD ODE System

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 28

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

The Streamline Diffusion POD Method

Harmonize POD-based model and POD basis functions by adding streamline diffusion regularization to the Galerkin POD projection: ˙ yj(t) = −ν (∇u, ∇Ψj) − ˙ γ(t) (uc, Ψj) − (u · ∇u, Ψj) − δT (u · ∇u, u · ∇Ψj) For large Reynolds numbers δT ≈ δm

T = δ ·

2hT u ∞ SDPOD ODE System ˙ y(t) = M0 + γ(t)M1 + γ2(t)M2 + ˙ γ(t)Mc + M3y + γ(t)M4y + M5(y, y)

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 29

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

The Streamline Diffusion POD Method

Harmonize POD-based model and POD basis functions by adding streamline diffusion regularization to the Galerkin POD projection: ˙ yj(t) = −ν (∇u, ∇Ψj) − ˙ γ(t) (uc, Ψj) − (u · ∇u, Ψj) − δT (u · ∇u, u · ∇Ψj) For large Reynolds numbers δT ≈ δm

T = δ ·

2hT u ∞ SDPOD ODE System ˙ y(t) = M0 + γ(t)M1 + γ2(t)M2 + ˙ γ(t)Mc + M3y + γ(t)M4y + M5(y, y) + δm

T

0 + γ(t)Mδ 1 + γ2(t)Mδ 2 + Mδ 3y + γ(t)Mδ 4y

+ Mδ

5(y, y) + γ3(t)Mδ 6 + γ2Mδ 7y + γ(t)Mδ 8(y, y) + Mδ 9(y, y, y)

Bret Kragel and Ekkehard W. Sachs

Proper Orthogonal Decomposition in Optimization

SLIDE 30

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

POD vs. SDPOD at Re = 10, 000 (δ = 1.0)

Discr. m %Energy EPROJ ESDPOD EPOD 4 × 4 5 99.9 8.4037 e-6 9.7677 e-6 1.8121 e-1 7 × 7 10 99.9 1.1408 e-5 1.3565 e-5 9.1854 e-2 13 × 13 15 99.9 6.6566 e-6 9.1258 e-6 9.8077 e-2 25 × 25 20 99.9 4.5393 e-6 7.4863 e-6 1.1682 e-2 49 × 49 24 99.9 3.6355 e-6 5.0878 e-6 7.4593 e-3 97 × 97 30 99.9 6.4540 e-6 9.2616 e-6 1.0731 e-2 Observations SDPOD and projection errors order of 10−5 SDPOD error slightly larger, but excellent agreement POD error comparatively large - even on finer grids

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 31

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

Graphical Comparison: POD vs. SDPOD at Re = 10, 000 (δ = 1.0)

2 4 6 8 10 12 14 16 18 20 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 Absolute Error (n=100 snapshots) 4 × 4 Mesh 7 × 7 Mesh 13 × 13 Mesh 25 × 25 Mesh 49 × 49 Mesh 97 × 97 Mesh

POD Method

2 4 6 8 10 12 14 16 18 20 0.5 1 1.5 2 2.5 3 x 10

−5

Absolute Error. (n=100 snapshots) 4 × 4 Mesh 7 × 7 Mesh 13 × 13 Mesh 25 × 25 Mesh 49 × 49 Mesh 97 × 97 Mesh

SDPOD Method

Similar phenomenon when underlying model data changes well-known from POD literature. Changing boundary condition leads to model deterioration ⇒ POD basis update Question: When to update? ⇒ Need framework for managing model updates. (SMF)

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 32

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Sources POD Failure The SDPOD Model

POD Basis Functions for u0 and u5 (red)

5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30 5 10 15 20 25 30

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 33

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Model Functions in Optimization

Given g ∈ I Rn, how to find a ’better’ point ? ’Better’: J (g+) < J (g). Difficult problem, since J nonlinear. Approximate J (·) by model function: linear or quadratic or less complex than J . (POD models, multifidelity models, neural networks,...) Surrogate Optimization. J (g + s) ≈ m(g + s) = J (g) + ∇J (g)Ts J (g + s) ≈ m(g + s) = J (g) + ∇J (g)Ts + 1 2sT∇2J (g)s

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 34

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Model Minimization

Given a model function m(g + s) for J (g + s). Replace minimize J (u + s) by minimize m(u + s). Validity of model m(g + s) ? Restrict s to a region where we trust model. ’Democratic’ choice: s : s ≤ ∆. (P) min m(g + s) s ≤ ∆ Trust Region Problem

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 35

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Model Adaption

Choice of ∆ ? When to update trust region ? Let s(∆) denote the solution of trust region problem. If m(g + s) is a good model for φ, choose large ∆, otherwise small ∆. If predicted reduction m(g + s(∆)) − m(g) is close to actual reduction J (g + s(∆)) − J (g), then accept s(∆) and keep or increase trust region radius ∆,

therwise take smaller ∆ and recompute s(∆).

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 36

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Inexact Gradient Information

Carter 1991 Suppose that no exact gradient information is available for model function m(g + s) = J (g) + nTs + 1 2sTHs Exact information for value J (g) is not necessary. Gradient information has to improve as iteration progresses. Theorem Let J be twice cont. differentiable and bounded below on level set. Let sufficient decrease condition hold, and Hk be bounded. If nk → 0 and ∇J (gk) − nk nk ≤ ζ < 1 then lim inf

k→∞ ∇J (gk) = 0

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 37

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Nonlinear Model Function

Toint 1988, Carter 1991, Fahl 2000 Nonlinear model functions m(g + s). Sufficient decrease condition m(g + s) − m(g) ≤ c(m(g + sCP) − m(g)) where sCP is a descent step in direction of negative gradient. Need a replacement for second order information: measure of curvature ω(m, g, s) = 2 s2 (m(g + s) − m(g) − ∇m(g)Ts)

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 38

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Nonlinear Model Function, continued

Set nk = ∇mk(gk). Then previous theorems can be extended to. Theorem Let f be twice cont. differentiable and bounded below on level set. Let sufficient decrease condition hold, and ω(mk, gk, sk) be bounded. If nk → 0 and ∇J (gk) − nk nk ≤ ζ < 1 then lim inf

k→∞ ∇J (gk) = 0

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 39

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Adaption of POD Model

Arian, Fahl, Sachs Afanasiev, Hinze Kunisch, Volkwein Gunzburger Willcox Model Management: Alexandrov, Lewis Dennis

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 40

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u (SD)POD-based model Constrain model mk(gk + sk) to trust-region sk ≤ δk Compute snapshot set U+ using gk + sk and accept or reject new iterate based on

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Fahl [2000]: Convergence of Trust-Region POD (TRPOD) method under appropriate conditions.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 41

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u (SD)POD-based model Constrain model mk(gk + sk) to trust-region sk ≤ δk Compute snapshot set U+ using gk + sk and accept or reject new iterate based on

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Fahl [2000]: Convergence of Trust-Region POD (TRPOD) method under appropriate conditions.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 42

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u (SD)POD-based model Constrain model mk(gk + sk) to trust-region sk ≤ δk Compute snapshot set U+ using gk + sk and accept or reject new iterate based on

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Fahl [2000]: Convergence of Trust-Region POD (TRPOD) method under appropriate conditions.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 43

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u (SD)POD-based model Constrain model mk(gk + sk) to trust-region sk ≤ δk Compute snapshot set U+ using gk + sk and accept or reject new iterate based on

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Fahl [2000]: Convergence of Trust-Region POD (TRPOD) method under appropriate conditions.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 44

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u (SD)POD-based model Constrain model mk(gk + sk) to trust-region sk ≤ δk Compute snapshot set U+ using gk + sk and accept or reject new iterate based on

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Fahl [2000]: Convergence of Trust-Region POD (TRPOD) method under appropriate conditions.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 45

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region POD (TRPOD)

Goal: Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u SDPOD-based model Initialization: Choose 0 < η1 < η2 < 1, 0 < γ1 < γ2 < 1 < γ3, an initial trust-region radius δ0 > 0 and an initial iterate g0. Compute snapshot set U0 and J (g0). Set k = 0 Step 1: Compute POD basis and POD-based model mk(gk) Step 2: Compute approximately: min mk(gk + sk) s.t. sk ≤ δ0 Step 3: Compute snapshot set U+ using gk + sk and set

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Step 4: Update Trust-Region Radius

ρk ≥ η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [δk, γ3δk), k = k + 1, → Step 1 η1 ≤ ρk < η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [γ2δk, δk), k = k + 1, → Step 1 ρk < η1: Uk+1 = Uk, δk+1 ∈ [γ1δk, γ2δk), k = k + 1, → Step 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 46

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region POD (TRPOD)

Goal: Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u SDPOD-based model Initialization: Choose 0 < η1 < η2 < 1, 0 < γ1 < γ2 < 1 < γ3, an initial trust-region radius δ0 > 0 and an initial iterate g0. Compute snapshot set U0 and J (g0). Set k = 0 Step 1: Compute POD basis and POD-based model mk(gk) Step 2: Compute approximately: min mk(gk + sk) s.t. sk ≤ δ0 Step 3: Compute snapshot set U+ using gk + sk and set

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Step 4: Update Trust-Region Radius

ρk ≥ η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [δk, γ3δk), k = k + 1, → Step 1 η1 ≤ ρk < η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [γ2δk, δk), k = k + 1, → Step 1 ρk < η1: Uk+1 = Uk, δk+1 ∈ [γ1δk, γ2δk), k = k + 1, → Step 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 47

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region POD (TRPOD)

Goal: Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u SDPOD-based model Initialization: Choose 0 < η1 < η2 < 1, 0 < γ1 < γ2 < 1 < γ3, an initial trust-region radius δ0 > 0 and an initial iterate g0. Compute snapshot set U0 and J (g0). Set k = 0 Step 1: Compute POD basis and POD-based model mk(gk) Step 2: Compute approximately: min mk(gk + sk) s.t. sk ≤ δ0 Step 3: Compute snapshot set U+ using gk + sk and set

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Step 4: Update Trust-Region Radius

ρk ≥ η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [δk, γ3δk), k = k + 1, → Step 1 η1 ≤ ρk < η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [γ2δk, δk), k = k + 1, → Step 1 ρk < η1: Uk+1 = Uk, δk+1 ∈ [γ1δk, γ2δk), k = k + 1, → Step 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 48

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region POD (TRPOD)

Goal: Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u SDPOD-based model Initialization: Choose 0 < η1 < η2 < 1, 0 < γ1 < γ2 < 1 < γ3, an initial trust-region radius δ0 > 0 and an initial iterate g0. Compute snapshot set U0 and J (g0). Set k = 0 Step 1: Compute POD basis and POD-based model mk(gk) Step 2: Compute approximately: min mk(gk + sk) s.t. sk ≤ δ0 Step 3: Compute snapshot set U+ using gk + sk and set

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Step 4: Update Trust-Region Radius

ρk ≥ η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [δk, γ3δk), k = k + 1, → Step 1 η1 ≤ ρk < η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [γ2δk, δk), k = k + 1, → Step 1 ρk < η1: Uk+1 = Uk, δk+1 ∈ [γ1δk, γ2δk), k = k + 1, → Step 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 49

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region POD (TRPOD)

Goal: Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u SDPOD-based model Initialization: Choose 0 < η1 < η2 < 1, 0 < γ1 < γ2 < 1 < γ3, an initial trust-region radius δ0 > 0 and an initial iterate g0. Compute snapshot set U0 and J (g0). Set k = 0 Step 1: Compute POD basis and POD-based model mk(gk) Step 2: Compute approximately: min mk(gk + sk) s.t. sk ≤ δ0 Step 3: Compute snapshot set U+ using gk + sk and set

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Step 4: Update Trust-Region Radius

ρk ≥ η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [δk, γ3δk), k = k + 1, → Step 1 η1 ≤ ρk < η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [γ2δk, δk), k = k + 1, → Step 1 ρk < η1: Uk+1 = Uk, δk+1 ∈ [γ1δk, γ2δk), k = k + 1, → Step 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 50

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region POD (TRPOD)

Goal: Replace objective J (g) = J (u(g), g) with cheaper objective m(g) := J (˜ u(g), g) during optimization – ˜ u SDPOD-based model Initialization: Choose 0 < η1 < η2 < 1, 0 < γ1 < γ2 < 1 < γ3, an initial trust-region radius δ0 > 0 and an initial iterate g0. Compute snapshot set U0 and J (g0). Set k = 0 Step 1: Compute POD basis and POD-based model mk(gk) Step 2: Compute approximately: min mk(gk + sk) s.t. sk ≤ δ0 Step 3: Compute snapshot set U+ using gk + sk and set

ρk = J (gk) − J (gk + sk) mk(gk) − mk(gk + sk)

Step 4: Update Trust-Region Radius

ρk ≥ η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [δk, γ3δk), k = k + 1, → Step 1 η1 ≤ ρk < η2: Uk+1 = U+, gk+1 = gk + sk, J (gk+1) = J (gk + sk), δk+1 ∈ [γ2δk, δk), k = k + 1, → Step 1 ρk < η1: Uk+1 = Uk, δk+1 ∈ [γ1δk, γ2δk), k = k + 1, → Step 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 51

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 1 x0

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 52

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 1 Radius: δ0 x0

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 53

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 1 Radius: δ0 ρ ≥ η2: Accept x∗ x0 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 54

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 1 Radius: δ0 ρ ≥ η2: Accept x∗ Choose δ1 ∈ (δ0, γ3δ0) x0 x1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 55

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 2 Radius: δ1 x1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 56

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 2 Radius: δ1 η1 ≤ ρ ≤ η2: Accept x∗ x1 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 57

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 2 Radius: δ1 η1 ≤ ρ ≤ η2: Accept x∗ Choose δ2 ∈ (γ2δ1, δ1] x1 x2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 58

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 3 Radius: δ2 x2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 59

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 3 Radius: δ2 η1 ≤ ρ ≤ η2: Accept x∗ x2 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 60

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 3 Radius: δ2 η1 ≤ ρ ≤ η2: Accept x∗ Choose δ3 ∈ (γ2δ2, δ2] x2 x3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 61

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 x3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 62

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 ρ ≤ η1: Reject x∗ x3 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 63

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 ρ ≤ η1: Reject x∗ Choose δ3 ∈ (γ1δ3, γ2δ3] x3 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 64

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 ρ ≤ η1: Reject x∗ Choose δ3 ∈ (γ1δ3, γ2δ3] x3 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 65

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 x3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 66

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 η1 ≤ ρ ≤ η2: Accept x∗ x3 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 67

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 η1 ≤ ρ ≤ η2: Accept x∗ Choose δ4 ∈ (γ2δ3, δ3] x3 x4

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 68

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 5 Radius: δ4 x4

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 69

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 5 Radius: δ4 η1 ≤ ρ ≤ η2: Accept x∗ x4 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 70

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 5 Radius: δ4 η1 ≤ ρ ≤ η2: Accept x∗ Choose δ5 ∈ (γ2δ4, δ4] x4 x5

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 71

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 6 Radius: δ5 x5

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 72

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 17

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 73

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region POD Algorithm

Theorem (Fahl, 2000) Objective J continuously differentiable, bounded below and ∇J Lipschitz continuous on U ⊂ R. Model differentiable on open convex set containing trust-region, and nk = ∇mk(gk) approximates ∇J (gk), such that ∇Jk(gk) − nk nk ≤ ζ ∈ (0, 1). Then lim

k→∞ nk = 0.

Gradient consistency condition: nk → 0 ⇒ ∇J (gk) → 0 Consistency condition difficult for multiple mesh levels Reduction in model on one grid does not guarantee reduction of model

n another grid

Need extension to multilevel case (Gratton etal.)

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 74

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Recursive Multilevel Trust-Region Algorithm (RMTR)

Goal: Replace objective J (g) = J (u(g), g) with cheaper objective mi(g) := J ( ˜ ui(g), g) during optimization – ˜ ui POD-based model Initialization: Choose 0 < η1 < η2 < 1, 0 < γ1 < γ2 < 1 < γ3, an initial trust-region radius δ0 > 0 and an initial iterate gi,0. Compute snapshot set Ui,0 and J (gi,0). Set k = 0 Step 1: Compute POD basis and POD-based model mi,k(gi,k) Step 2: Compute approximately: min mi,k(gi,k + si,k) s.t. si,k ≤ δ0 Step 3: Compute snapshot set U+ using gi,k + si,k and set

ρi,k = J (gi,k) − J (gi,k + si,k) mi,k(gi,k) − mi,k(gi,k + si,k)

Step 4: If ρi,k ≥ η1, then gi,k+1 = gi,k + si,k, otherwise keep gi,k Step 5: Trust region update as usual, new snapshot set on finer grid for gi,k+1.

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 75

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Convergence of Multilevel Trust-Region Procedure for Quadratic Case

Theorem (Gratton, etal. 2005) If the algorithm RMTR is called at the highest level r with the gradient tolerance ǫg

r = 0, then global first-order convergence can be shown, that is,

lim

k→∞ nr,k = 0

provided we have quadratic model functions. Convergence results for quadratic model functions Extension to non-quadratic models requires additional assumptions Rigorous adaption of RMTR to POD-based model functions in work

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 76

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 1 x0

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 77

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 1 Radius: δ0 x0

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 78

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 1 Radius: δ0 ρ ≥ η2: Accept x∗ x0 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 79

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 1 Radius: δ0 ρ ≥ η2: Accept x∗ Choose δ1 ∈ (δ0, γ3δ0) x0 x1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 80

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 2 Radius: δ1 x1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 81

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 2 Radius: δ1 η1 ≤ ρ ≤ η2: Accept x∗ x1 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 82

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 2 Radius: δ1 η1 ≤ ρ ≤ η2: Accept x∗ Choose δ2 ∈ (γ2δ1, δ1] x1 x2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 83

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 3 Radius: δ2 x2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 84

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 3 Radius: δ2 η1 ≤ ρ ≤ η2: Accept x∗ x2 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 85

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 3 Radius: δ2 η1 ≤ ρ ≤ η2: Accept x∗ Choose δ3 ∈ (γ2δ2, δ2] x2 x3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 86

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 x3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 87

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 ρ ≤ η1: Reject x∗ x3 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 88

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 ρ ≤ η1: Reject x∗ Choose δ3 ∈ (γ1δ3, γ2δ3] x3 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 89

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 ρ ≤ η1: Reject x∗ Choose δ3 ∈ (γ1δ3, γ2δ3] x3 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 90

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 x3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 91

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 η1 ≤ ρ ≤ η2: Accept x∗ x3 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 92

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 4 Radius: δ3 η1 ≤ ρ ≤ η2: Accept x∗ Choose δ4 ∈ (γ2δ3, δ3] x3 x4

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 93

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 5 Radius: δ4 x4

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 94

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 5 Radius: δ4 η1 ≤ ρ ≤ η2: Accept x∗ x4 x∗

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 95

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 5 Radius: δ4 η1 ≤ ρ ≤ η2: Accept x∗ Choose δ5 ∈ (γ2δ4, δ4] x4 x5

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 96

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 6 Radius: δ5 x5

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 97

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region Procedure [0 < η1 ≤ η2 < 1, 0 < γ1 ≤ γ2 < 1 ≤ γ3] Iteration: 17

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 98

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region: Computational Complexity Iteration: 1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 99

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region: Computational Complexity Iteration: 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 100

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region: Computational Complexity Iteration: 3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 101

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region: Computational Complexity Iteration: 4

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 102

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region: Computational Complexity Iteration: 5

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 103

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region: Computational Complexity Iteration: 6

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 104

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Standard Trust-Region: Computational Complexity Iteration: 17

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 105

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 0 Iteration: 1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 106

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 0 Iteration: 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 107

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 0 Iteration: 3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 108

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 0 Iteration: 4

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 109

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 0 Iteration: 5

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 110

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 1 Iteration: 1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 111

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 1 Iteration: 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 112

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 1 Iteration: 3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 113

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 1 Iteration: 4

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 114

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 2 Iteration: 1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 115

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 2 Iteration: 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 116

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 2 Iteration: 3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 117

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 3 Iteration: 1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 118

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 3 Iteration: 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 119

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 3 Iteration: 3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 120

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 4 Iteration: 1

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 121

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Trust-Region Optimization

Recursive Multilevel Trust Region: Computational Complexity Level: 4 Iteration: 2

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 122

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

A Trust-Region Example

Target solution generated using truncated Fourier series introduced earlier Initial velocity profile: γ0(t) ≡ −1.0 Compare standard TRPOD method for Re = 10, 000 using 97 × 97 mesh With multilevel trust-region method using 4 × 4 to 97 × 97 meshes Tracking-type objective denoted by f

Level (i) Mesh ǫg

i (Re = 400)

ǫg

i (Re = 10, 000)

4 × 4 3.0e − 3 1.0e − 1 1 7 × 7 3.0e − 3 1.0e − 1 2 13 × 13 3.0e − 3 1.0e − 1 3 25 × 25 3.0e − 3 1.0e − 1 4 49 × 49 1.0e − 3 1.0e − 3 5 97 × 97 1.0e − 3 1.0e − 3

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 123

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Standard vs. Multilevel Trust-Region at Re = 400

Standard Trust-Region Method

i k △i,k

si,k
g0

i,k

gi,k
f(x−)

m(x−) m(xi,k) ρi,k m tNS 5 1 1.00 1.00e-0 4.229e-1 1.146e-1 2.76e-0 2.76e-0 5.06e-1 0.8220 6 3387 5 2 1.00 1.00e-0 1.094e-1 1.209e-2 9.07e-1 9.06e-1 2.76e-1 1.4009 7 3214 5 3 2.00 1.89e-1 2.549e-2 1.348e-3 2.32e-2 2.30e-2 1.55e-3 1.0649 10 3228 5 4 4.00 1.56e-2 1.827e-3 9.057e-4 4.02e-4 6.30e-4 4.98e-4 0.8396 11 3239 5 5 4.00 — 8.446e-4 8.446e-4 — — — — 11 3244

∆ trust region radius; s norm of steps; g gradient norms m(x) values of model function before and after minimization ρ (un)successful iteration; m dimension of snapshot set

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 124

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Standard vs. Multilevel Trust-Region at Re = 400

Multilevel Trust-Region Method

i k △i,k

si,k
g0

i,k

gi,k
f(x−)

m(x−) m(xi,k) ρi,k m tNS 1 1.00 1.00e-0 4.274e-1 2.133e-1 5.23e-0 5.23e-0 2.13e-0 1.1007 4 6 2 2.00 1.19e-0 2.637e-1 2.983e-4 1.81e-0 1.81e-0 2.45e-1 0.7763 2 6 3 2.00 1.86e-1 4.654e-2 1.181e-3 5.96e-1 5.96e-1 5.54e-1 0.9642 4 6 4 4.00 4.64e-2 9.227e-3 1.478e-3 5.57e-1 5.55e-1 5.51e-1 0.4854 3 6 5 4.00 — 1.281e-3 1.281e-3 — — — — 3 6 1 1 1.00 4.31e-1 5.889e-3 3.507e-4 4.88e-1 4.98e-1 4.97e-1 1.9853 5 14 1 2 2.00 — 3.518e-4 3.518e-4 — — — — 5 14 2 1 1.00 2.92e-1 2.204e-2 2.164e-3 2.50e-1 2.50e-1 2.18e-1 0.9989 8 48 2 2 2.00 — 2.374e-3 2.374e-3 — — — — 7 48 3 1 1.00 7.16e-2 5.441e-3 1.635e-3 4.73e-2 4.74e-2 4.55e-2 0.7238 11 182 3 2 2.00 — 1.469e-3 1.469e-3 — — — — 11 182 4 1 1.00 1.38e-1 2.233e-2 6.121e-4 1.84e-2 1.85e-2 3.17e-3 1.0104 11 733 4 2 2.00 8.59e-3 1.165e-3 5.212e-4 2.87e-3 3.11e-3 3.06e-3 0.9465 11 730 4 3 2.00 — 5.960e-4 5.960e-4 — — — — 11 731 5 1 1.00 5.78e-2 1.125e-2 3.907e-4 3.21e-3 3.52e-3 3.43e-4 0.9942 11 3233 5 2 2.00 — 4.563e-4 4.563e-4 — — — — 11 3191

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 125

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Standard vs. Multilevel Trust-Region at Re = 400

2 4 6 8 10 12 14 16 18 20 −1 −0.5 0.5 1 1.5 2 2.5 Trust−region optimization at Re=400 time t γ(t) uopt iter 1 iter 2 iter 3 iter 4

Standard Trust-Region

2 4 6 8 10 12 14 16 18 20 0.5 1 1.5 2 2.5 Trust−region optimization at Re=400 time t γ(t) uopt Level 1 Level 2 Level 3 Level 4

Multilevel Trust-Region

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 126

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Standard vs. Multilevel Trust-Region at Re = 10, 000

Standard Trust-Region Method with SDPOD

i k △i,k

si,k
g0

i,k

gi,k
f(x−)

m(x−) m(xi,k) ρi,k m tNS 5 1 1.00 9.97e-1 5.666e-2 1.109e-2 5.75e-1 5.74e-1 1.33e-1 0.8141 24 3332 5 2 1.00 9.99e-1 1.609e-2 5.841e-3 2.17e-1 2.17e-1 7.58e-2 1.2879 9 3147 5 3 2.00 2.48e-1 4.617e-3 1.407e-3 3.54e-2 3.51e-2 8.43e-3 2.9592 29 3154 5 4 4.00 9.65e-3 1.003e-3 7.025e-4 1.05e-2 1.04e-2 1.02e-2 27.507 31 3186 5 5 8.00 9.44e-3 1.040e-3 6.920e-4 8.62e-3 8.56e-3 8.49e-3 3.1643 31 3142 5 6 16.0 — 6.550e-4 6.550e-4 — — — — 31 3171

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 127

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Multilevel Trust-Region Method with SDPOD

i k △i,k

si,k
g0

i,k

gi,k
f(x−)

m(x−) m(xi,k) ρi,k m tNS 1 1.00 1.00e-0 4.703e-1 1.687e-1 5.69e-0 5.69e-0 1.52e-0 0.9990 4 6 2 2.00 3.91e-1 1.356e-1 5.611e-2 1.52e-0 1.52e-0 1.17e-0 1.2248 3 6 3 4.00 — 5.089e-2 5.089e-2 — — — — 4 6 1 1 1.00 — 2.281e-2 2.281e-2 — — — — 8 15 2 1 1.00 — 1.094e-2 1.094e-2 — — — — 11 50 3 1 1.00 2.58e-1 1.032e-2 3.572e-3 2.34e-1 2.34e-1 2.13e-1 0.3351 13 186 3 2 1.00 — 2.284e-3 2.284e-3 — — — — 17 186 4 1 1.00 3.80e-1 9.162e-3 1.480e-3 1.25e-1 1.25e-1 1.01e-1 0.7223 17 748 4 2 1.00 — 3.001e-3 3.001e-3 — — — — 21 756 5 1 1.00 1.50e-1 9.244e-3 5.135e-4 2.28e-2 2.27e-2 1.68e-2 2.4468 28 3193 5 2 2.00 6.32e-2 1.450e-3 4.611e-4 8.32e-3 8.24e-3 7.58e-3 3.2435 26 3845 5 3 2.00 — 8.977e-4 8.977e-4 — — — — 31 3161

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 128

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion Trust-Region Methods Recursive Multilevel Trust-Region Methods Multilevel Optimization with TRPOD

Standard vs. Multilevel Trust-Region at Re = 10, 000

2 4 6 8 10 12 14 16 18 20 −1 −0.5 0.5 1 1.5 2 2.5 Trust−region optimization at Re=10,000 time t γ(t) uopt iter 1 iter 2 iter 3 iter 4 iter 5

Standard Trust-Region

2 4 6 8 10 12 14 16 18 20 −1 −0.5 0.5 1 1.5 2 2.5 Trust−region optimization at Re=10,000 time t γ(t) uopt level 0 level 3 level 4 level 5

Multilevel Trust-Region

Selected iterations for both procedures show similar results Multilevel procedure requires less computational effort

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization

SLIDE 129

The Standard POD Method Streamline Diffusion POD Models Optimization with TRPOD Conclusion

Conclusion

POD-based models can reduce computational effort required for

ptimization problems

POD approximation based on numerical scheme, not physical model Use of models derived from coarse grids can reduce the number of high-order solutions required Adapt POD models according to a trust region strategy

Bret Kragel and Ekkehard W. Sachs Proper Orthogonal Decomposition in Optimization