Adaptive Surrogate Modeling for Response Surface Approximations with - - PowerPoint PPT Presentation

Adaptive Surrogate Modeling for Response Surface Approximations with Application to Bayesian Inference

Serge Prudhommea and C. Bryantb

aDepartment of Mathematics, Ecole Polytechnique de Montr´

eal SRI Center for Uncertainty Quantification, KAUST

bICES, The University of Texas at Austin, USA

Advances in UQ Methods, Algorithms, and Applications KAUST, Thuwal, Saudi Arabia

January 6-9, 2015

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 1 / 35

Outline

Outline

• Introduction.
• Error estimation for PDEs with uncertain coefficients.
• Adaptive scheme.
• Numerical examples.
• Application to Bayesian Inference.

“. . . It is not possible to decide (a) between h or p refinement and (b) whether one should enrich the approximation space Vh or Sh . . . better approaches, yet to be conceived, are consequently needed.”

Spectral Methods for Uncertainty Quantification, Le Maˆ ıtre & Knio 2010

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 2 / 35

Introduction

Introduction

A(λ; u) = f(λ) → Q(u(λ))

• M(λ)=Q(u)

λ

1

λ2 QN(u)

Surrogate model MN

M ≈ MN(λ) = Q(uN) Ah(λ; uh) = fh(λ) → Q(uh(λ))

• Mh(λ)=Q(uh)

λ

1

λ2 Qh,N(u)

Surrogate model Mh,N

Mh ≈ Mh,N(λ) = Q(uh,N)

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 3 / 35

Introduction

References

Le Maˆ ıtre et al., 2007, 2010

◮ Polynomial chaos, Stochastic Galerkin, Burger’s equation

Almeida and Oden, 2010

◮ convection-diffusion, sparse grid collocation

Butler, Dawson, and Wildey, 2011

◮ Stochastic Galerkin, PC representation of the discretization error

(ignore truncation error)

Butler, Constantine, and Wildey, 2012

◮ Ignore physical discretization error, pseudo-spectral projection,

improved linear functional

. . .

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 4 / 35

Model Problem

Model Problem and Discretization

Model Problem: A(λ; u) = f(λ),

∀x ∈ D

Assume: λ = Λ(θ) =

k∈IN λkΨk(ξ(θ))

Non-intrusive approach (“pseudo-spectral projection method”):

u(x, ξ) ≈

• k∈IN

uk(x)Ψk(ξ) ≈

• k∈IN

um

k (x)Ψk(ξ) := uN(x, ξ)

with

uk(x) = u(x, ·), Ψk :=

• Ω

u(x, ξ)Ψk(ξ) ρ(ξ) dξ ≈

m(N)

• j=1

u(x, ξj) Ψk(ξj) wj := um

k (x)

Pros: fast convergence, sampling-like u(x, ξj), choice of ξj

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 5 / 35

Model Problem

Model Problem and Discretization

Gaussian quadrature:

• select quadrature rule {ξj, wj}m(N)

j=1

according to ρξ

• integrand (uNΨN) is at least of order 2N in each dimension

m(N) ≥ (N + 1)n

Parameterized discrete solution (the surrogate model): Solve for uh(x, ξj) → uh,m

k

(x) = m(N)

j=1

uh(x, ξj) Ψk(ξj) wj uh,N(x, ξ) =

• k∈IN

uh,m

k

(x)Ψk(ξ)

Evaluate:

Qξ(u) − Qξ

• uh,N
• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 6 / 35

Goal-oriented error estimation

Goal-oriented error estimation

Weak formulation of A(λ(ξ); u) = f(ξ) Find u(ξ) ∈ V such that

Bξ(u, v) = Fξ(v) ∀v ∈ V

Find uh(ξ) ∈ V h ⊂ V such that

Bξ(uh, vh) = Fξ(vh) ∀vh ∈ V h

Quantity of interest (QoI):

Qξ(u) =

• D

q(x)u(x, ξ) dx

Adjoint problem: Find p(·, ξ) ∈ V such that

Bξ(v, p) = Qξ(v) ∀v ∈ V Error representation Qξ(u) − Qξ(uh) = Fξ(p) − Bξ(uh, p) := Rξ(uh; p)

• Note: Qξ(u) − Qξ(uh) = Rξ(uh; p) + ∆B ≈ Rξ(uh; p)
• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 7 / 35

Goal-oriented error estimation

Goal-oriented error estimation

Error estimator:

Qξ(u) − Qξ(uh) = Rξ(uh; p) ≈ η(ξ)

Orthogonality property: If ph ∈ V h then Rξ(uh; ph) = 0 Higher-order approximation of adjoint solution: Compute p+(ξ) ∈ V +, V h ⊂ V + ⊂ V and

η(ξ) = Rξ(uh; p+)

Other choices

• Local interpolation: Rξ(uh; p) ≈ Rξ(uh; π+ph − ph)
• Residual based: Rξ(uh; p) = Bξ(eu, ep) ≈ ηu(ξ)ηp(ξ)

1Becker & Rannacher 2001, Oden & Prudhomme, 2001

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 8 / 35

Goal-oriented error estimation

Case with Uncertain Parameters

Since uh,N(·, ξ) ∈ V h ⊂ V , the adjoint equation still holds

Bξ

• uh,N, p
• = Qξ
• uh,N

New error representation:

Qξ

• u
• − Qξ
• uh,N

= Rξ

• uh,N; p
• = Rξ
• uh,N; p+

+ Rξ

• uh,N; p − p+

= Rξ

• uh,N; p+,N

+ Rξ

• uh,N; p+ − p+,N

+ Rξ

• uh,N; p − p+

Total Error Estimate:

Qξ

• u
• − Qξ
• uh,N

≈ Rξ

• uh,N; p+,N

1Butler et al., 2012, Almeida and Oden, 2010

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 9 / 35

Goal-oriented error estimation

Proposed error decomposition

Qξ

• u
• − Qξ
• uh,N

= Qξ

• u − uh
• error due to

physical discretization

+ Qξ

• uh − uh,N
• error due to approx

in parameter space

• + ∆Qξ
• Total error:

Qξ

• u
• − Qξ
• uh,N

≈ Rξ

• uh,N; p+,N

:= E(ξ) 2 × poly. eval 1 × inner product

Physical space discretization error:

Qξ

• u
• − Qξ
• uh

≈ Rξ

• uh; p+

:= ED(ξ) 2 × pde solve 1 × inner product

Parameter space discretization error:

Qξ

• uh − uh,N

≈ Rξ

• uh,N; p+,N

− Rξ

• uh; p+

:= EΩ(ξ)

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 10 / 35

Goal-oriented error estimation

Summary of the procedure

• Solve forward and adjoint problems at quadrature points,
• uh(x, ξj)

m(N)

j=1

{p+(x, ξj)}m(N)

j=1

→ Rξj

• uh; p+
• Construct fully discrete solutions and PC expansion for ED

uh,N(x, ξ) =

k∈IN uh,m k

(x)Ψk(ξ) p+,N(x, ξ) =

k∈IN p+,m k

(x)Ψk(ξ) ED(ξ) =

• k∈IN

eD

k Ψk(ξ)

• Construct E and EΩ

E(ξ) =

• k∈IM

ekΨk(ξ) EΩ(ξ) = E(ξ) − ED(ξ)

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 11 / 35

Adaptive Strategy

Adaptivity Strategy

if

• ED

>

• EΩ
• Refine physical approximation space V h

(h ← h

2 )

else

Refine random approximation space SN (N ← N + 1)

end

• for a given physical mesh, refine approximation in Ω to the level of

physical discretization error

• use error indicator to guide h refinement in parameter space
• anisotropic p-refinement in higher dimensions
• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 12 / 35

Numerical results

Example 1: Smooth response surface in 2D

Convection-diffusion problem in 2D:

−∇ · (2∇u) + 10 sin π

2 ξ1

• 10 cos (πξ2)
• · ∇u = f(ξ)

in D = (0, 1)2

u = 0

• n ∂D

Loading f is chosen such that, with ξ1, ξ2 ∼ U(0, 1):

u(x, y, ξ) = 400

• ξ1(x − x2)e− 20

ξ1 (x−ξ1)2

ξ2(y − y2)e− 20

ξ2 (y−ξ2)2

Quantity of interest:

Q(u(·, ξ)) = 1 4 1

0.5

1

0.5

u(x, y, ξ) dxdy ≈

• D

q(x, y) u(x, y, ξ) dxdy

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 13 / 35

Numerical results

Example 1: Effectivity indices

EΩL2

Ω

EDL2

Ω

EL2

Ω

EL2

Ω

Q(u)−Q(uh,p,N )L2

Ω

5.12427e-01 3.28574e-03 4.34727e-01 .851 1.79962e-01 3.48349e-03 1.95149e-01 .782 5.23817e-02 6.59002e-03 4.25596e-02 .921 2.30547e-02 3.77558e-03 2.85842e-02 .949 6.17006e-03 5.77325e-03 8.41438e-03 .998 2.21929e-03 4.48790e-03 7.25161e-03 .987 2.20458e-03 3.98680e-04 2.80610e-03 .984 7.00606e-04 4.31703e-04 9.24221e-04 .990 3.58282e-04 4.13817e-04 8.06397e-04 1.01 3.58118e-04 1.47612e-04 5.17592e-04 1.03 1.38497e-04 1.49756e-04 2.71081e-04 1.11 8.78811e-05 2.61145e-05 1.06502e-04 1.02 5.10997e-05 2.59334e-05 7.73100e-05 1.00 1.34534e-05 2.59640e-05 3.87553e-05 .985 1.33674e-05 1.22096e-05 2.57607e-05 .981

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 14 / 35

Numerical results

Example 2: Response surface with discontinuity

Convection-diffusion model in 2D:

−2∆u + sin( 3π

2 ξ1)

4⌊ξ2 − ξ1⌋

• · ∇u = f(ξ)

in D = (0, 1)2

u = 0

• n ∂D

Loading f is chosen so that

u(x, y, ξ) = 10 sin 3π 2 ξ1

• 4⌊ξ2 − ξ1⌋
• · (x − x2)(y − y2)

where

⌊ξ2 − ξ1⌋ =

• ξ1 ≤ ξ2

−1 ξ1 > ξ2

with ξ1, ξ2 ∼ U(0, 1).

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 15 / 35

Numerical results

Example 2: Response surface with discontinuity

Q(u(·, ξ)) = u 1 3, 1 3, ξ

• ≈
• D

q(x, y) u(x, y, ξ) dxdy q(x, y) = 100 π exp

• − 100(x − 1/3)2 − 100(y − 1/3)2

0.2 0.4 0.6 0.8 1 0.5 1 −2 −1 1 2 ξ1 ξ2

True response for QoI over parameter space.

Adaptive scheme:

If EΩiL2

Ωi > 0.75 maxj

• EΩj
• L2

Ωj

split Ωi into 2n new elements by bisection in each stochastic direction end

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 16 / 35

Numerical results

Example 2: Response surface with discontinuity

Adaptive hΩ refinement

0.2 0.4 0.6 0.8 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Total error ξ1 ξ2 −0.5 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 0.5 0.2 0.4 0.6 0.8 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Parameter space mesh ξ1 ξ2

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 17 / 35

Numerical results

Example 2: Convergence of true error

100 101 102 103 104 number of PDE solves 10−2 10−1 100 true error ||Q(u(·, ξ)) − Q(U(·, ξ))||L2

Ω

−1

2

−1

4

||ED||L2

Ω = O(10−3)

uniform h-refinement adaptive h-refinement

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 18 / 35

Numerical results

Example 3: Flow at low Reynolds numbers

Navier-Stokes equations:

−ν∆u + u · ∇u + ∇p = 0 ∇ · u = 0 u = uin, x ∈ Γin u = 0, x ∈ Γw ∪ Γcyl σ · n = 0, x ∈ Γo

Parameterization of uncertainty:

   ν = ξ1 uin,x = ξ2 3 32(4 − y2)

−2 −1 1 2 3 4 5 6 −2 −1 1 2

Γw Γw Γi Γo Γcyl

QoI: Qξ (u) = ux (x0, ξ) Let ξ1 ∼ U(0.01, 0.1), ξ2 ∼ U(1, 3) s.t. Re = ξ2

8ξ1 ∈ [1.25, 37.5]

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 19 / 35

Numerical results

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 20 / 35

Numerical results

Example 3: Flow at low Reynolds numbers

104 105 106 107 Total Degrees of Freedom 10−4 10−3 10−2 10−1

||E||L2(Ξ) ||Q||L2(Ξ)

Ω and Ξ Refinement Adaptive Refinement Ξ Refinement Ω Refinement

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 21 / 35

Efficient Bayesian inference - model selection for RANS

Adaptive Response Surface for Parameter Estimation

Objective The main objective here is to develop a methodology based on response surface models and goal-oriented error estimation for efficient and reliable parameter estimation in turbulence modeling.

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 22 / 35

Efficient Bayesian inference - model selection for RANS

Bayesian inference for RANS using surrogate models

Efficient Bayesian inference: Approximate response surface models can be used to reduce the computational cost of the process.

• Ma and Zabaras, 2009.
• Li and Marzouk, 2014;

Marzouk and Xiu, 2009; Marzouk and Najm, 2009. UQ for RANS models: Uncertainty in the RANS model parameters is a known issue in the turbulence community, but quantifying the effect of this uncertainty is seldom analyzed in the computational fluid dynamics literature.

• Cheung et al., 2011.
• Oliver and Moser, 2011.
• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 23 / 35

Efficient Bayesian inference - model selection for RANS

Fully developed incompressible channel flow

Mean flow equations: u = U + u′

     DUi Dt = −1 ρ ∂P ∂xi + ∂ ∂xj

• ν ∂Ui

∂xj − u′

iu′ j

• ∇ · U = 0

Eddy viscosity assumption:

u′

iu′ j = −νT (Ui,j + Uj,i)

Channel equations: assuming homogeneous turbulence in x

∂ ∂y

• (ν + νT )∂U

∂y

• = 1,

y ∈ (0, H)

1Durbin and Petterson Reif, 2001; Pope, 2000

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 24 / 35

Efficient Bayesian inference - model selection for RANS

Spalart-Allmaras (SA) model

Eddy viscosity is given by:

νT = ˜ νfv1, fv1 = χ3/(χ3 + cv13), χ = ˜ ν/ν

where ˜

ν is governed by the transport equation D˜ ν Dt = P˜

ν(κ, cb1) − ε˜ ν(κ, cb1, σSA, cw2)

+ 1 σSA ∂ ∂xj

• (ν + ˜

ν) ∂˜ ν ∂xj

• + cb2

∂˜ ν ∂xj ∂˜ ν ∂xj

• with
• P˜

ν = production term

• D˜

ν = wall destruction term

Parameter Values

κ

0.41

cb2

0.622

cb1

0.1355

cv1

7.1

σSA

2/3

cw2

0.3

1Allmaras, Johnson, and Spalart, 2012; Oliver and Darmofal, 2009

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 25 / 35

Efficient Bayesian inference - model selection for RANS

Forward model: Find U and ˜

ν such that

       1 = ∂ ∂y

• (ν + νT (cv1))∂U

∂y

• 0 = P˜

ν(κ, cb1) − ε˜ ν(κ, cb1, σSA, cw2) +

1 σSA ∂ ∂y

• (ν + ˜

ν)∂˜ ν ∂y

• + cb2

∂˜ ν ∂y 2 Boundary conditions:

U(0) = 0, ∂yU(H) = 0, ˜ ν(0) = 0, ∂y˜ ν(H) = 0

Weak formulation: Find (U, ˜

ν) ∈ V s.t. B((U, ˜ ν); (V, µ)) = F(V, µ), ∀(V, µ) ∈ V

Quantity of interest and adjoint problem: Find (Z, ζ) ∈ V s.t.

B′((U, ˜ ν); (Z, ζ), (V, µ)) = Q(V, µ) = H

0 V dy

∀(V, µ) ∈ V

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 26 / 35

Efficient Bayesian inference - model selection for RANS

Adaptive Response Surface

• Uniform priors for all parameters with range (0.5, 1.5) times nominal

value (e.g. κ ∼ U(0.205, 0.615))

• Adapted expansion order (after 17 iterations):

κ cb1 σSA cb2 cv1 cw2

6 3 3 1 2 2

100 101 102 103 104 Number of evaluations 10−3 10−2 10−1 100 101 102 ||E||L2(Ξ) adaptive uniform

10 20 30 40 50 60 70 80 Q 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 surrogate full model

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 27 / 35

Efficient Bayesian inference - model selection for RANS

Bayesian inference

Bayes rule:

p(ξ|q) = L(ξ|q) p(ξ) p(q)

where

       q ∈ Rn = Calibration data L(ξ|q) = Likelihood p(ξ) = Prior p(ξ|q) = Posterior

Model selection:

• Set of models M = {M1, M2, . . . , Mn}
• Posterior plausibility = p(Mi|q, M)
• Likelihood =

E(Mi|q, M) := p(q|Mi, M) =

• Ξ

p(q|ξ, Mi, M)p(ξ|Mi, M) dξ p(Mi|q, M) ∝ E(Mi|q, M) p(Mi|M)

1Calvetti and Somersalo, 2007; Jaynes, 2003; Kaipio and Somersalo, 2005

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 28 / 35

Efficient Bayesian inference - model selection for RANS

Calibration data:

• Data is obtained from direct numerical simulation (DNS) 1
• Mean velocity measurements were taken at Reτ = 944 and

Reτ = 2003

Uncertainty models:

• Three multiplicative error models

u+ (z; ξ) = (1 + ǫ(z; ξ))U +(z; ξ)

◮ independent homogeneous covariance ◮ correlated homogeneous covariance ◮ correlated inhomogeneous covariance

• Reynolds stress model
• u′

iu′ j

+ (z; ξ) = T +(z; ξ) − ǫ(z; ξ)

1Del Alamo et al., 2004; Hoyas and Jim´

enez, 2006

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 29 / 35

Efficient Bayesian inference - model selection for RANS

Numerical Results

Independent homogeneous covariance:

u+ (z; ξ) = (1 + ǫ(z; ξ))U +(z; ξ)

• ǫ(z)ǫ(z′)
• = σ2δ(z − z′)

0.85 0.9 0.95 1 1.05 5 10 15 20 25

κ

0.7 0.8 0.9 1 1.1 5 10 15

cv1

0.8 1 1.2 1.4 1.6 1.8 1 2 3 4 5

σ

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 30 / 35

Efficient Bayesian inference - model selection for RANS

Numerical Results

Correlated homogeneous covariance:

u+ (z; ξ) = (1 + ǫ(z; ξ))U +(z; ξ)

• ǫ(z)ǫ(z′)
• = σ2 exp
• −1/2(z − z′)2

l2

• 0.85

0.9 0.95 1 1.05 5 10 15 20

κ

0.7 0.8 0.9 1 1.1 2 4 6 8 10 12

cv1

0.5 1 1.5 2 1 2 3 4

σ

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 31 / 35

Efficient Bayesian inference - model selection for RANS

Numerical Results

Correlated inhomogeneous covariance:

u+ (z; ξ) = (1 + ǫ(z; ξ))U +(z; ξ)

• ǫ(z)ǫ(z′)
• = σ2
• 2l(z)l(z′)

l2(z) + l2(z′) 1/2 exp

• −

(z − z′)2 l2(z) + l2(z′)

• 0.8

1 1.2 1.4 1.6 2 4 6 8

κ

0.8 1 1.2 1.4 1.6 1 2 3 4

cv1

2 4 6 8 0.5 1 1.5

σ

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 32 / 35

Efficient Bayesian inference - model selection for RANS

Numerical Results

Reynolds stress uncertainty:

• u′

iu′ j

+ (z; ξ) = T +(z; ξ) − ǫ(z; ξ)

• ǫ(z)ǫ(z′)
• = kin(z, z′) + kout(z, z′)

where kin models the error near the walls and kout far from the walls.

0.9 1 1.1 1.2 1.3 5 10 15 20 25

κ

0.8 1 1.2 1.4 1.6 2 4 6 8

cv1

2 4 6 8 0.5 1 1.5

σin

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 33 / 35

Efficient Bayesian inference - model selection for RANS

Numerical Results: Model selection

Model evidence (log(E)):

Surrogate Full model Independent homogeneous

• 1.457

8.862 Correlated homogeneous 1.963 8.045 Correlated inhomogeneous 164.9 164.0 Reynolds stress 164.8 169.0

Relative runtimes (in seconds):

Surrogate Full model Independent homogeneous 130 1720 Correlated homogeneous 162 1906 Correlated inhomogeneous 151 1735 Reynolds stress 147 1743 Cumulative 590 7104

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 34 / 35

Conclusions

Concluding remarks and future work

• Error representation for the total error in surrogate models and

contributions from each approximation space.

• Development of adaptive refinement strategies based on error

decomposition.

• Extension to statistical QoI (sQoI), such as probabilities of failure.
• Methodology applied to parameter identification of turbulence models

with some success.

• S. Prudhomme and C. Bryant

Adaptive Surrogate Modeling January 6-9, 2015 35 / 35