Simulator Analysis and Validation Engine 2 Fei Liu Duke University - - PowerPoint PPT Presentation

Simulator Analysis and Validation Engine 2

Fei Liu

Duke University Summer School on the Design and Analysis of Computer Experiments Simon Fraser University August 11-16, 2006

Outline

Objective The SAVE2 methodology The SAVE2 software package The Example Resources

Outline

Objective The SAVE2 methodology The SAVE2 software package The Example Resources

Objective

◮ Computer models may have functional outputs:

yM(x, u, t)

◮ Inputs to the computer models include:

◮ x: characteristics associated with the field experiments. ◮ u: calibration parameters associated with the computer

model.

◮ t: time when the measurements are taken. And the

computer model outputs are functions of t.

Collected data consist of:

◮ Field: Time-histories of field experiments, denoted by

yF(x, t).

◮ Computer model: Time histories obtained by running the

computer model at different design points, denoted by yM(x, u, t).

10 20 30 40 50 60 5 10 d y

Figure: A example data: field experiments (red); computer model runs (black).

Questions

SAVE-2 provides answers to questions such as,

◮ Does the computer model adaquetely represent the reality

in terms of validation perspectives / intended uses, e.g. peaks / valleys?

◮ How can we provide optimal prediction for the reality? ◮ How can we learn the associated (unknown) characteristic

parameters for the field experiments?

◮ How can we predict for a new system?

Outline

Objective The SAVE2 methodology The SAVE2 software package The Example Resources

Wavelet representation

Represent the history data as: yM(zj; t) =

W

• i=1

wM

i (zj)ψi(t),

j = 1, . . . , m; yF

r (x∗; t) = W

• i=1

wF

ir (x∗)ψi(t),

j = 1, . . . , f. Inputs: x = (x1, . . . , xp1): key (unknown) characteristics. u = (u1, . . . , up2): calibration parameters for the computer models. z = (x, u).

GaSP approximation

For each i (index of wavelet coefficient), model the coefficient as, wM

i (·) ∼ GP

• µi, 1

λM

i

Corri(·, ·)

• ◮ Inputs to GaSP: Design matrix, and wM

i

from model runs.

◮ Output from GaSP:

• α(i), β(i), µi, λM

i

• , i = 1, . . . , W.

◮ Response Surface:

• wM

i (z) | wM i

• ∼ N
• ˆ

µi(z), ˆ σ2

i (z)

• .

Bayesian analysis

◮ Simulator Analysis and Validation Engine for each

coefficient: wR

i (x∗) = wM i (x∗, u∗) + bi(x∗),

i = 1, . . . , W wF

ir (x∗) = wR i (x∗) + ǫir,

r = 1, . . . , f

◮ bi(x∗) ∼ N

• 0, τ 2

j

• , ǫir ∼ N
• 0, σ2

i

• . j is the level for index i.

◮ Sufficient statistics: ¯

wF

i , S2 i = f r=1(wF ir − ¯

wF

i )2. ◮ Data:

D = ( ¯ wF

i , S2 i , ˆ

µi(·), ˆ σ2

i (·)),

i = 1, . . . , W

Outline

Objective The SAVE2 methodology The SAVE2 software package The Example Resources

Steps of Analysis

• 1. Input data;
• 2. Data registration and wavelet representation;
• 3. Gaussian stochastic response surface approximations;
• 4. Bayesian analysis
• x∗, u∗, {bi}, {τ 2

j }, {σ2 i }, {wR i (x∗)}

• ;
• 5. Wavelet reconstruction;
• 6. Output and plots;

SAVE-2 software

Data Registration Computer model Field runs Wavelet Decomposition GaSP Posterior (MCMC) Prior Information Inferences

Wavelet

Reconstruction Posterior reports Posterior Curves histograms

R Language

C Language

Use of SAVE2

◮ To install save2,

[chuck:save2]$ make Installation

◮ To load data,

[chuck:save2]$ make LoadData

◮ To remove the failed model runs,

[chuck:save2]$ make PreprocessData

Use of SAVE2

◮ To apply the data registration procesure and wavelet

representation, [chuck:save2]$ make RegisWave

◮ To apply the GaSP approximation to the coefficients of the

model runs, [chuck:save2]$ make GASP

◮ To run the Bayesian analysis,

[chuck:save2]$ make MCMC-LoadData [chuck:save2]$ make MCMC-Run

Use of SAVE2

◮ To summarize the posterior samples, reconstruct functions,

and create an R object to store the result, [chuck:save2]$ make Reconstruction-all

◮ To visualize the results, and get confidence intervals for the

peaks / valleys, [chuck:save2]$ make PLOT-all

◮ To remove old analysis for new data,

[chuck:save2]$ make removeOldAnalysis

Input files

Three types of input files are required by the software:

◮ The design matrix for the computer model runs. ◮ The time histories for the computer model runs. ◮ The time histories for the field experiments.

Example Inputs

◮ The design matrix:

u01 u02 x01 x02 x03 0.625000 0.546875 0.406250 0.593750 0.406250 1.000000 0.343750 0.718750 0.921875 0.203125 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

◮ The history data:

Time CH 1 CH 2 CH 3 Velocity .000000 3.83 0.29 0.092 25.049 .002441 3.84 0.13

• 0.099

25.048 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Data registration

Peaks / Valleys of field and model-run should occur at the same “location”. Steps:

• 1. Convert time-histories into distance-histories.
• 2. Reference curve: Average of the model curves.
• 3. Align curves to match the peaks of both major peaks.

7 8 9 10 5 10 Time Load 37 38 39 40 41 5 10 Time Load 7 8 9 10 5 10 Time Load 37 38 39 40 41 5 10 Time Load

Output Files

Output Files consist of,

◮ Posterior distribution of x∗, u∗. ◮ Posterior distribution of the bias function. ◮ Posterior distribution of the reality. ◮ Reality for a new field run. ◮ Extrapolate to new system.

Outline

Objective The SAVE2 methodology The SAVE2 software package The Example Resources

The Data

◮ The time histories are collected at 90843 time points. ◮ The computer model has two calibration parameters

(u1, u2), and 7 characteristics are measured for the field experiments, (x1, x2, . . . , x7).

◮ The computer model has been exercised at 60 design

points.

◮ The field data consists of 7 replicates associated with the

same characteristic parameters.

π(x∗, u∗)

Parameter Type Variable Type Uncertainty Damping1 Calibration Calibration 15% Damping2 Calibration Calibration 15% x1 Manufacturing Uncertain 10% x2 Manufacturing Uncertain 10% x3 Manufacturing Uncertain 7% x4 Manufacturing Uncertain 8% x5 Manufacturing Uncertain 5% x6 Manufacturing Uncertain 12% x7 Manufacturing Uncertain 8%

Table: I/U Map. Uncertainty ranges for calibration (first two) and manufacturing (last seven) parameters.

The Input/Uncertainty Map

u− 1

u− 1 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0

u− 2

u− 2 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0

x− 1

x− 1 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0

x− 2

x− 2 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 2.0 2.5 3.0

x− 3

x− 3 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 1 2 3 4

x− 4

x− 4 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 1 2 3 4

x− 5

x− 5 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 5 10 15 20

x− 6

x− 6 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 2 4 6 8

x− 7

x− 7 Freq. 0.0 0.2 0.4 0.6 0.8 1.0 1 2 3 4

The prior distributions

π(u1) = π(u2) = Uniform on [0.125, 0.875] π(x1) = π(x2) ∼ N(0, 0.11112) truncated to the interval[0.1667, 0.8333] π(x3) ∼ N(0, 0.097232) truncated to the interval [0.2083, 0.7917] π(x4) = π(x7) ∼ N(0, 0.10262) truncated to the interval[0.1923, 0.8077] π(x5) ∼ N(0, 0.49032) truncated to the interval [0.3529, 0.6471] π(x6) ∼ N(0, 0.11762) truncated to the interval [0.1471, 0.8529]

Posterior distribution of the bias function

7 8 9 10 −2 −1 1 2 Time Load

Bias function MCMC (90% Tolerance bounds)

Posterior distribution of the reality

(yR)h(x∗, u∗, ·) =

W

• i=1

((wM

i (x∗, u∗))h + bh i )ψi(·), h = 1, . . . , N

7 8 9 10 5 10 Time Load

Bias Corrected Prediction Field Data Pure Model Prediction 90% Tolerance bands from Reality

Reality for a new field run

(yR)h(x, u∗, ·) =

W

• i=1

((wM

i (x, u∗))h+bh i )ψi(·), h = 1, . . . , N; x ∼ π(x)

7 8 9 10 5 10 Time Load

Bias Corrected Prediction Field Data Pure Model Prediction 90% Tolerance bands from Field

Extrapolate to new system

(yRB)h(x, u∗, ·) =

W

• i=1
• (wMB

i

(x, u∗))h · (bi)h ψi(·), x ∼ π(x)

7 8 9 10 5 10 15 20 25 30 Time Load

Field Data Bias Corrected Prediction 90% Tolerance bounds

Outline

Objective The SAVE2 methodology The SAVE2 software package The Example Resources

Resources

◮ System requirements:

◮ The R statistical software package (version 1.8.1 or higher). ◮ The wavethresh package (version 2.2-8 or higher) for R. ◮ The GaSP software. ◮ The ATLAS library. ◮ The GNU Scientific library.

◮ Software package and Manual may be available at NISS

webpage ...

Thank you!