The DiceKriging and DiceOptim packages: kriging-based metamodeling - - PowerPoint PPT Presentation

▶
the dicekriging and diceoptim packages kriging based
SMART_READER_LITE
LIVE PREVIEW

The DiceKriging and DiceOptim packages: kriging-based metamodeling - - PowerPoint PPT Presentation

Dec 13, 2023 9 likes •274 views

The DiceKriging and DiceOptim packages: kriging-based metamodeling and optimization for computer experiments UseR! 2009 Conference - Rennes Olivier Roustant, Ecole des Mines de St-Etienne (France) David Ginsbourger, Universit de Neuchtel

slide-1
SLIDE 1

July 9th 2009 1

The DiceKriging and DiceOptim packages: kriging-based metamodeling and

  • ptimization for computer experiments

UseR! 2009 Conference - Rennes

Olivier Roustant, Ecole des Mines de St-Etienne (France) David Ginsbourger, Université de Neuchâtel (Switzerland) Yves Deville, Statistical Consultant (France)

slide-2
SLIDE 2

Scientific framework

  • Analysis of costly numerical simulators

– Crash-test simulators, thermo-hydraulic simulators or neutronic simulators for nuclear safety… – 1 run = several hours !

  • Some issues

– Optimization (ex: minimization of the vehicle weight) – Risk assessment (ex: probability that the temperature exceeds a threshold ?) – Calibration

2

slide-3
SLIDE 3

Scientific framework

  • Some mathematical issues and tools

– To approximate the simulator with a cheaper-to-run proxy

  • > metamodeling: linear models, PolyMars, Splines,

Gaussian processes (kriging), … – To choose design points in a relevant way

  • > computer experiments: space-filling designs, quality

criteria, optimal designs… – To use metamodels to solve problems

  • > metamodel-aided optimization with EGO method

3

slide-4
SLIDE 4

4

Some references

BOOKS

  • Fang K.-T., Li R.and Sudjianto A. (2006), Design and Modeling for Computer Experiments,

Chapman & Hall

  • Rasmussen C.E., Williams C.K.I. (2006), Gaussian Processes for Machine Learning, the

MIT Press, www.GaussianProcess.org/gpml

  • Santner T.J., Williams B.J. and Notz W.I. (2003). The Design and Analysis of Computer
  • Experiments. Springer, 121-161.

ARTICLES

  • Franco J. (2008), Planification d’expériences numériques en phase exploratoire pour la

simulation des phénomènes complexes, PHD thesis.

  • Ginsbourger D. (2009), Multiples Métamodèles pour l’Approximation et l’Optimisation de

Fonctions Numériques Multivariables, PHD thesis.

  • Jones D.R., Schonlau M. and Welch W.J. (1998), Efficient Global Optimization of Expensive

Black-Box Functions, Journal of Global optimization, 13, 455-492.

  • Park J-S, Baek J. (2001), Efficient computation of maximum likelihood estimators in a spatial

linear model with power exponential covariogram, Computer Geosciences, 27, 1-7

slide-5
SLIDE 5

5

Some R packages about computer experiments

  • BACCO [Bayes. Analysis of Comp. Code Output, R. Hankin]

At least: Bayesian modelling – Calibration – Prediction when a proxy (e.g. fast code) is available

  • tgp [bayesian Treed Gaussian Process models, R. Gramacy]

At least: Bayesian modelling – For an irregular output – EGO method

  • mlegp [Max. Lik. Estim. of Gauss. processes, G.M. Dancik]

At least: Univariate & multidimensional outputs – Constant or 1st order polynomial trend – Gaussian covariance - Stochastic simulators – Sensitivity analysis

slide-6
SLIDE 6

6

The DiceKriging package

  • DiceKriging (now split in DiceKriging & DiceOptim)

– Univariate output – Trend is a linear model (including any transformation of inputs) – Max. Lik. Est. of Gaussian Processes with analytical gradients - BFGS and genetic algorithm (with rgenoud) – Deterministic or stochastic simulators – Several choices of covariance functions – EGO method, with analytical gradient (genetic algorithm) – Extension of EGO method for parallel computing – Prediction, validation, conditional simulations – Tested on several case studies (2D, 3D, … 30D)

slide-7
SLIDE 7

Kriging: a stochastic metamodeling method

  • Kriging (Gaussian processes):

Y(x) = F(x)β + Z(x) with

– F(x)β a linear deterministic trend – (Z(x)) a centered stationary Gaussian Process with covariance kernel CZ(x,y)=σ2 R(x-y)

7

slide-8
SLIDE 8

Kriging: a stochastic metamodeling method

8

Some simulations with:

  • a 2nd order poly. trend
  • a Matérn covar. kernel
slide-9
SLIDE 9

Kriging: a stochastic metamodeling method

9

Some conditional simulations with:

  • a 2nd order poly. trend
  • a Matérn covar. kernel
slide-10
SLIDE 10

Kriging: a stochastic metamodeling method

10

More conditional simulations with:

  • a 2nd order poly. trend
  • a Matérn covar. Kernel

In bold:

  • Kriging mean
  • > BLUP interpolator
  • kriging variance
  • > measure of uncertainty
slide-11
SLIDE 11

Kriging: a stochastic metamodeling method

11

Smoothness and choice

  • f covariance kernels
slide-12
SLIDE 12

Kriging: a stochastic metamodeling method

12

Can also be used to deal with stochastic simulators

Below: kriging estimation with noisy observations (constant budget)

slide-13
SLIDE 13

Kriging – What is implemented ?

– Simulation: conditional or non-conditional simulations – Parameter estimation including nugget effect (if wished). By Maximum Likelihood, with analytical gradients.

  • > not a Bayesian point of view
  • > also suited for stochastic simulators

– Prediction: simple & universal kriging formulae (mean, variance) – Validation: leave-one-out, k-fold cross validation (in DiceEval) – Covariance functions: (at now) Gaussian, Power- Exponential, Matern 3/2, 5/2 and Exponential

13

slide-14
SLIDE 14

Trustworthy software ?

  • Some tests we conducted

– Simulate and re-estimate parameters

14

slide-15
SLIDE 15

Trustworthy software ?

  • Some tests we conducted

– Check the simple kriging formulae by simulation

15

slide-16
SLIDE 16

Kriging-aided optimization

16

  • The Expected

Improvement criterion

EI(x) = E( [min(Y(X)) - Y(x)]+|Y(X)=Y)

slide-17
SLIDE 17

Kriging-aided optimization Some illustrations

17

slide-18
SLIDE 18

Kriging-aided optimization Some illustrations

18

slide-19
SLIDE 19

Kriging-aided optimization Some illustrations

19

slide-20
SLIDE 20

Kriging-aided optimization Some illustrations

20

slide-21
SLIDE 21

Kriging-aided optimization Some illustrations

21

slide-22
SLIDE 22

Kriging-aided optimization Some illustrations

22

10 steps of EGO with a Gaussian kernel

slide-23
SLIDE 23

Kriging-aided optimization Some illustrations

23

Parallel EGO:

for i in 1:10 do

  • compute a new point with

EGO step

  • instead of running the

simulator at this point, give the current minimum value The 10 points can be given to 10 different computers

slide-24
SLIDE 24

Kriging-aided optimization Some illustrations

24

EGO.parallel.CL.nsteps At each step

  • Parallel EGO
  • Evaluate the simulator at the

new points (using different computers)

  • Re-estimate the kriging model

Step 1 -> red points Step 2 -> violet points Step 3 -> green points

slide-25
SLIDE 25

Kriging-based optimization: what is implemented ?

25

– EI maximization with genetic algorithm genoud (package rgenoud), and analytical gradient (cst trend) – Sequential EI maximization (EGO method)

  • The simulator runs must be done sequentially

– Multipoints EI maximization (EGO for parallel computing)

  • The simulator runs can be done with ≠ computers
slide-26
SLIDE 26

26

Acknowledgements

  • This work was conducted within the frame of the DICE Consortium between

ARMINES, Renault, EDF, IRSN, ONERA and TOTAL S.A.

  • Gregory Six, Gilles Pujol (Ecole des Mines de Saint-Etienne) for their help in R and

C development.

  • Laurent Carraro (Télécom Saint-Etienne), Delphine Dupuy, Céline Helbert (Ecole

des Mines de Saint-Etienne) for their help in the intial R structure.

  • Anestis Antoniadis (Université Joseph Fourier), Raphaël T. Haftka (University of

Florida), Bertrand Iooss (Commissariat à l’Energie Atomique), André Journel (Stanford University), Rodolphe Le Riche (CNRS), Yann Richet (Institut de Radio- protection et Sûreté Nucléaire) for their relevant advices.