Towards an Equivalence Theorem for Computer Simulation Experiments? - - PowerPoint PPT Presentation

ENBIS EMSE 2009 - St.Etienne 1 2.7.2009

Towards an Equivalence Theorem for Computer Simulation Experiments?

Werner G. Müller Department of Applied Statistics (IFAS) Based on joint work with M.Stehlík and Luc Pronzato (started a week ago)

ENBIS EMSE 2009 - St.Etienne 2 2.7.2009

I take on challenge #1 from David’s list: „create designs that are tied to

• ur methods of analysis“
• D. Steinberg (2009), ENBIS-EMSE workshop

ENBIS EMSE 2009 - St.Etienne 3 2.7.2009

The Setup

Random field: with

Two purposes: prediction or estimation

Universal Kriging: using the EBLUP and the corresponding GLS-estimator. Alternative: Full ML or REML of (β,θ) and insert above.

( )

( )

( )

( ) , y z x z z η β ε = +

[ ]

( ) ( ') ( , '; ) ( , ). E z z c z z c d ε ε θ θ = =

ENBIS EMSE 2009 - St.Etienne 4 2.7.2009

Optimal Designs (for estimation)

Classical: select the inputs (and weights) such that a prespecified criterion is optimized. Well developed theory for standard (uncorrelated) regression based on Kiefer’s (1959) concept of design measures.

1 2 1 2

, , , , , ,

n N n

p p p z z z ξ   =     ⋯ ⋯

( )

,

max

i i

N z p

M ξ   Φ  

ENBIS EMSE 2009 - St.Etienne 5 2.7.2009

Three Practical Cases:

Case 1: We are interested only in the trend parameters β and

consider θ as known or a nuisance. Case 2: We are interested only in the covariance parameters θ (sometimes we set β =0). Case 3: We are interested in both sets of parameters equally.

ENBIS EMSE 2009 - St.Etienne 6 2.7.2009

D-optimal designs for estimating trend and covariance parameters

For the full parameter set the information matrix is Use the (weighted) product of the respective determinants as an

• ptimum design criterion (Müller and Stehlík, 2009):

Xia G., Miranda M.L. and Gelfand A.E. (2006) suggest to use the trace. ( ) ( ) ( ) ( ) ( ) ( )

T T T T

lnL lnL M E M lnL lnL

β θ

β θ β θ ξ θ β β β β θ ξ θ β θ β θ θ β θ θ ∂ , ∂ ,   − −   ; , ∂ ∂ ∂ ∂     = .     ; ∂ , ∂ ,     − −   ∂ ∂ ∂ ∂  

1

'[ ] ( ) ( ) M M M M

α α β θ β θ

ξ ξ

−

Φ , =| | ⋅| |

ENBIS EMSE 2009 - St.Etienne 7 2.7.2009

Compound Designs

Single purpose criterion is inefficient, thus construct weighted averages were introduced by Läuter (1976), related to constrained designs: (cf. Cook and Wong, 1994); sometimes standardized (Mcgree et al., 2008):

[ ] [ ( )] (1 ) [ ( )]. M M ξ α α ξ α ξ ′ ′ Φ | = Φ + − Φ

arg max [ ( )] s t [ ( )] ( ) M M

ξ

ξ ξ ξ κ α

∗ ∈Ξ

′ ′ = Φ . . Φ > ,

[ ] [ ( )]/ [ ( *)] (1 ) [ ( )]/ [ ( *)]. M M M M ξ α α ξ ξ α ξ ξ ′ ′ ′ ′ Φ | = Φ Φ + − Φ Φ

ENBIS EMSE 2009 - St.Etienne 8 2.7.2009

7 (9) Issues (surveyed in Müller & Stehlík, 2009)

• 1. Nonconvexity
• 2. Asymptotic unidentifiability (Mθ)
• 3. Nonreplicability
• 4. Non-additivity
• 5. Smit’s paradox
• 6. Näther’s paradox
• 7. Impact of dependence on information (M&S paradox)
• 8. Choice of dependence structure
• 9. Singular designs (the role of the nugget effect)

ENBIS EMSE 2009 - St.Etienne 9 2.7.2009 Information from D-optimal design:

when θ is estimated or not estimated respectively. (Müller & Stehlík, 2004)

The Impact of Dependence

ENBIS EMSE 2009 - St.Etienne 10 2.7.2009

Design for prediction (EK-optimality)

Criterion often based on kriging variance, e.g. Additional uncertainty from estimation of θ is taken into account by Zhu (2002) and Zimmerman (2006): Abt (1999) and Zhu and Stein (2007) supplement this by

2

ˆ min max [( ( ) ( )) ]

z

E y z y z

ξ

ξ | −

{ }

{ }

1

ˆ ˆ min max Var[ ( )] tr ' Var[ ( ) ]

z

y z M y z

θ ξ

θ

−

+ ∂ /∂

' 1

ˆ ˆ Var[ ( )] Var[ ( )]

T

y z y z Mθ θ θ

−

∂ ∂    .     ∂ ∂    

ENBIS EMSE 2009 - St.Etienne 11 2.7.2009

Recall the Kiefer-Wolfowitz Equivalence Theorem (1960)

(Case 1 with uncorrelated errors)

D-criterion: and G-criterion: yield same (approximate) optimal designs.

( )

max M β

ξ

ξ

ˆ min max Var[ ( ) | ]

z

y z

ξ

ξ

ENBIS EMSE 2009 - St.Etienne 12 2.7.2009

Conjecture:

One can always find an α such that the compound design based upon is (in some to be defined sense) close to designs following from Zhu’s EK-(empirical kriging)-optimality.

'[ ] M M

β θ

Φ ,

ENBIS EMSE 2009 - St.Etienne 13 2.7.2009

Example : Ornstein-Uhlenbeck process

2

cov( ´)

d

z z e

θ

σ

−

, =

constant trend η(.) = β

Case 1 (Kiselak and Stehlík, 2007, Dette et al., 2007): uniform (space-filling) design is D-optimal! Case 2 (Müller and Stehlik, 2009, Zagoraiou and Baldi-Antognini, 2009): D-optimal design collapses! Case 3: regulatory version.

0.2 0.4 0.6 0.8 1.0 0.814 0.816 0.818 0.820 0.822 0.824

ENBIS EMSE 2009 - St.Etienne 14 2.7.2009

Exchange algorithms (Fedorov/Wynn, 1972)

consist in a simple exchange of points from the two sets

s

Sξ and

s

s

S X

ξ

∖

at every iteration, namely where

arg max ( ) arg min ( )

s s s

s s s s x S x S X

x x x x

ξ ξ

φ ξ φ ξ

+ − ∈ ∈

= , = , .

∖

and

Version: Cook and Nachtsheim, 1980 Survey: Royle, 2002

1

1 1

s s s s s s

x x n n ξ ξ

                              

− + + =

, ∪ , , ∖

ENBIS EMSE 2009 - St.Etienne 15 2.7.2009

Suggested Variant: Hybrid with Simulated Annealing

• Make the best exchange between a point from sets

s

Sξ and a randomly chosen point from

s

s

S X

ξ

∖ at every iteration

• If there is no improvement, give more weight to points

farer from the selected and draw anew.

• Perhaps use a stochastic acceptance operator

(decreasing temperature) to improve performance.

ENBIS EMSE 2009 - St.Etienne 16 2.7.2009

Case 3 for θ =1 and varying α=0,0.7,1

0.5 1

ENBIS EMSE 2009 - St.Etienne 17 2.7.2009

Case 3 for α=0.9 and varying θ=0.1,1,10

0.5 1

ENBIS EMSE 2009 - St.Etienne 18 2.7.2009

Efficiency Comparison

ENBIS EMSE 2009 - St.Etienne 19 2.7.2009

References (www.ifas.jku.at)

Müller, W.G., "Collecting Spatial Data. ", 3rd revised and extended edition, Springer Verlag, Heidelberg, 2007. Müller, W.G. and Stehlík, M., “An Example of D-optimal Designs in the Case of Correlated Errors”, in J. Antoch (Ed.), COMPSTAT2004 Proceedings in Computational Statistics, Springer, 1519-1526, 2004. Müller, W.G. and Stehlík, M., “D-Optimal Spatial Designs for Estimating Trend and Covariance Parameters”, in Statistics for Spatio-Temporal Modelling, Proceedings of the 4th International Workshop on Statistics for Spatio-Temporal Modelling (METMA 5), Mateu, Porcu, Zocchi (Eds.), 69-77, 2008. Müller, W.G. and Stehlík, M., “Issues in the Optimal Design of Computer Simulation Experiments” in Applied Stochastic Models in Business and Industry, 25(2), 153- 177, 2009. Müller, W.G. and Stehlík, M., “Compound Optimal Spatial Design”, accepted for Environmetrics. Stehlík, M. and Müller, W.G., “Fisher Information in the Design of Computer Simulation Experiments” in Journal of Physics: Conference Series, 2008.

ENBIS EMSE 2009 - St.Etienne 20 2.7.2009

ISBIS is a new international society and one of the newest Sections of the International Statistical Institute (ISI) founded in April 2005. http://www.isbis.org/ Soliciting proposals for invited sessions at ISI Dublin 2011 !

ENBIS EMSE 2009 - St.Etienne 21 2.7.2009

ENBIS EMSE 2009 - St.Etienne 22 2.7.2009

Problem #1: Non-additivity of the Information Matrix

Leads to unseparability of information contributions through design measures! Remedy: e.g. interpretation of design measures as amount of noise suppression (Pázman & M., 1998, M.+P., 2003)

( ) ( ) ( )

1 , ' '

1 ( ) '

T N N z z z z

M X z C X z N ξ ξ

−

  =  

∑∑

ENBIS EMSE 2009 - St.Etienne 23 2.7.2009

Problem #2: Use of Fisher Information Matrix

If covariance parameters θ are included in the estimation (cases 2 & 3), the FI matrix contains a block Then its interpretation as being inversely proportional to asymptotic covariance matrix of parameters fails (Abt & Welch, 1998). Remedy: small normal error theory by Pázman (2007).

1 1

1 ( ) ( ) ´( )} ( ) ( ) 2

ij i j

C C M tr C C ξ θ ξ θ ξ θ ξ θ ξ θ θ θ

− −

  ∂ , ∂ ,   , = , ,   ∂ ∂    