A Variable-Neighbourhood Search Algorithm for Finding Optimal Run - - PowerPoint PPT Presentation

▶
a variable neighbourhood search algorithm for finding
SMART_READER_LITE
LIVE PREVIEW

A Variable-Neighbourhood Search Algorithm for Finding Optimal Run - - PowerPoint PPT Presentation

Oct 01, 2023 509 likes •978 views

8th Model-Oriented Design and Analysis workshop A Variable-Neighbourhood Search Algorithm for Finding Optimal Run Orders of Experimental Designs in the Presence of Serial Correlation JJ. Garroi, , P. Goos and K. Srensen, , Universiteit

slide-1
SLIDE 1

8th Model-Oriented Design and Analysis workshop

A Variable-Neighbourhood Search Algorithm for Finding Optimal Run Orders of Experimental Designs in the Presence of Serial Correlation

  • JJ. Garroi,

, P. Goos and K. Sörensen, ,

Universiteit Antwerpen Katholieke Universiteit Leuven

slide-2
SLIDE 2

2 2

Table of contents

  • Some solutions given in the literature
  • Features of our algorithm
  • Brief analysis of the optimalisation results
  • Conclusions
  • The importance of the run order
  • Context of the research
slide-3
SLIDE 3

3 3

The source of variation of an industrial experiment is influenced by environmental factors, which are not subject to the experimentation.

Context of the research

Industries perform

  • classical designs function of their prior belief
  • one run at a time,
  • without restriction on the order of the run.

As pointed out by Constantine (1989), it is likely that the successive observations are correlated and that this influences the design strategy of the experiment.

My research elaborates on how to find optimal run order in such situations.

slide-4
SLIDE 4

4 4

Assumptions for this talk

. ) , ( ~

2 3 1 3 1 2 1 3 1

V X X X X Y

with

i i ii i j j i ij i i i i

ε ε β β β β

+ + + + =

∑ ∑ ∑ ∑

= + = = =

An AR(1) model is assumed to fit the real correlation structure.

( ) ( ) .

ˆ . ˆ

1 1 ) ( 1 1 1

X V X Var Y V X X V X

T T T

GLS GLS

− − = − − − =

β β

. > ρ with

Only central composite designs will be examined. We assume that there is a positive dependence between the observations.

⎟ ⎟ ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛ − − + − − + − − =

−

1 1 1 1

2 2 1

ρ ρ ρ ρ ρ ρ ρ ρ

V

slide-5
SLIDE 5

5 5

D crit = D crit =

In the presence of serial correlation , run orders play a role

1 2 3 4 5 1 2 3 4 5 6 7 9 6 7 8 9 10 12 11 13 14 15 10 12 11 13 14 15 How to organize the measurements

The importance of run orders

8 A crit = V crit = V crit = A crit = to have precise estimations ? to have accurate predictions ?

127.3957 120.528 5.813 6.146 4.847 4.785

slide-6
SLIDE 6

6 6

Some solutions given in the literature

When the data are correlated,

1.

Constantine (1989) sought efficient run orders for factorial designs and showed that designs with maximal level changes possess optimal properties with respect to the D-optimality.

2.

Cheng & Steinberg (1991) outlined a reverse foldover algorithm to produce a design with maximal level changes.

3.

A simulated annealing was suggested to find D-optimal design for factorial

  • experiments. See Elliot, Eccleston & Martin (1998) and Zhu (2003).

4.

An exchange-type algorithm was proposed to construct D-optimum design. See Brimkulov (1980), Ucinski & Atkinson (2004), Stehlik (2006).

slide-7
SLIDE 7

7 7

We develop a new heuristic to solve this problem.

  • It has to produce good solutions,
  • It has to gain computational performance,

There is a drawback : the solution proposed may be not the (global) optimal one.

Definition : A neighbourhood of the solution x is the set of all solutions reached from x with a specific move. Criticism :

  • Poor results of the foldover algorithm if interaction terms have to be estimated,
  • Exchanges are not the only action to do to maximize the D-criterion,
  • No attention has been paid to designs involving a large number of factors.

We solve these problems by implementing a Variable-Neighbourhood algorithm.

Some solutions given in the literature

slide-8
SLIDE 8

8 8

Variable Neighbourhood Search Algorithm

It is a heuristic developed by Mladenovic and Hansen (1997), It explores a multi-neighbourhood structure, The neighbourhoods are sortered by size, It systematically changes of neighbourhoods during the search, It is composed by a local search and a diversification procedure : The local search will intensify the search and converge to a local

  • ptimum.

A perturbation is used as a diversification factor to escape from a

local optimum.

slide-9
SLIDE 9

9 9

Pseudo-code

Notations : Nk , k=1,...,6 the set of neighbourhood structures Nk(x) the set of solutions in the kth neighbourhood of x Let x a starting solution Repeat Set k <-1 While k<=kmax , do : Find the best solution x’ Є Nk(x) If x’ better than x then : Set x<-x’ Set k<-1 Else Set k<-k+1 Perturb x until the stopping condition is met.

slide-10
SLIDE 10

10 10

Description of the neighbourhoods’structure

1°) Shift left 12345 23451 2°) Swap adjacent 12345 21345 3°) Relabel 12435 23541 4°) Swap 12345 42315 5°) Move 12345 23145 6°) 2-Opt 12345 43215

Linear neighbourhoods Quadratic neighbourhoods

slide-11
SLIDE 11

11 11

Ordering the neighbourhoods

To determine the neighbourhoods’ order and the strategy to adopt, a study

  • f the efficiency of each neighbourhood was carried out.

The performance of the neighbourhoods decreases with the problem size.

200 100 80 60 40 20 62,84 63,78 63,96 69,43 82,99 99,78

2-Opt

63,05 66 66,83 77,63 97,18 99,51

Move

63,05 69,15 74,05 98,34 99,45 98,49

Swap

63,51 64,27 64,11 67,91 72,45 76,58

Shift left

61,15 62,27 63,41 67,15 70,54 75,38

Relabeling

75,79 90,02 89,3 93,39 92,33 93,2

Swap adjacent Central Composite Designs D-efficiency (%) in 10 seconds CPU time

slide-12
SLIDE 12

12 12

A good order of the neighbourhoods is : 1. Swap adjacent 2. Shift left 3. Relabeling 4. Swap 5. Move 6. 2-opt Linear Neighbourhoods Quadratic Neighbourhoods

Ordering the neighbourhoods

slide-13
SLIDE 13

13 13

Brief analysis of the optimalisation results

Time required to outperform the randomization Starting from the same 100 random start solutions, the VNS algorithm is applied during 60 sec. We generate random samples of 1000 run orders and pick the best run order. This selection lasts 60 sec and is executed 100 times. The time needed to the VNS to

  • utperform the randomization

method is showed in the graph. Applying randomization is consequently not appropriate.

slide-14
SLIDE 14

14 14

Brief analysis of the optimalisation results

Loss in D-efficiency (%) if randomization Max 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 2 Factors 0.70 1.29 1.87 2.58 3.44 3.96 4.36 4.84 5.37 3 Factors 1.07 2.15 3.23 4.25 5.33 6.02 6.64 7.12 7.48 4 Factors 1.45 3.01 4.58 5.93 7.22 8.08 8.92 9.41 9.59 5 Factors 1.05 2.28 3.50 4.49 5.69 6.44 7.15 7.79 8.29 6 Factors 1.00 1.95 2.90 3.71 5.17 5.47 5.56 6.14 6.35 Min 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 2 Factors 0.01 0.03 0.09 0.14 0.18 0.22 0.28 0.28 0.29 3 Factors 0.39 0.84 1.38 1.82 2.23 2.79 2.95 3.19 3.27 4 Factors 0.78 1.68 2.77 3.64 4.45 5.59 5.90 6.39 6.55 5 Factors 0.49 1.09 1.42 2.35 3.31 3.74 4.50 4.37 4.74 6 Factors 0.10 0.10 0.31 0.41 0.81 0.75 0.88 0.46 1.50 Applying randomization is consequently not appropriate.

slide-15
SLIDE 15

15 15

Brief analysis of the optimalisation results

In the following, the results for a rotatable central composite design involving three factors and three center points will be presented.

  • 1. The features of D-optimal run orders are outlined.
  • 2. The structure of D-optimal run orders are showed for GLS estimations.
  • 3. The structure of D-optimal run orders are showed for OLS estimations
slide-16
SLIDE 16

16 16

Brief analysis of the results

There are many D-optimal run orders for each specification of the correlation parameter. The D-optimal run order are Bayesian D-optimal indeed, i.e. they are nearly D-optimal for all specification of the correlation parameter. There are many zero in the information matrix for each specification. The optimal run order has a specific structure.

2 0 0 0 0 0 0 1 1 1 23 -4 3 -3 0 -6 0 0 2 23 -1 0 0 0 3 3 -3 26 4 -1 -2 1 -2 -1 15 0 0 0 0 2 15 0 0 0 2 15 0 0 -2 27 -8 -10 27 -4 23 14 0 0 0 0 0 0 11 11 11 14 -1 0 0 0 -1 0 0 0 14 0 0 0 0 0 0 0 15 1 0 0 0 0 0 8 0 0 0 0 0 8 0 0 0 0 8 0 0 0 23 5 5 23 6 22

IGLS = IGLS =

When ρ = 0.7 When ρ = 0.1

slide-17
SLIDE 17

17 17

A) Optimal run order for GLS estimations

Brief analysis of the results

2 4 3 1 6 5 7

They begin and end with a centerpoint. The axial points always appear together in the sequence and

  • ne change each

time of axis. There is a symmetry in the sequence with respect to the center point.

slide-18
SLIDE 18

18 18

8 9

12

11 10 13 14 15

17

16

The factorial points appear in small dimensions in two groups ABC=1 & ABC=-1 . A centerpoint separates the groups. A centerpoint forms the end of the sequence.

Brief analysis of the results

B) Optimal run order for GLS estimations

slide-19
SLIDE 19

19 19

15 13

5

16 14 4 1 3

12

2

The factorial points appear in small dimensions in two groups ABC=1 & ABC=-1 . A centerpoint forms the end of the sequence.

Brief analysis of the results

C) Optimal run order for OLS estimations

6 11 7 8 9 10

17

The axial points always appear together in the sequence and one change each time of axis. A centerpoint separates the groups. For large values of ρ, OLS is not a good idea (6% in loss of D-efficiency ) D-optimal run orders are not robust to the estimation method

slide-20
SLIDE 20

20 20

  • In the presence of serial correlation, a variable neighbourhood algorithm

has been presented to optimize the D-criterion.

Conclusions

  • All neighbourhoods are essential and efficient for this optimality criterion.
  • The method is fast and outperforms the randomization procedure.
  • The VNS algorithm outperforms the exchange algorithms.
  • The D-optimal run order is well structured and sensible.
  • The VNS outperforms the simulated annealing algorithm proposed by

Zhou (2003).

But, if more than six factors are considered, the method does not produce in the required time a good run order.

  • Our results are identical to those presented by Elliot & all (1998) and

by Cheng & Steinberg (1991) in the situation they investigated.

slide-21
SLIDE 21

21 21

Moving windows : a motivation Moving windows : a motivation

  • Evaluation the objective function is time consuming.
  • Neighbourhood sizes are polynomial functions of the length of

the permutation.

Our method to go faster through the solution space :

  • Explore the entire sequence in pieces of the same length using

moving window.

  • Apply the previous VNS method successively to each of the windows.

Analysis of the Problem Much more time is used by the evaluation in larger problems.

slide-22
SLIDE 22

22 22

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-23
SLIDE 23

23 23

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-24
SLIDE 24

24 24

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-25
SLIDE 25

25 25

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-26
SLIDE 26

26 26

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-27
SLIDE 27

27 27

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-28
SLIDE 28

28 28

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-29
SLIDE 29

29 29

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-30
SLIDE 30

30 30

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-31
SLIDE 31

31 31

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-32
SLIDE 32

32 32

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-33
SLIDE 33

33 33

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-34
SLIDE 34

34 34

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-35
SLIDE 35

35 35

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-36
SLIDE 36

36 36

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-37
SLIDE 37

37 37

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-38
SLIDE 38

38 38

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-39
SLIDE 39

39 39

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-40
SLIDE 40

40 40

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When no more improvement in a window, apply the VNS algorithm to the adjacent one.

Moving windows : Principles Moving windows : Principles

slide-41
SLIDE 41

41 41

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 When all windows of a certain size have been tried without success,

Moving windows : Principles Moving windows : Principles

slide-42
SLIDE 42

42 42

Moving windows : Principles Moving windows : Principles

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 3 2 1 Finally, the algorithm is applied on the whole sequence

slide-43
SLIDE 43

43 43

Moving windows : results Moving windows : results

Good efficiency for all neighbourhoods !!! Slightly better results in small dimensions

60 40 20 60 40 20 94,34 98,64 99,63 69,43 82,99 99,78

2-Opt

96 99,84 99,84 77,63 97,18 99,51

Move

98,61 99,84 99,84 98,34 99,45 98,49

Swap

92,13 92,63 93,52 67,91 72,45 76,58

Shift left

90,76 91,58 93,58 67,15 70,54 75,38

Relabeling

92,14 94,1 95,46 93,39 92,33 93,2

Swap adjacent with moving window without moving window Central Composite Designs D-efficiency (%) in 10 seconds CPU time

slide-44
SLIDE 44

44 44

Larger improvements are obtained in large dimensions The neighbourhood order should be maintained

200 100 80 200 100 80 95,56 92,48 91,38 62,84 63,78 63,96

2-Opt

83,37 91,06 91,63 63,05 66 66,83

Move

86,04 92,78 93,88 63,05 69,15 74,05

swap

90,66 90,29 89,73 63,51 64,27 64,11

Shift left

87,77 88,94 88,22 61,15 62.27 63,41

Relabeling

90,1 90,45 89,63 75,79 90,02 89,3

Swap adjacent with moving window without moving window Central Composite Designs D-efficiency (%) in 10 seconds CPU time

Moving windows : results Moving windows : results

slide-45
SLIDE 45

45 45

Research Perspectives

It may be interesting

  • to find update formulas for the D-criterion,
  • to examine blocked experiments,
  • to look at other optimal criteria,
  • to examine other statistical problems that require a

specific run order. (Ex: time trends)

slide-46
SLIDE 46

8th Model-Oriented Design and Analysis workshop

A Variable-Neighbourhood Search Algorithm for Finding Optimal Run Orders of Experimental Designs in the Presence of Serial Correlation

  • JJ. Garroi,

, P. Goos and K. Sörensen, ,

Universiteit Antwerpen Katholieke Universiteit Leuven