Evolutionary algorithms paper Overview Laurits Tani - - PowerPoint PPT Presentation

▶

Oct 20, 2022 29 likes •281 views

Evolutionary algorithms paper Overview Laurits Tani laurits.tani@gmail.com National Institute of Chemical Physics and Biophysics 15.05.2020 Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 1 / 25 ATLAS Higgs Challenge (using

SLIDE 1

Evolutionary algorithms paper

Overview Laurits Tani laurits.tani@gmail.com

National Institute of Chemical Physics and Biophysics

15.05.2020

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 1 / 25

SLIDE 2

ATLAS Higgs Challenge (using XGBoost) [from before]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 2 / 25

SLIDE 3

Using the default XGBoost parameters

Parameter value num-boost-round 10 learning-rate 0.3 max-depth 6 gamma min-child-weight 1 subsample 1 colsample-bytree 1 Threshold: 0.710 Public score Private score 3.16119 3.11380

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 3 / 25

SLIDE 4

PSO found best hyperparameters

Parameter best-value num-boost-round 153 learning-rate 0.3 max-depth 4 gamma 3.86 min-child-weight 323.6 subsample 0.830 colsample-bytree 1 Threshold: 0.841 Public score Private score 3.54407 3.45578

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 4 / 25

SLIDE 5

GA found best hyperparameters

Parameter best-value num-boost-round 500 learning-rate 0.12859 max-depth 4 gamma 5.0 min-child-weight 257.60083 subsample 1 colsample-bytree 1 Threshold: 0.841 Public score Private score 3.54464 3.44630

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 5 / 25

SLIDE 6

Final comparison

Method Threshold Public score Private score Default 0.710 3.16119 3.11380 PSO 0.841 3.54464 3.44630 GA1 0.841 3.54407 3.45578

1With the old set-up. Now bigger mutations (e.g from O(10) → O(100) less likely) Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 6 / 25

SLIDE 7

Conclusions

Hyperparameters vary alot, result not too much Point was to automatize hyperparameter optimization to save time? Unfortunately not been able to find out the reason for much worse performance of NN and XGB, so this is not included.

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 7 / 25

SLIDE 8

Rosenbrock function

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 8 / 25

SLIDE 9

Brief reminder

Used for testing the performance of optimization algorithms Known minimum at (a, a2) [in our case this would be at (1, 1) since a = 1 and b = 100] To find minimum 106 evaluations done

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 9 / 25

SLIDE 10

Stability [10k particles, 100 iterations]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 10 / 25

SLIDE 11

Stability [10k particles, 100 iterations]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 11 / 25

SLIDE 12

Performance [10k particles, 100 iterations]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 12 / 25

SLIDE 13

Performance [10k particles, 100 iterations]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 13 / 25

SLIDE 14

GA distance performance [10000 particles, 100 iterations]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 14 / 25

SLIDE 15

GA fitness performance [10000 particles, 100 iterations]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 15 / 25

SLIDE 16

GA distance stability [1000 particles, 100 iterations]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 16 / 25

SLIDE 17

GA fitness stability [1000 particles, 100 iterations]

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 17 / 25

SLIDE 18

(modified) Gradient descent

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 18 / 25

SLIDE 19

Gradient descent

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 19 / 25

SLIDE 20

Gradient descent

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 20 / 25

SLIDE 21

Gradient descent

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 21 / 25

SLIDE 22

Gradient descent

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 22 / 25

SLIDE 23

Gradient descent

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 23 / 25

SLIDE 24

Backup slides

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 24 / 25

SLIDE 25

Rosenbrock

Laurits Tani (NICPB) Evolutionary algorithms 15.05.2020 25 / 25