Results of the 2013 IEEE CEC Competition on Niching Methods for - - PowerPoint PPT Presentation

▶
results of the 2013 ieee cec competition on niching
SMART_READER_LITE
LIVE PREVIEW

Results of the 2013 IEEE CEC Competition on Niching Methods for - - PowerPoint PPT Presentation

Aug 17, 2022 153 likes •385 views

Results of the 2013 IEEE CEC Competition on Niching Methods for Multimodal Optimization X. Li 1 , A. Engelbrecht 2 , and M.G. Epitropakis 3 1 School of Computer Science and Information Technology, RMIT University, Australia 2 Department of

slide-1
SLIDE 1

Results of the 2013 IEEE CEC Competition on Niching Methods for Multimodal Optimization

  • X. Li1, A. Engelbrecht2, and M.G. Epitropakis3

1School of Computer Science and Information Technology, RMIT University,

Australia

2Department of Computer Science, University of Pretoria, South Africa 3CHORDS Group, Computing Science and Mathematics, University of Stirling, UK

IEEE Congress on Evolutionary Computation, 20-23 June, Cancun, Mexico, 2013

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 1

slide-2
SLIDE 2

Outline

1

Introduction

2

Participants

3

Results

4

Winners

5

Summary

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 2

slide-3
SLIDE 3

Introduction

Introduction

Numerical optimization is probably one of the most important disciplines in optimization Many real-world problems are “multimodal” by nature, i.e., multiple satisfactory solutions exist Niching methods: promote and maintain formation of multiple stable subpopulations within a single population

Aim: maintain diversity and locate multiple globally optimal solutions.

Challenge: Find an efficient optimization algorithm, which is able to locate multiple global optimal solutions for multimodal problems with various characteristics.

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 3

slide-4
SLIDE 4

Introduction

Competition

Provide a common platform that encourages fair and easy comparisons across different niching algorithms.

  • X. Li, A. Engelbrecht, and M.G. Epitropakis, “Benchmark

Functions for CEC’2013 Special Session and Competition

  • n Niching Methods for Multimodal Function

Optimization”, Technical Report, Evolutionary Computation and Machine Learning Group, RMIT University, Australia, 2013 20 benchmark multimodal functions with different characteristics 5 accuracy levels: ε ∈ {10−1,10−2,10−3,10−4,10−5} The benchmark suite and the performance measures have been implemented in: C/C++, Java, MATLAB

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 4

slide-5
SLIDE 5

Introduction

Benchmark function set

  • X. Li, A. Engelbrecht, and M.G. Epitropakis, “Benchmark Functions for CEC’2013 Special

Session and Competition on Niching Methods for Multimodal Function Optimization”, Technical Report, Evolutionary Computation and Machine Learning Group, RMIT University, Australia, 2013 Id Dim. # GO Name Characteristics F1 1 2 Five-Uneven-Peak Trap Simple, deceptive F2 1 5 Equal Maxima Simple F3 1 1 Uneven Decreasing Maxima Simple F4 2 4 Himmelblau Simple, non-scalable, non-symmetric F5 2 2 Six-Hump Camel Back Simple, not-scalable, non-symmetric F6 2,3 18,81 Shubert Scalable, #optima increase with D, unevenly distributed grouped optima F7 2,3 36,216 Vincent Scalable, #optima increase with D, unevenly distributed optima F8 2 12 Modified Rastrigin Scalable, #optima independent from D, symmetric F9 2 6 Composition Function 1 Scalable, separable, non-symmetric F10 2 8 Composition Function 2 Scalable, separable, non-symmetric F11 2,3,5,10 6 Composition Function 3 Scalable, non-separable, non-symmetric F12 2,3,5,10 8 Composition Function 4 Scalable, non-separable, non-symmetric

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 5

slide-6
SLIDE 6

Introduction

Measures: Peak Ratio (PR) measures the average percentage of all known global optima found over multiple runs: PR = ∑NR

run=1 # of Global Optimai

(# of known Global Optima)∗(# of runs) Who is the winner: The participant with the highest average Peak Ratio performance on all benchmarks wins. In all functions the following holds: the higher the PR value, the better

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 6

slide-7
SLIDE 7

Participants

Participants

Submissions to the competition: E-1682: (PNA-NSGAII) A Parameterless-Niching-Assisted Bi-objective Approach to Multimodal Optimization E-1419: (N-VMO) Variable Mesh Optimization for the 2013 CEC Special Session Niching Methods for Multimodal Optimization E-1449: (dADE/nrand/1,2) A Dynamic Archive Niching Differential Evolution algorithm for Multimodal Optimization Mike Preuss: (NEA1, NEA2) Niching the CMA-ES via Nearest-Better Clustering [2]

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 7

slide-8
SLIDE 8

Participants

Participants (2)

Implemented algorithms for comparisons: (A-NSGAII) A Bi-objective NSGA-II for multimodal

  • ptimization (taken from E-1682)[1]

(CrowdingDE) Crowding Differential Evolution [3] (DECG, DELG, DELS-aj) [4] (DE/nrand/1,2) Niching Differential Evolution algorithms with neighborhood mutation strategies [5] (CMA-ES, IPOP-CMA-ES) CMA-ES/IPOP-CMA-ES with a restart procedure and a dummy archive. [6,7] Mike Preuss: CMA-ES, IPOP-CMA-ES, MG Epitropakis: DE/nrand/1,2, DECG, DELG, DELS-aj, CrowdingDE

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 8

slide-9
SLIDE 9

Results

Results

Summary: 4 submissions/teams from six countries (four continents) 15 algorithms 20 benchmark functions 5 accuracy levels ε ∈ {10−1,10−2,10−3,10−4,10−5} Results: per accuracy level & over all accuracy levels

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 9

slide-10
SLIDE 10

Results

Accuracy level ε = 10−1

Accuracy level 1.0e−1

Benchmark function

5 10 15 20 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I 0.0 0.2 0.4 0.6 0.8 1.0

  • 0.00

0.25 0.50 0.75 1.00 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I

Peak Ratio in all benchmark functions

Algorithms A−NSGAII CMA−ES CrowdingDE dADE/nrand/1 dADE/nrand/2 DECG DELG DELS−aj DE/nrand/1 DE/nrand/2 IPOP−CMA−ES NEA1 NEA2 N−VMO PNA−NSGAII

Accuracy level 1.0e−1

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 10

slide-11
SLIDE 11

Results

Accuracy level ε = 10−2

Accuracy level 1.0e−2

Benchmark function

5 10 15 20 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I 0.0 0.2 0.4 0.6 0.8 1.0

  • 0.00

0.25 0.50 0.75 1.00 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I

Peak Ratio in all benchmark functions

Algorithms A−NSGAII CMA−ES CrowdingDE dADE/nrand/1 dADE/nrand/2 DECG DELG DELS−aj DE/nrand/1 DE/nrand/2 IPOP−CMA−ES NEA1 NEA2 N−VMO PNA−NSGAII

Accuracy level 1.0e−2

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 11

slide-12
SLIDE 12

Results

Accuracy level ε = 10−3

Accuracy level 1.0e−3

Benchmark function

5 10 15 20 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I 0.0 0.2 0.4 0.6 0.8 1.0

  • 0.00

0.25 0.50 0.75 1.00 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I

Peak Ratio in all benchmark functions

Algorithms A−NSGAII CMA−ES CrowdingDE dADE/nrand/1 dADE/nrand/2 DECG DELG DELS−aj DE/nrand/1 DE/nrand/2 IPOP−CMA−ES NEA1 NEA2 N−VMO PNA−NSGAII

Accuracy level 1.0e−3

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 12

slide-13
SLIDE 13

Results

Accuracy level ε = 10−4

Accuracy level 1.0e−4

Benchmark function

5 10 15 20 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I 0.0 0.2 0.4 0.6 0.8 1.0

  • 0.00

0.25 0.50 0.75 1.00 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I

Peak Ratio in all benchmark functions

Algorithms A−NSGAII CMA−ES CrowdingDE dADE/nrand/1 dADE/nrand/2 DECG DELG DELS−aj DE/nrand/1 DE/nrand/2 IPOP−CMA−ES NEA1 NEA2 N−VMO PNA−NSGAII

Accuracy level 1.0e−4

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 13

slide-14
SLIDE 14

Results

Accuracy level ε = 10−5

Accuracy level 1.0e−5

Benchmark function

5 10 15 20 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I 0.0 0.2 0.4 0.6 0.8 1.0

  • 0.00

0.25 0.50 0.75 1.00 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I

Peak Ratio in all benchmark functions

Algorithms A−NSGAII CMA−ES CrowdingDE dADE/nrand/1 dADE/nrand/2 DECG DELG DELS−aj DE/nrand/1 DE/nrand/2 IPOP−CMA−ES NEA1 NEA2 N−VMO PNA−NSGAII

Accuracy level 1.0e−5

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 14

slide-15
SLIDE 15

Results

Overall performance (1)

  • 0.00

0.25 0.50 0.75 1.00 A − N S G A I I C M A − E S C r

  • w

d i n g D E d A D E / n r a n d / 1 d A D E / n r a n d / 2 D E C G D E L G D E L S − a j D E / n r a n d / 1 D E / n r a n d / 2 I P O P − C M A − E S N E A 1 N E A 2 N − V M O P N A − N S G A I I

Peak Ratio in all benchmark functions

Algorithms A−NSGAII CMA−ES CrowdingDE dADE/nrand/1 dADE/nrand/2 DECG DELG DELS−aj DE/nrand/1 DE/nrand/2 IPOP−CMA−ES NEA1 NEA2 N−VMO PNA−NSGAII

All Accuracy levels

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 15

slide-16
SLIDE 16

Results

Overall performance (2)

Algorithm Statistics Friedman’s Test Median Mean St.D. Rank Score A-NSGAII 0.0740 0.3275 0.4044 15 3.1450 CMA-ES 0.7550 0.7137 0.2807 3 10.2300 CrowdingDE 0.6667 0.5731 0.3612 8 7.7900 dADE/nrand/1 0.7488 0.7383 0.3010 2 10.6700 dADE/nrand/2 0.7150 0.6931 0.3174 5 9.6200 DECG 0.6567 0.5516 0.3992 13 6.4950 DELG 0.6667 0.5706 0.3925 11 7.0350 DELS-aj 0.6667 0.5760 0.3857 12 7.0250 DE/nrand/1 0.6396 0.5809 0.3338 9 7.7600 DE/nrand/2 0.6667 0.6082 0.3130 6 8.3200 IPOP-CMA-ES 0.2600 0.3625 0.3117 14 3.8900 NEA1 0.6496 0.6117 0.3280 10 7.6300 NEA2 0.8513 0.7940 0.2332 1 11.9300 N-VMO 0.7140 0.6983 0.3307 4 10.1550 PNA-NSGAII 0.6660 0.6141 0.3421 7 8.3050

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 16

slide-17
SLIDE 17

Winners

Winners

Ranking based on average PR values

1

NEA2 (Mike Preuss) Niching the CMA-ES via Nearest-Better Clustering

2

dADE/nrand/1 (E-1449) A Dynamic Archive Niching Differential Evolution algorithm

3

CMA-ES (Mike Preuss) CMA-ES with simple archive

4

N-VMO (E-1419) Niching Variable Mesh Optimization algorithm Note: The algorithms have not been fine-tuned for the specific benchmark suite!

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 17

slide-18
SLIDE 18

Summary

Conclusions

Summary Four teams from six countries (four continents) Winner: NEA2 (Mike Preuss) Niching the CMA-ES via Nearest-Better Clustering

Competitive on average performance, (nearest-better clustering, archive mechanism, CMA-ES)

Places 2 to 4 very close:

dADE/nrand/1 (E-1449) A Dynamic Archive Niching Differential Evolution algorithm CMA-ES (Mike Preuss) CMA-ES with simple archive N-VMO (E-1419) Niching Variable Mesh Optimization algorithm

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 18

slide-19
SLIDE 19

Summary

Conclusions (2)

The competition gave a boost to the multimodal

  • ptimization community

New competitive and very promising approaches Key characteristics of the algorithms: Many attempts to overcome the influence of the algorithm’s parameters (niching parameters, population size) Usage of Archives to maintain good solutions Multiobjectivization, Clearing, Clustering and neighborhood mutation-based niching techniques Algorithms: Differential Evolution, CMA-ES, Variable Mesh Optimization and NSGAII

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 19

slide-20
SLIDE 20

Summary

Future Work

Possible objectives: Re-organize the competitions in future Enhance the benchmark function set Introduce new performance measures Automate the experimental design and results output Boost multimodal optimization community

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 20

slide-21
SLIDE 21

Summary

Acknowledgment

We really want to thank for their help: The participants :-)

  • Dr. Jerry Swan, University of Stirling, Scotland, UK
  • Dr. Mike Preuss, TU Dortmund, Germany
  • Dr. Daniel Molina Cabrera, University of Cadiz, Spain
  • Dr. Catalin Stoean, University of Craiova, Romania
  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 21

slide-22
SLIDE 22

(-: Thank you very much for your attention :-)

Questions ???

Xiaodong Li: xiaodong.li@rmit.edu.au Andries Engelbrecht: engel@driesie.cs.up.ac.za Michael G. Epitropakis: mge@cs.stir.ac.uk

  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 22

slide-23
SLIDE 23

References

[1 ] K. Deb and A. Saha, "Multimodal optimization using a bi-objective evolutionary algorithm," Evolutionary Computation, 20(1), pp. 27-62, 2012. [2 ] M. Preuss. "Niching the CMA-ES via nearest-better clustering." In Proceedings

  • f the 12th annual conference companion on Genetic and evolutionary

computation (GECCO ’10). ACM, New York, NY, USA, pp. 1711-1718, 2010. [3 ] R. Thomsen, "Multimodal optimization using crowding-based differential evolution," In the IEEE Congress on Evolutionary Computation, 2004. CEC2004, vol.2, pp. 1382-1389, 19-23 June, 2004 [4 ] J. Ronkkonen, Continuous Multimodal Global Optimization with Differential Evolution-Based Methods, Ph.D. thesis, Lappeenranta University of Technology, 2009 [5 ] M. G. Epitropakis, V. P . Plagianakos, and M. N. Vrahatis, "Finding multiple global optima exploiting differential evolution’s niching capability," in 2011 IEEE Symposium on Differential Evolution (SDE), April 2011, pp. 1-8. [6 ] N. Hansen and A. Ostermeier (2001). Completely Derandomized Self-Adaptation in Evolution Strategies. Evolutionary Computation, 9(2), pp. 159-195; [7 ] A. Auger and N. Hansen, "A restart CMA evolution strategy with increasing population size," In the 2005 IEEE Congress on Evolutionary Computation,

  • 2005. vol.2, pp.1769-1776, 2-5 Sept. 2005
  • X. Li, A. Engelbrecht, and M.G. Epitropakis

IEEE CEC 2013 Competition on Niching Methods 23