Seminar: Black-Box Challenge Michael Gla Faramarz Khosravi Moritz - - PowerPoint PPT Presentation

▶
seminar black box challenge
SMART_READER_LITE
LIVE PREVIEW

Seminar: Black-Box Challenge Michael Gla Faramarz Khosravi Moritz - - PowerPoint PPT Presentation

Feb 23, 2023 97 likes •409 views

Seminar: Black-Box Challenge Michael Gla Faramarz Khosravi Moritz Mhlenthaler Tobias Schwarzer Friedrich-Alexander-Universitt Erlangen-Nrnberg (FAU), Germany 26.10.2015 Outline Black-Box Optimization Algorithms Particle Swarm

slide-1
SLIDE 1

Seminar: Black-Box Challenge

Michael Glaß Faramarz Khosravi Moritz Mühlenthaler Tobias Schwarzer Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Germany 26.10.2015

slide-2
SLIDE 2

Outline

Black-Box Optimization Algorithms Particle Swarm Optimization Evolutionary Algorithm Differential Evolution Local Search

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 2

slide-3
SLIDE 3

Black-Box Optimization

slide-4
SLIDE 4

Black-Box Optimization Definition (Black-Box Optimization Problem (BBOP))

  • domain D
  • objective function f : D → R
  • goal ∈ {min, max}

Task: Find x ∈ D such that f(x) = goal{f(y) | y ∈ D} No further assumptions about the problem structure! x ∈ D f : D → R f(x)

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 4

slide-5
SLIDE 5

Algorithms

slide-6
SLIDE 6

Particle Swarm Optimization (PSO)

Inspiration: cooperative behaviour of animals, “swarm intelligence”

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 6

slide-7
SLIDE 7

PSO (1)

  • D ⊆ Rn
  • a swarm of particles cooperatively explores the domain
  • particle i is. . .
  • a position xi ∈ Rn (candidate solution)
  • a velocity vi ∈ Rn
  • a local guide ℓi ∈ D (best solution found by this particle)
  • global guide g ∈ D: best solution found by the swarm so far

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 7

slide-8
SLIDE 8

PSO (2)

particles move according to equations vi ← ωvi + U[0, c1] ⊙ (pi − xi) + U[0, c2] ⊙ (ℓi − xi) (1) xi ← xi + vi (2) xi

ℓi

g x′

i

vi

ℓi − xi

g − xi

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 8

slide-9
SLIDE 9

Evolutionary Algorithm (EA)

Inspiration: natural selection

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 9

slide-10
SLIDE 10

The Evolutionary Cycle

termination? parent population

  • ffspring population

mutated offsprings initialization parent selection crossover mutation s u r v i v

  • r

s e l e c t i

  • n

variety of mutation, crossover, and selection methods; may include problem dependent knowledge

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 10

slide-11
SLIDE 11

Differential Evolution (DE)

Inspiration: Evolutionary Algorithms

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 11

slide-12
SLIDE 12

DE Algorithm

  • similar to evolutionary cycle
  • population of k candidate solutions (individuals), domain Rn
  • create mutant for each individual xi:
  • pick distinct r1, r2, r3 ∈ {1, . . . , k}, F u.a.r. ∈ [0, 2],
  • let the mutant mi := xr1 + F(xr2 − xr3)
  • crossover: create new solution ci from mi and xi
  • pick index j and set cj

i = mj i

  • roll the dice for all other entries of ci
  • selection: replace xi with ci if f(ci) < f(xi)

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 12

slide-13
SLIDE 13

Local Search

Ridiculously simple, but apparently effective. . .

Example: Late-acceptance Hill Climbing

Algorithm 1: LATEACCEPTANCHILLCLIMBING

input : s❝✉r: initial solution, ▲: history size

  • utput: s❜❡st: best solution found so far

initialize history: ∀i ∈ {0, . . . , ▲ − 1} : ❈[i] = f(s❝✉r) while stopping criterion not met do

s♥❡①t ← ♥❡✐❣❤❜♦r✭s❝✉r✮ ❈next ← f(s♥❡①t)

if ❈next ≤ ❈[✐ mod ▲] then

s❝✉r ← s♥❡①t, update ❈[✐ mod ▲]

return s❜❡st

26.10.2015 | | Friedrich-Alexander-Universität Erlangen-Nürnberg | Black-Box Challenge 13

slide-14
SLIDE 14
slide-15
SLIDE 15

Black Box Challenge

Meta-heuristic Optimization for Arbitrary Problems

Michael Glaß, Moritz Mühlenthaler, Faramarz Khosravi, Tobias Schwarzer, Rolf Wanka

slide-16
SLIDE 16

Organizational Matters

  • Credit criteria (5 ECTS):
  • Implementation of the algorithm in the Black Box Challenge / OPT4J

environment

  • Submission of documented source code at the end of the seminar
  • A talk of 45 minutes including 15 minutes for Q&A
  • A 4-to-6-page paper documenting your algorithm concepts and

implementation

  • Talks will be given en bloc at the end of the semester
  • Date and time to be decided
  • Meetings:
  • After 3 weeks: Questions? Problems?
  • After 6 weeks: Show committed reference implementation
  • Two weeks before the talks: Show committed extensions and variations

3 Black Box Challenge | Introduction | October 2015

slide-17
SLIDE 17

Contact

  • Michael Glaß
  • glass@cs.fau.de
  • Moritz Mühlenthaler
  • moritz.muehlenthaler@cs.fau.de
  • Faramarz Khosravi
  • faramarz.khosravi@cs.fau.de
  • Tobias Schwarzer
  • tobias.schwarzer@cs.fau.de
  • Rolf Wanka
  • rwanka@cs.fau.de

Black Box Challenge | Introduction | October 2015 4

slide-18
SLIDE 18

Multi-Objective Optimization

Black Box Challenge | Introduction | October 2015 5

energy consumption costs

multiple, often conflicting design

  • bjectives

dominance convergence diversity

slide-19
SLIDE 19

This Semester: Single-Objective Optimization

Black Box Challenge | Introduction | October 2015 6

slide-20
SLIDE 20

Complex Search Problems

7 Black Box Challenge | Introduction | October 2015

slide-21
SLIDE 21

Optimization based on Evolutionary Algorithms

synthesis evaluation

8 Black Box Challenge | Introduction | October 2015

slide-22
SLIDE 22

Distinction between Genotype and Phenotype

9 Black Box Challenge | Introduction | October 2015

slide-23
SLIDE 23

Distinction between Genotype and Phenotype solution space search space

rng

  • bjective

space g random number generator

decode

p

create evaluate

y g – genotype p – phenotype y – objectives

10 Black Box Challenge | Introduction | October 2015

slide-24
SLIDE 24

Choose the Genotype according to your Problem

  • For continuous problems
  • e.g.,
  • For discrete problems
  • e.g., knapsack problem
  • For sequences
  • e.g., travelling salesman problem

Real Genotype (0.3 , 0.44 , 0.7)

Integer Genotype (27 , 3 , -2 , 8 , 10) Boolean Genotype (1 , 0 , 0 , 1 , 1) Permutation Genotype (a < d < c < b)

11 Black Box Challenge | Introduction | October 2015

slide-25
SLIDE 25

Composite Genotypes for Complex Problems

Composite Genotype Composite Genotype

Boolean Genotype (1 , 0 , 0 , 1 , 1)

Real Genotype (0.3 , 0.44 , 0.7)

Boolean Genotype (0 , 1 , 1 , 1 , 0 , 1 , 1)

12 Black Box Challenge | Introduction | October 2015

slide-26
SLIDE 26

Eclipse Project

  • MOEAD07.java
  • Implementation of the optimizer
  • MOEAD07Module.java
  • Configuration
  • See Opt4J tutorial
  • bhc.jar
  • The jar file to submit
  • blackholecompetition.launch
  • Eclipse run configuration
  • export.jardesc
  • Creates the jar file
  • Moead07.xml
  • Configuration for MOEAD and WFG problem
  • Submit optimizer-specific part

Black Box Challenge | Introduction | October 2015 13

slide-27
SLIDE 27

Accessing SVN code repository

  • Generate SSH key:

ssh-keygen

  • 2 files in /home/YOU/.ssh

id_rsa id_rsa.pub <- public key

  • Send public key to
  • faramarz.khosravi@cs.fau.de
  • tobias.schwarzer@cs.fau.de

Black Box Challenge | Introduction | October 2015 14

slide-28
SLIDE 28

Optimizer Submission

  • Web-based submission system
  • Comparison of your optimizer to other optimizers (Opt4J

implementations)

  • A number of benchmarks already available
  • Daily update of results
  • Task 1:
  • Submit exact implementation of previous work
  • Task 2:
  • Try to enhance previous work to achieve better results

Black Box Challenge | Introduction | October 2015 15

slide-29
SLIDE 29

Topics

  • Online Black-Box Algorithm Portfolios for Continuous Optimization
  • P. Baudiš and P. Pošík 2014 [doi>10.1007/978-3-319-10762-2_4]
  • It’s Fate: A Self-organising Evolutionary Algorithm
  • J. Bim et al. 2012 [doi>10.1007/978-3-642-32964-7_19]
  • A Local Genetic Algorithm for Binary-Coded Problems
  • T. Runarsson et al. 2006 [doi>10.1007/11844297_20]
  • A Compass to Guide Genetic Algorithms
  • G. Rudolph 2008 [doi>10.1007/978-3-540-87700-4_26]
  • Threshold accepting: A general purpose optimization algorithm

appearing superior to simulated annealing

  • G. Dueck and T. Scheuer 1990 [doi>10.1016/0021-9991(90)90201-B]

Black Box Challenge | Introduction | October 2015 16