[PPT] - Harnessing Evolution: Evolution Strategies Christian Jacob Dept. PowerPoint Presentation

SLIDE 1

www.swarm-design.org Christian Jacob, University of Calgary

Christian Jacob

Dept. of Computer Science
Dept. of Biochemistry & Molecular Biology

University of Calgary

Harnessing Evolution: Evolution Strategies

SLIDE 2

www.swarm-design.org Christian Jacob, University of Calgary

Intelligent Designs

SLIDE 3

www.swarm-design.org Christian Jacob, University of Calgary

global maximum local maxima local maxima

In Search for Better “Designs”

SLIDE 4

www.swarm-design.org Christian Jacob, University of Calgary

Evolution in Action …

SLIDE 5

www.swarm-design.org Christian Jacob, University of Calgary

Evolutionary Search

Knowledge Reservoir

Set of possible solutions

Gleaning a reservoir of knowledge om interactions

ith the environment.

Selection

Fitness-dependent number of offspring

The sieve of selection cus out incorrect / unuseful

“knowledge”.

V

ariation V ariations of individual solutions

The learning system invents new variants of its old

ideas that are tested against environmental demands.

SLIDE 6

www.swarm-design.org Christian Jacob, University of Calgary

Evolution Strategies

1. Motivation: Cumulative Selection 2. Evolution Strategies

a. Enlightening Experiments b. Evolutionary Search Spaces c. Evolution Schemes

3. ES Chromosomes and Mutations

a. Object and Strategy Parameters b. Mutations of Object Parameters c. Adaptation of Strategy Parameters d. Recombinations

4. Evolution Strategies Visualized in Action E. Appendix: Meta-Evolution Strategies F. Appendix: EC Test Functions

SLIDE 7

www.swarm-design.org Christian Jacob, University of Calgary

Cumulative Selection

,LPYJK,ZPBGXWKTEKSQ,KLVCFZSJFGVZQWG ETTLXTKOL RF STRZGPURE CSYEPYBY SQEP EVOLUDION OF STRUKTURE STEP BZ,STEB (a) (b) (c) EVOLUTION OF STRUCTURE, STEP BY STEP (O)

SLIDE 8

www.swarm-design.org Christian Jacob, University of Calgary

A simplified version of the evolutionary

principle of adaptation is used to search for a predefined string

– starting from an initially random sequence of

characters and

– using iterated mutation and cumulative selection.

Random strings are compared to an objective

sentence.

Cumulative Selection

SLIDE 9

www.swarm-design.org Christian Jacob, University of Calgary

Cumulative Selection Algorithm

1. Initialization: Generate an initial set S = {s1, …, sn} of n individuals.
2. Initial evaluation: Evaluate all individuals and calculate their fitnesses.
3. Selection: Choose the best individual sbest ∈ S.
4. Mutation: From the best individual, generate a set of n-1 mutants:

M = { si’ := mut(sbest) | i = 1, …, n-1 }.

5. Evaluation: Evaluate all mutants and calculate their fitnesses.
6. Termination check:

a. If at least one of the individuals has achieved the maximum fitness, stop. b. Otherwise, generate a new selection set: S = {sbest} ∪ M.

7. Continue with step 3.

SLIDE 10

www.swarm-design.org Christian Jacob, University of Calgary

Strings Encoded by Numbers

From Evolvica Notebooks: http://www.cpsc.ucalgary.ca/~jacob/IEC

Each letter is encoded by its ASCII code.

Input: Input: Input: Output: Output: Output:

τ τinv

SLIDE 11

www.swarm-design.org Christian Jacob, University of Calgary

W

e define string mutation on a string s = s1 … sN as follows: mut(s, r, p) = s1’… sN’ where si’ = si if χreal (0,1) > p. si’ = m(si, r)

therwise.

m(x, r) = τinv( τ(x) + χint(-r, r) ).

χint(y, z) or χreal (y, z) returns a uniformly distributed, integer or

real random number from the interval [y, z].

The character x is translated into its number encoding τ(x).

Mutation on Strings

SLIDE 12

www.swarm-design.org Christian Jacob, University of Calgary

String Mutations

EVOLUTION OF STRUCTURE, STEP BY STEP FVOLUTIONYOF STTUCTURE, QTEP BY STEP mut(s,2,0.2): EVOLUTION OF STRUCTURE, STEP BY STEP DVOLUTIONZOF STRUDSUQE, SSEP,CY SSEP mut(s,1,0.2): EVOLUTION OF STRUCTURE, STEP BY STEP EVOLUTNON OFCOTRYFTUME, STEPBB STFP mut(s,5,0.2): s: s: s: EVOLUTION OF STRUCTURE, STEP BY STEP EVNLVTION OF SURUCTURE, STEP BY STEP mut(s, 1, 0.1) : EVOLUTION OF STRUCTURE, STEP BY STEP EVOLUTIOM OF STRVCTURE. STEP BZ STEP mut(s, 1, 0.2) : EVOLUTION OF STRUCTURE, STEP BY STEP EWNLVUHON,OE SSSUCUVRD.ZSUEP,CY,STEQ mut(s, 1, 0.5) : s: s: s:

Mutation on strings with mutation radius 1 and different mutation rates Mutation on strings with mutation rate 0.2 and varying mutation radii

From Evolvica Notebooks: http://www.cpsc.ucalgary.ca/~jacob/IEC

SLIDE 13

www.swarm-design.org Christian Jacob, University of Calgary

Mut. Radius: 1
Mut. Rate: 0.1
Mut. Radius: 5
Mut. Rate: 0.1
Mut. Radius: 1
Mut. Rate: 0.5
Mut. Radius: 5
Mut. Rate: 0.5

String Evolution: DEMOS

SLIDE 14

www.swarm-design.org Christian Jacob, University of Calgary

Mut. Radius: 1
Mut. Rate: 0.5
Mut. Radius: 1
Mut. Rate: 0.1

String Evolution: DEMOS

Mut. Radius: 5
Mut. Rate: 0.1
Mut. Radius: 5
Mut. Rate: 0.5

SLIDE 15

www.swarm-design.org Christian Jacob, University of Calgary

String Evolution

Mut. Radius: 2, Mut. Rate: 0.1

SLIDE 16

www.swarm-design.org Christian Jacob, University of Calgary

String Evolution

Hamming Distance Plots

Mutation radius: 2 Mutation rate: 0.2 Mutation radius: 4 Mutation rate: 0.1 Mutation radius: 2 Mutation rate: 0.1

SLIDE 17

www.swarm-design.org Christian Jacob, University of Calgary

Evolution Strategies

1.

Motivation: Cumulative Selection

2.

Evolution Strategies

Enlightening Experiments
Evolutionary Search Spaces
Evolution Schemes

3.

ES Chromosomes and Mutations

Object and Strategy Parameters
Mutations of Object Parameters
Adaptation of Strategy Parameters
Recombinations

4.

Evolution Strategies Visualized in Action

E.

Appendix: Meta-Evolution Strategies

F.

Appendix: EC Test Functions

SLIDE 18

www.swarm-design.org Christian Jacob, University of Calgary

Ingo Rechenberg Hans-Paul Schwefel (1973) Evolution for Engineering Design

Evolution Strategies (ES)

SLIDE 19

www.swarm-design.org Christian Jacob, University of Calgary

Evolution of Joint Plates

SLIDE 20

www.swarm-design.org Christian Jacob, University of Calgary

Evolution of Joint Plates: Results

Mutations Mutations Drag Drag

SLIDE 21

www.swarm-design.org Christian Jacob, University of Calgary

Evolution of Bent Pipes

SLIDE 22

www.swarm-design.org Christian Jacob, University of Calgary

Evolution of Bent Pipes: Results

Mutations

Drag

SLIDE 23

www.swarm-design.org Christian Jacob, University of Calgary

Evolution of a Jet Nozzle

SLIDE 24

www.swarm-design.org Christian Jacob, University of Calgary

[I. Rechenberg: Evolutionsstrategie ‘94 Frommann-Holzboog, 1994.]

Evolution of a Jet Nozzle: Results

Experiment performed in 1968.

SLIDE 25

www.swarm-design.org Christian Jacob, University of Calgary

Evolution Strategies

1.

Motivation: Cumulative Selection

2.

Evolution Strategies

Enlightening Experiments
Evolutionary Search Spaces
Evolution Schemes

3.

ES Chromosomes and Mutations

Object and Strategy Parameters
Mutations of Object Parameters
Adaptation of Strategy Parameters
Recombinations

4.

Evolution Strategies Visualized in Action

E.

Appendix: Meta-Evolution Strategies

F.

Appendix: EC Test Functions

SLIDE 26

www.swarm-design.org Christian Jacob, University of Calgary

Evolutionary Algorithm Building Blocks

. . . . . . . . .

w µ Random selection

. . . . . .

2x Duplication

. . . . . . . . .

µ P

pulation of

µ individuals

. . .

Genotype of an individual

. . .

Phenotype of an individual, Individuals: Selection and Evaluation: Genetic Operators:

Q . . . . . . . . .

µ Selection

. . .

Mutation

. . . . . .

Recombination realization

. . .

Evaluation

Q . . . . . . . . .

t µ Isolation for t time units

x x x x x x x x x x x

SLIDE 27

www.swarm-design.org Christian Jacob, University of Calgary

(1+1)-ES and (1,1)-ES

. . . . . . . . . . . . . . . . . .

2x

Q Q

(1+1) ES (a)

. . . . . . . . . . . . . . . . . .

2x

Q Q

(1,1) ES (b)

SLIDE 28

www.swarm-design.org Christian Jacob, University of Calgary

(1+λ)-ES and (1, λ)-ES

(1+λ) ES (a)

. . . . . . . . . . . .

2x

Q . . . . . . Q . . . . . . Q

... ...

1 2 λ 2x 2x

Q . . . . . . . . .

1+λ (1,λ) ES (b)

. . . . . . . . . . . .

2x

Q . . . . . . Q . . . . . . Q

... ...

1 2 λ 2x 2x

Q . . . . . . . . .

λ

Mutation Evaluation Selection

SLIDE 29

www.swarm-design.org Christian Jacob, University of Calgary

(µ +λ)-ES and (µ, λ)-ES

. . . . . .

2x

Q . . . . . . Q . . . . . . Q

... ...

1 2 λ 2x 2x

Q . . . . . . . . .

µ+λ

. . . . . . . . .

µ

. . . . . . . . .

µ w

(a) . . . . . .

2x

Q . . . . . . Q . . . . . . Q

... ...

1 2 λ 2x 2x

Q . . . . . . . . .

λ

. . . . . . . . .

µ

. . . . . . . . .

µ w

(b)

SLIDE 30

www.swarm-design.org Christian Jacob, University of Calgary

(µ / 2, λ)-ES

. . . . . .

2x

Q . . . . . . Q . . . . . . Q

... ...

1 2 λ 2x 2x

Q . . . . . . . . .

λ

. . . . . . . . .

µ

. . . . . . . . .

µ w

. . . . . .

w

. . . . . .

w

. . . . . .

w

...

2x 2x 2x Selection Evaluation Recombination Mutation

SLIDE 31

www.swarm-design.org Christian Jacob, University of Calgary

Evolution Strategies

1.

Motivation: Cumulative Selection

2.

Evolution Strategies

Enlightening Experiments
Evolutionary Search Spaces
Evolution Schemes

3.

ES Chromosomes and Mutations

Object and Strategy Parameters
Mutations of Object Parameters
Adaptation of Strategy Parameters
Recombinations

4.

Evolution Strategies Visualized in Action

E.

Appendix: Meta-Evolution Strategies

F.

Appendix: EC Test Functions

SLIDE 32

www.swarm-design.org Christian Jacob, University of Calgary

ES Mutation & Selection

Gen. 1
Gen. 2
Gen. 3
Gen. 4
Gen. 5
Gen. 9
Gen. 10
Gen. 15

SLIDE 33

www.swarm-design.org Christian Jacob, University of Calgary

Basic ES Mutation & Selection

SLIDE 34

www.swarm-design.org Christian Jacob, University of Calgary

ES Object and Strategy Parameters

p'5 p'4 p'3 p'2 p'1 p5 p4 p3 p2 p1

p:

s'5 s'4 s'3 s'2 s'1 s5 s4 s3 s2 s1 Object parameters Strategy parameters

pmut:

+

N0 N0

... ... +

smut:

MSA, α MSA, α

s:

SLIDE 35

www.swarm-design.org Christian Jacob, University of Calgary

Gaussian Random Numbers

A random number is called normay or Gaussia

distributed with expected value µ and standard deviation σ if its density function is of the form

µ = 0

SLIDE 36

www.swarm-design.org Christian Jacob, University of Calgary

ES Object and Strategy Parameters

p'5 p'4 p'3 p'2 p'1 p5 p4 p3 p2 p1

p:

s'5 s'4 s'3 s'2 s'1 s5 s4 s3 s2 s1 Object parameters Strategy parameters

pmut:

+

N0 N0

... ... +

smut:

MSA, α MSA, α

s:

σ Who adapts the strategy parameters?

SLIDE 37

www.swarm-design.org Christian Jacob, University of Calgary

Mutative Step Size Adaptation (MSA)

The strategy parameters can be adapted using the following “Rechenberg” heuristics: smut = (s1 ε1, …, sn εn), where εi can be, for example, εi = β if r := rand(0,1) < 0.5 εi = 1/β if r >= 0.5 and β ∈ [1.3, 1.5].

SLIDE 38

www.swarm-design.org Christian Jacob, University of Calgary

Evolution Strategies

1.

Motivation: Cumulative Selection

2.

Evolution Strategies

Enlightening Experiments
Evolutionary Search Spaces
Evolution Schemes

3.

ES Chromosomes and Mutations

Object and Strategy Parameters
Mutations of Object Parameters
Adaptation of Strategy Parameters
Recombinations

4.

Evolution Strategies Visualized in Action

E.

Appendix: Meta-Evolution Strategies

F.

Appendix: EC Test Functions

SLIDE 39

www.swarm-design.org Christian Jacob, University of Calgary

ES Recombination: Two V ectors

p15 p14 p13 p12 p11

p1 s1 , ( ) :

s15 s14 s13 s12 s11 Object and strategy parameters p25 p24 p23 p22 p21 s25 s24 s23 s22 s21 p25 p14 p23 p22 p11 s25 s14 s23 s12 s11

p2 s2 , ( ) : pr sr , ( ) :

. . . . . .

ρ ρ ρ

Discrete recombination

(p1, s1) = ( 1 4 2 7 9 3 ) (p2, s2) = ( 3 5 9 8 4 2 ) (pr, sr) = ( 1 5 9 7 4 3 )

SLIDE 40

www.swarm-design.org Christian Jacob, University of Calgary

ES Recombination: Two V ectors

pa 5 pa 4 pa 3 pa 2 pa 1

pa sa , ( ) :

sa 5 sa 4 sa 3 sa 2 sa 1 Object and strategy parameters pb 5 pb 4 pb 3 pb 2 pb 1 sb 5 sb 4 sb 3 sb 2 sb 1 p'5 p'4 p'3 p'2 p'1 s'5 s'4 s'3 s'2 s'1

pb sb , ( ) : p' s' , ( ) :

. . . . . .

∅ ∅ ∅ ρ ρ ρ

Intermediate recombination

(p1, s1) = ( 1 4 2 7 9 3 ) (p2, s2) = ( 3 5 9 8 4 2 ) (pr, sr) = ( 2 4.5 5.5 7.5 6.5 2.5 )

SLIDE 41

www.swarm-design.org Christian Jacob, University of Calgary

ES Multi-Recombination

p15 p14 p13 p12 p11

p1 s1 , :

s15 s14 s13 s12 s11 p25 p24 p23 p22 p21

p2 s2 , :

s25 s24 s23 s22 s21 p35 p34 p33 p32 p31

p3 s3 , :

s35 s34 s33 s32 s31 p45 p44 p43 p42 p41

p4 s4 , :

s45 s44 s43 s42 s41 p55 p54 p53 p52 p51

p5 s5 , :

s55 s54 s53 s52 s51 p65 p64 p63 p62 p61

p6 s6 , :

s65 s64 s63 s62 s61 p75 p74 p73 p72 p71

p7 s7 , :

s75 s74 s73 s72 s71 p15 p64 p13 p62 p31 s65 s14 s33 s62 s31

p s , :

Global, discrete multi-recombination Local, discrete multi-recombination

p15 p14 p13 p12 p11

p1 s1 , :

s15 s14 s13 s12 s11 p25 p24 p23 p22 p21

p2 s2 , :

s25 s24 s23 s22 s21 p35 p34 p33 p32 p31

p3 s3 , :

s35 s34 s33 s32 s31 p45 p44 p43 p42 p41

p4 s4 , :

s45 s44 s43 s42 s41 p55 p54 p53 p52 p51

p5 s5 , :

s55 s54 s53 s52 s51 p65 p64 p63 p62 p61

p6 s6 , :

s65 s64 s63 s62 s61 p75 p74 p73 p72 p71

p7 s7 , :

s75 s74 s73 s72 s71 p15 p64 p13 p62 p31 s65 s14 s33 s62 s31

p s , :

SLIDE 42

www.swarm-design.org Christian Jacob, University of Calgary

Evolution Strategies

1.

Motivation: Cumulative Selection

2.

Evolution Strategies

Enlightening Experiments
Evolutionary Search Spaces
Evolution Schemes

3.

ES Chromosomes and Mutations

Object and Strategy Parameters
Mutations of Object Parameters
Adaptation of Strategy Parameters
Recombinations

4.

Evolution Strategies Visualized in Action

E.

Appendix: Meta-Evolution Strategies

F.

Appendix: EC Test Functions

SLIDE 43

www.swarm-design.org Christian Jacob, University of Calgary

Parametrized Faces (Chernoff Figures)

1. Head shape
2. Eye size
3. Eye distance
4. Eyes ratio
5. Pupil size

SLIDE 44

www.swarm-design.org Christian Jacob, University of Calgary

Parametrized Faces (2)

6. Eyebrow angle
7. Nose size
8. Mouth angle
9. Mouth width
10. Mouth opening

SLIDE 45

www.swarm-design.org Christian Jacob, University of Calgary

(GA) Point Mutations — pm = 0.1

SLIDE 46

www.swarm-design.org Christian Jacob, University of Calgary

(GA) Point Mutations — pm = 0.2

SLIDE 47

www.swarm-design.org Christian Jacob, University of Calgary

(GA) Point Mutations — pm = 0.5

SLIDE 48

www.swarm-design.org Christian Jacob, University of Calgary

(GA) Point Mutations — pm = 1.0

SLIDE 49

www.swarm-design.org Christian Jacob, University of Calgary

Recombination

Parents Offspring Parents Offspring Parents Offspring Parents Offspring Offspring Parents Offspring Parents

(1,1,1,1,1,2,2,2,2,2) (2,2,2,2,2,2,2,2,1,1) (2,1,1,1,1,2,1,2,2,1) (2,2,2,3,1,1,1,2,2,2) (3,2,3,2,2,3,1,3,2,2)

SLIDE 50

www.swarm-design.org Christian Jacob, University of Calgary

ES in Action: Three Populations

Gen. 0
Gen. 1
Gen. 2
Gen. 2

SLIDE 51

www.swarm-design.org Christian Jacob, University of Calgary

ES in Action: Three Populations (2)

Gen. 3
Gen. 4
Gen. 5
Gen. 10

SLIDE 52

www.swarm-design.org Christian Jacob, University of Calgary

Evolution Strategies

1.

Motivation: Cumulative Selection

2.

Evolution Strategies

Enlightening Experiments
Evolutionary Search Spaces
Evolution Schemes

3.

ES Chromosomes and Mutations

Object and Strategy Parameters
Mutations of Object Parameters
Adaptation of Strategy Parameters
Recombinations

4.

Evolution Strategies Visualized in Action

E.

Appendix: Meta-Evolution Strategies

F.

Appendix: EC Test Functions

SLIDE 53

www.swarm-design.org Christian Jacob, University of Calgary

Meta ES Evolution Scheme

2x 2x

. . . . . .

2x

Q

. . . . . .

Q

. . . . . .

Q

... ...

1 2 λ0

2x 2x

Q

. . . . . . . . .

µ0+λ0

. . . . . . . . .

µ0

. . . . . . . . .

µ0

w γ0

. . . . . . . . . . . . . . . . . .

1 λ1

. . . . . . . . .

Q

. . . . . . . . . . . . . . . . . .

...

µ1

. . . . . . . . . . . . . . . . . .

...

λ1

. . . . . . . . . . . . . . . . . .

...

1 µ1

w

Q Q

. . . . . .

2x

Q

. . . . . .

Q

. . . . . .

Q

... ...

1 2 λ0

2x 2x

Q

. . . . . . . . .

µ0+λ0

. . . . . . . . .

µ0

. . . . . . . . .

µ0

w γ0

SLIDE 54

www.swarm-design.org Christian Jacob, University of Calgary

Meta-ES Evolution in Action

Meta-Gen. 0 Meta-Gen. 1 Meta-Gen. 1 Meta-Gen. 2

SLIDE 55

www.swarm-design.org Christian Jacob, University of Calgary

Meta-Gen. 3 Meta-Gen. 4 Meta-Gen. 5 Meta-Gen. 5

Meta-ES Evolution in Action (2)

SLIDE 56

www.swarm-design.org Christian Jacob, University of Calgary

Evolution Strategies

1.

Motivation: Cumulative Selection

2.

Evolution Strategies

Enlightening Experiments
Evolutionary Search Spaces
Evolution Schemes

3.

ES Chromosomes and Mutations

Object and Strategy Parameters
Mutations of Object Parameters
Adaptation of Strategy Parameters
Recombinations

4.

Evolution Strategies Visualized in Action

E.

Appendix: Meta-Evolution Strategies

F.

Appendix: EC Test Functions

SLIDE 57

www.swarm-design.org Christian Jacob, University of Calgary

EC Test Functions

F1: Sphere Model F2: Rosenbrock Fct. F3: Step Function

F4: Parabola wt. Gaussian Noise

F5: Shekel’s Foxholes F6: Rastrigin Fct. F7: Schwefel Fct

F8: Griewangk Fct

SLIDE 58

www.swarm-design.org Christian Jacob, University of Calgary

Initiated from engineering design problems.
Usually, no distinction between search and solution space.
Individuals are represented as real-valued vectors.
Simple Evolution Strategies (ES):

– One parent and one child, – Child solution is generated by randomly mutating the

problem parameters of the parent.

More advanced ES

– have pools of parents and children.

Unlike GA and GP

,

– ES separate parent individuals from child individuals, and – ES selects its parent solutions deterministically.

Evolution Strategies: Summary

SLIDE 59

www.swarm-design.org Christian Jacob, University of Calgary

Bäck, T. (1996). Evolutionary Algorithms in Theory and Practice. Oxford,

Oxford University Press.

Jacob, C. (2001). Illustrating Evolutionary Computation with
Mathematica. San Francisco, CA, Morgan Kaufmann Publishers. Chapter 4.
Rechenberg, I. (1965). "Cybernetic solution path of an experimental

problem." Royal Aircraft Establishment, Library Translation 1122.

Schwefel, H.-P. (1981). Numerical optimization of computer models.

Chichester, Wiley.