Evolutionary Algorithms Keith L. Downing The Norwegian University - - PowerPoint PPT Presentation

Evolutionary Algorithms

Keith L. Downing

The Norwegian University of Science and Technology (NTNU) Trondheim, Norway keithd@idi.ntnu.no

February 4, 2014

Keith L. Downing Evolutionary Algorithms

Malthus and Darwin

Thomas Malthus (1789) Population growth rate = f(pop size) ⇒ Exp. Growth Limited Resources Competition for those resources = Malthusian Crunch As many more individuals of each species are born than can possibly survive; and as, consequently, there is a frequently recurring Struggle for Existence, it follows that any being, if it vary however slightly in any manner profitable to itself, under the complex and sometimes varying conditions of life, will have a better chance of surviving, and thus be naturally selected (Charles Darwin, On the Origin of Species by Means

• f Natural Selection (1859), pg. 5)

Keith L. Downing Evolutionary Algorithms

3 Prerequisites for Evolution

1

Selection - some environmental factors must favor certain traits over others.

2

Variation - individuals must consistently arise that are significantly (although not necessarily considerably) different from their ancestors.

3

Heritability - children must, on average, inherit a good many traits from their parents to insure that selected traits survive generational turnover.

Keith L. Downing Evolutionary Algorithms

Basic Cycle of Evolution

Gtypes Immature Ptypes Adult Ptypes Mating Ptypes Gametes Selection Selection Variation Inheritance + Selection Development Maturation Mating Competition Mitosis & Meiosis Aging & Death Reproduction: Gamete Pairing Keith L. Downing Evolutionary Algorithms

Basic Cycle of an Evolutionary Algorithm

!"#$%& '()*+,

• "#$%&

.+/*"

• "#$%&
• 01%2"
• "#$%&
• 01%2"

!"#$%& 301)0")42 52(%1)"026% 7%8%*4$9%2" :)"2%&& ;%&")2< .+/*", =%*%6")42

• 01%2",

=%*%6")42 !"#$% '4$#)2< >%$14+/6")42? >%649@)20")42,A B/"0")42 .<%CD0&%+ :)*"%1)2< =%*%6")42 Keith L. Downing Evolutionary Algorithms

Evolutionary Algorithm Flowchart

Initialize child genotype population Development: Generate Phenotypes from Genotypes Test Fitness of Phenotypes G G G G G P G P G P G P G P G P 3 G P 7 G P 5 2 Adult Selection G P 7 G P 5 G P 8 G P 6 Parent Selection Young Adults G P 8 G P 8 G P 5 G P 6 G G G G Reproduction Begin Next Generation Retain Some Adults

Keith L. Downing Evolutionary Algorithms

Essential Components of an Evolutionary Algorithm

1

Genotype representation

2

Phenotype representation

3

Translator of genotypes into phenotypes (Development)

4

Genetic operators for forming new genotypes from existing

• nes (e.g. crossover and mutation)

5

Fitness assessment procedure

6

Selection strategy for child → adult phenotypes

7

Selection strategy for adult → parent phenotypes

Keith L. Downing Evolutionary Algorithms

Short Taps Puzzle: A Simple Example

Dennis Shasha,Scientific American, August 2003 Problem:

1

A CIA official must send N messages (of varying length) within a time interval T.

2

Several messages can have overlapping sending periods.

3

A spy is tapping signals, but to avoid detection, the spy can tap for at most K consecutive time units during the T-step interval.

4

To be intercepted, a message’s entire duration must be within the tap window.

5

The sender is willing to allow at most M messages to be tapped. Goal: Assign start times to the N messages such that, no matter when the K-step tap interval occurs, no more than M messages will be tapped. Example Scenario N = 7 messages of durations 2, 3, 4 .. 8 T = 15, K = 10, M = 3

Keith L. Downing Evolutionary Algorithms

Genotype & Phenotype

2 4 6 8 10 12 14 1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 0000 1011 1010 0111 Start sending mesage 1 at time 0 Start sending mesage 3 at time 10 Genotype Phenotype 0011 0000 0011 Start sending mesage 7 at time 3 Keith L. Downing Evolutionary Algorithms

Genetic Operators

1 1 1 1 1 1 1 1 1 1 1 1 1 1

Parents Children 1-Point Crossover

1 1 1 1 1 1 1 1

2-Point Crossover

1 1 1 1 1 1 1 1

Parents Children 5 4 9 3 1 4 13 3 7 4 12 3 4 2 15 5

Main operators for binary chromosomes crossover, mutation, inversion

Keith L. Downing Evolutionary Algorithms

Fitness

Message 3 Message 4 Message 6 Message 1 Message 7 K-Step Window Message 2 Message 5

Vulnerability (V) = # K-unit windows in which M or more messages are intercepted. Fitness =

1 1+V

Keith L. Downing Evolutionary Algorithms

Adult and Parent Selection

Adult Selection: Full Generational Replacement Parent Selection: Fitness Proportionate Roulette Wheel Fitness Individual Die Children Adults Keith L. Downing Evolutionary Algorithms

Evolving a Broadcast Schedule

Scenario N = 10 messages of durations 2, 3, 4, 4, 4, 4, 5, 6, 7, 8 T = 20, K = 10, M = 3 GA population size = 20

5 10 15 20 25 30 35 40 45 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Generation Fitness M ax Avg M in

Keith L. Downing Evolutionary Algorithms

The Evolved Schedule

2 4 6 8 10 12 14 16 18 1 2 3 4 5 6 7 8 9 10 11

1 2 3 4 5 6 7 8 9 10

Keith L. Downing Evolutionary Algorithms

Genotype Classifications

Syntactic - based on how they look - DeJong (2006)

1

Fixed-Length Linear Objects

2

Fixed-Size Nonlinear Objects

3

Variable-Length Linear Objects

4

Variable-Size NonLinear Objects Semantic - based on what they encode

1

Data Oriented - these encode several data values whose usage in the phenotype may vary but does not include actual data manipulation or program control.

2

Program Oriented - these encode explicit data processing and control information - supplementary data may also be encoded - to form the kernel of an executable program at the phenotypic level.

Keith L. Downing Evolutionary Algorithms

Program-Oriented Genotypes

*

• +

*

• 2

3 X * + 3 + * X

• 3

2 X X + + + *

• *

* X X X X 3 3

• 3
• 2

2 *

(lambda (X) (+ (* (* (* X X) X) (+ 3 3)) (+ (* -2 X) (- -3 2)))) Keith L. Downing Evolutionary Algorithms

Crossover in Program-Oriented Genotypes

• *

X X + + 3 1 X * + 2 X

• 3

X

(lambda (X) (- (* (+ X 1) X) (+ X 3))) (lambda (X) (* (+ X X) (- 2 3)))

• X

+ 3 * + X X 2

• 3

* X + 1 X (lambda (X) (- (- 2 3) (+ X 3))) (lambda (X) (* (+ X X) (* (+ X 1) X))) Keith L. Downing Evolutionary Algorithms

Crossover in Linear Program-Oriented Genotypes

Main trick: find valid subtrees in the linear chromosome

+ + + *

• *

* X X X X 3 3

• 3
• 2

2 *

1 2 3 2 1 1

• 1

Complete Subtree

Keith L. Downing Evolutionary Algorithms

Fitness

Biology Ability to produce offspring Or even....ability to produce viable offspring → ability to produce grandchildren! Evolutionary Computation Performance level on a problem or set of problems Used as basis for reproduction: biases adult and parental selection.

Keith L. Downing Evolutionary Algorithms

Fitness Variance

Biology: Fundamental Theorem of Natural Selection Evolutionary rate ∝ Population fitness variance ⇒ Evolutionary rate ∝ Reproduction-rate variance ⇒ No significant change in the population if all individuals have same reproduction rate. Evolutionary Algorithms: Selection Mechanisms Early in Runs: Too much fitness variance ⇒ Premature Convergence Late in Runs: Mastery Effect - all individuals are good ⇒ Low fitness variance Selection mechanisms enable regulation of reproductive variance by scaling fitness values prior to choosing of parents.

Keith L. Downing Evolutionary Algorithms

Selection Pressure

Biology: Ruthlessness level of the world Degree to which superior phenotypes dominant the inferior w.r.t. reproductive rates Evolutionary Computation: Fitness scaling Degree to which individuals with higher (lower) fitness have higher (lower) reproductive rates. High Slightly higher (lower) fitness ⇒ Drastically higher (lower) reproduction rate. Low Drastically higher (lower) fitness ⇒Only slightly higher (lower) reproduction rate.

Keith L. Downing Evolutionary Algorithms

Fitness-Landscape

AS F

Genotype Space − → Phenotype Space − → Fitness Landscape

Keith L. Downing Evolutionary Algorithms

Selection Pressure = Fitness-Landscape Morph

Fitness Phenotype Fitness Phenotype Fitness Phenotype Early in an EA Run Late in an EA Run Fitness Phenotype

(Early) ⇓ variance via low selection pressure. (Late) ⇑ variance via high selection pressure.

Keith L. Downing Evolutionary Algorithms

Selection in Evolutionary Computation

a a b b

Adults

b

1 3 2 4

Fitness Scaling

Children Old Adults 5 2 1 3 4 5 Parents

Reproduction

Global Competition Local Tournaments 1 3 4 c 3 4 5 d

Sizes of rectangles denote relative fitness.

Keith L. Downing Evolutionary Algorithms

Adult Selection

Adult Pool Child Pool 3 18 11 12 Adult Pool Child Pool 7 10 12 20 Death 7 18 11 12 6 3 10 13 Adult Pool Child Pool 18 11 6 3 10 13 10 12 20 7 12 7 7 10 12 20 Death Death Full Generational Replacement Over-Production Replacement Generational Mixing

Keith L. Downing Evolutionary Algorithms

Parent (Mating) Selection

Global Fitness values are scaled to sectors on a virtual roulette wheel, which is then spun many times to select winners. Local Individuals square off head-to-head or in small groups, with winner normally the individual with the highest fitness value. No need for scaling the fitness values, unless the choice of winner is stochastic (i.e. using a mini roulette wheel).

Keith L. Downing Evolutionary Algorithms

Expected Value = Probable number of matings

Fitness Proportionate Selection ExpVal(i,g) = f(i) ¯ f(g) (1) i = individual, g = generation Average individuals reproduce one time, on average. Sigma-Scaling Selection ExpVal(i,g) = 1+ f(i)− ¯ f(g) 2σ(g) (2) Scale fitness by variance. Common consequence = reduced pressure early + increased late.

Keith L. Downing Evolutionary Algorithms

Comparing Roulette Wheels

Population Fitness Values = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] Fitness Proportionate Scaling Sigma Scaling

15% 16% 13% 18% 11% 2% 4% 9% 5% 7% 14% 16% 12% 17% 11% 3% 9% 4% 6% 8%

Keith L. Downing Evolutionary Algorithms

Early in an Evolutionary Run

Population Fitness Values = [1, 1, 1, 1, 1, 1, 1, 1, 3, 6] Fitness Proportionate Scaling Sigma Scaling

18% 35% 6% 6% 6% 6% 6% 6% 6% 6% 14% 8% 8% 8% 23% 8% 8% 8% 8% 8%

Sigma Scaling reduces variance.

Keith L. Downing Evolutionary Algorithms

Late in an Evolutionary Run

Population Fitness Values = [8, 8, 8, 8, 8, 8, 8, 8, 9, 10] Fitness Proportionate Scaling Sigma Scaling

10% 10% 10% 11% 10% 12% 10% 10% 10% 10% 15% 8% 8% 8% 23% 8% 8% 8% 8% 8%

Sigma Scaling increases variance.

Keith L. Downing Evolutionary Algorithms

Boltzmann Selection

ExpVal(i,g) = ef(i)/T ef(i)/Tg (3) T = a gradually-decreasing temperature parameter. (Early) High T → Low selection pressure. (Late) Low T → High selection pressure. Same principle as Simulated Annealing.

Keith L. Downing Evolutionary Algorithms

Roulette Wheels after Boltzmann Scaling

Population Fitness Values = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

12% 11% 14% 10% 15% 9% 6% 8% 7% 7%

T = 10

17% 14% 12% 21% 9% 3% 4% 8% 5% 6%

T = 5

40% 24% 15% < 1% < 1% 1% 2% 3% 5% 9%

T = 2

63% < 1% < 1% < 1% < 1% < 1% 1% 3% 23% 9%

T = 1

Keith L. Downing Evolutionary Algorithms

Rank Selection

ExpVal(i,g) = Min +(Max −Min)rank(i,g)−1 N −1 (4) N = population size; Min, Max = parameters (user-defined) expressing expected values for worst and best individuals. Normally, 1 ≤ Max ≤ 2 and Min = 2−Max. Helps keep selection pressure low early but higher later.

Keith L. Downing Evolutionary Algorithms

Roulette Wheel for Rank Selection

Population Fitness Values = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] Fitness-Proportionate Boltzmann (T = 10) Rank

15% 16% 13% 18% 11% 2% 4% 9% 5% 7% 12% 11% 14% 10% 15% 9% 6% 8% 7% 7%

T = 10

9% 11% 8% 12% 7% 6% 13% 5% 14% 15%

Rank Selection Scaling Range = [0.5, 1.5] Rank reduces selection pressure (i.e. evens out the distribution) in this case.

Keith L. Downing Evolutionary Algorithms

Tournament Selection (a local mechanism)

To select a parent, choose k (tournament-size) participants. With probability 1−ε, pick the participant with highest fitness as the winner. With probability ε, pick a random participant as the winner. Controlled Selection Pressure: High k or Low ε ⇒ High pressure Low k or High ε ⇒ Low pressure * Used very often!!

Keith L. Downing Evolutionary Algorithms

Classification of Search Techniques

Search Algorithms Whole Solutions Partial Solutions Serial (Traditional) Parallel (Evolutionary) Exhausive Local Linear Programming Whole Solutions Greedy Algs Divide & Conquer Branch & Bound Dynamic Prog'ing Best First (A*) Breadth-1st Hill- Climbing Simplex GA, ES, EP, GP How to Solve It: Modern Heuristics (Michalewicz & Fogel 2000) * Traditional methods can be parallel, but it's usually an independent parallelism Keith L. Downing Evolutionary Algorithms

Exploration -vs- Exploitation

Exploration Investing resources of a search on untried areas of the solution space. Exploitation Investing search resources on known (and promising) areas of the solution space. Examples

1

Mining: resource = people + machinery searching/digging at specific locations

2

Research funding: resource = money (to particular groups)

3

Sports coaching: resource = playing time.

4

Gambling: Finding a generous slot machine

Keith L. Downing Evolutionary Algorithms

Exploration -vs- Exploitation in EAs

General Relationships

Exploration Exploitation Genetic Operations Selection Strategies

More Specific Relationships

Exploration Exploitation Mutation Rate Crossover Rate Selection Pressure Degree of Elitism Tournament Size Adult Turnover Genetic Variation Direct Inverse Link Types

Keith L. Downing Evolutionary Algorithms

EC -vs GOFAI

Generator Tester Hs Hs Generator Tester Hs Hs Standard AI Problem Solvers Evolutionary Algorithms

EA = Dumb Generator + Intelligent Filter

Keith L. Downing Evolutionary Algorithms