[PDF] - Last time Genetics and evolution Genetic algorithms Assignment 4 PDF Document

SLIDE 1

1

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Last time

Genetics and evolution Genetic algorithms Assignment 4 Assignment 3

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Outline for today

Evolutionary computation

Overview

Genetic programming Genetic algorithms Aspects of evolution Classifier systems

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Evolutionary Computation - History

Evolutionary programming

Fogel, Owens, and Walsh (1966) Differs from genetic algorithms in three ways:

Representation: not constrained to be a string
No crossover
Different form of mutation, and typically reduced rate of

mutation during a run

Evolution strategies

Rechenberg (1965,1973), Schwefel (1975,1977) Independently developed Slightly different way of selection and mutation

compared to EP

Recombination is possible

SLIDE 2

2

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Evolutionary Computation - History

Genetic algorithms

John Holland (1960s)

Classifier Systems

John Holland (1976 ?) A cross between a Post production system, a genetic

algorithm, and a market economy

A hybrid nature: Both evolution and learning

Genetic programming

John Koza (1992) Evolving of whole programs Resembles GA, but program fragments are used instead

f strings

LISP 11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

The No Free Lunch Theorem

”The NFL theorem states that over all possible

search spaces, all methods perform equally well, including the simple technique of randomly guessing.” – Flake

No single method of optimization is best for all

applications

Evolutionary algorithms performs relatively well

when:

there is a large number of parameters to be determined the surface of solutions is complex, having many

intermediate optima

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Genetic Programming

An attempt to deal with one of the central

questions in computer science (posed by Arthur Samuel in 1959), namely

How can computers learn to solve problems without being

explicitly programmed? In other words, how can computers be made to do what needs to be done, without being told exactly how to do it? Any computer program can be graphically depicted

as an rooted point-labeled tree with ordered branches

The search space in genetic programming is the

space of all possible computer programs composed

f functions and terminals appropriate to the

problem domain

SLIDE 3

3

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Genetic Programming - Steps

In applying genetic programming to a

problem, there are five major preparing steps:

The set of terminals The set of primitive functions The fitness measure The parameters for controlling the run The method for designating a result and the

criterion for terminating a run Start with an initial population of randomly

generated computer programs

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Genetic Programming - Example

Koza, Rice, and Roughgarden (1992) Foraging strategies of Anolis lizards Questions:

”What makes for an optimal foraging strategy?” ”How can an evolutionary process assemble

strategies that require complex calculations from simple components?”

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Genetic Programming - Example

Four variables:

The abundance a of insects The sprint velocity v of the lizard The coordinate x, y of the insect in the lizard’s

view A strategy is a function of these variables

that returns 1 or -1

The goal: A function that maximizes food

capture per unit time

SLIDE 4

4

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Genetic Programming - Example

10 x 20 meter viewing area (fig 1a)

Region 1: Insects always escape Region 2: Insects never escape Region 3: Insects escape with probability zero

n the x axis and linearly increasing with the

angle to a maximum of 0.5 on the y axis Result, the best individual at generation

0 (fig 1b) 12 (fig 1c) 46 (fig 1d)

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Genetic Algorithms - Schema

How do genetic algorithm’s work? The Schema Theorem (Holland, 1975)

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Genetic Algoritms – Example: Coevolution

Hillis (1990) Host-parasite coevolution Adaptation in a static environment results

in

loss of diversity

verfit solutions

Problem:

Evolving minimal sorting networks for sorting

lists with a fixed number n of elements

Ex: (3,8), (14,8), (4,9), ... With n = 16, best known solution is 60

comparisons

SLIDE 5

5

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

GA – Example: Coevolution

Hillis used a GA Two criteria for networks in the population

Small size, implicitly favored through the

encoding

Correctness, explicitly through the fitness

function The fitness of a network, equal to the

percentage of correctly sorted cases

Spatial implementation, each individual

were placed on a two-dimensional lattice

Result (with static environment):

The GA got stuck on local optima 65 comparisons

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

GA – Example: Coevolution

Reason:

After a while the test cases were not

challenging enough Solution:

Let the test cases evolve The network’s fitness was the percentage of

test cases in the parasite that it sorted correctly

The fitness of the parasite was the percentage

f its test cases that the network sorted

incorrectly New result:

61 comparisons

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

The Blind Watchmaker

40% of all Americans (25% of college-

educated Americans) do not believe in Darwinian evolution (M. Mitchell, 1999)

Richard Dawkins (1996) ”Biomorphs”

A way to teach how evolution works

Variants

SimLife Creatures

SLIDE 6

6

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Lamarckian Evolution

”... the evolution of traits that are

modified through experience and passed

n, in their modified form, to the genotype
f the next generation” – M. Mitchell

Not possible in natural systems But artificial systems can use it

Needs a mean for adapting within a generation and a way of passing new gains to the genotype

f the next generation

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Lamarckian Evolution

Often more effective than Darwin

evolution in static environments

Each individual can try out many possibilities in

each generation But, not so effective when the environment

is dynamic

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

The Baldwin Effect

Also known as genetic assimilation ”... If learning or other forms of adaptation during

individuals’ lifetime are available, the desired configuration can arise via these mechanisms, and while the trait itself will not be passed on to

ffspring, the genetic background producing it will

be favored. Thus, according to Baldwin, learning and other forms of within-lifetime adaptation can lead to increased survival, which can eventually lead to genetic variation that produces the trait genetically.” – M. Mitchell

SLIDE 7

7

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Classifier Systems

Adaptation

Learning – in the lifetime of the agent Evolution – across generations

What about adaptation in systems between

learning and evolution

Culture Social Economic

Classifier systems combine

Genetic algorithms Environmental feedback Simple reinforcement learning

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Feedback and Control

Visible features usually correspond to a subset of

environment

Reinforcement

What differs adaptive systems from non-adaptive

Delayed rewards and punishments How does one find the optimal controller?

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Classifier Systems

Rules

if condition then action

Production, Expert, and Classifier systems Classifier systems

Are mostly used to control-like problems Almost never ”programmed”

SLIDE 8

8

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Classifier Systems

A classifier system consists of

List of classifiers

condition : message : strength
Ex: 1#0#:1001:37

List of messages

Messages describe the ”current” environment
Temporary storage space
Actions to take

Detectors

Sensory organs, post on the message list

Effectors

Can be used to modify the environment

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Classifier Systems

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Classifier Systems

1. The effectors place messages on the message list

2. A match set is formed from all suitable classifiers

3. The classifiers bid against each other. A function

f strength and maybe specificity. An action set

is formed from the highest bidders

4. The classifiers in the action set pay a portion of their bids to the other classifiers (if any) that were responsible for posting the message that matched their condition. The paid classifiers have their strengths increased as a result

SLIDE 9

9

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Classifier Systems

5. The message list is erased and a new message list

is formed from the message portions of all the classifiers in the action set

6. If any of the new messages in the message list

correspond to a real action, then the effectors process the action appropriately

7. If the environment rewards the classifier

system, then the reward is divided among the classifiers in the action set, which increases the strengths of the winning classifiers

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Classifier Systems

The ”bucket brigade” algorithm

Payments are passed down a line of classifiers,

reinforcing all in the chain

Represent the basis of a long-term memory

Genetic algorithms

Initially randomly selected classifiers After a while some classifiers will be strong A GA weed out the weak classifiers and form new ones

from the stronger classifiers Thus,

GA remove bad classifiers and introduces new,

potentially good classifiers

The ”bucket brigade” algorithm strengthening the good

nes

11/2 - 08 Emergent Systems, Jonny Pettersson, UmU

Summary

Evolutionary computation

Overview

Genetic programming Genetic algorithms Aspects of evolution Classifier systems