Last time Genetics and evolution Genetic algorithms 4/2 - 10 - - PDF document

1

Last time

Genetics and evolution Genetic algorithms

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Outline for today

Evolutionary computation

Overview

Genetic programming Genetic algorithms

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Genetic algorithms Aspects of evolution Classifier systems

Evolutionary Computation - History

AI and ALife

Alan Turing, John von Neumann, Norbert Wiener Self-replicate and adaptivity

Evolutionary programming

Fogel, Owens, and Walsh (1966) Differs from genetic algorithms in three ways:

• Representation: not constrained to be a string

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

• Representation: not constrained to be a string
• No crossover
• Different form of mutation, and typically reduced rate of

mutation during a run

Evolution strategies

Rechenberg (1965,1973), Schwefel (1975,1977) Independently developed Slightly different way of selection and mutation

compared to EP

Recombination is possible

2 Evolutionary Computation - History

Genetic algorithms

John Holland (1960s)

Classifier Systems

John Holland (1976 ?) A cross between a Post production system, a genetic

algorithm and a market economy

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

algorithm, and a market economy

A hybrid nature: Both evolution and learning

Genetic programming

John Koza (1992) Evolving of whole programs Resembles GA, but program fragments are used instead

• f strings

LISP

The No Free Lunch Theorem

”The NFL theorem states that over all possible

search spaces, all methods perform equally well, including the simple technique of randomly guessing.” – Flake

No single method of optimization is best for all

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

No single method of optimization is best for all

applications

Evolutionary algorithms performs relatively well

when:

there is a large number of parameters to be determined the surface of solutions is complex, having many

intermediate optima

Genetic Programming

An attempt to deal with one of the central

questions in computer science (posed by Arthur Samuel in 1959), namely

How can computers learn to solve problems without being

explicitly programmed? In other words, how can computers be made to do what needs to be done, without b in t ld x ctl h t d it?

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

being told exactly how to do it? Any computer program can be graphically depicted

as an rooted point-labeled tree with ordered branches

The search space in genetic programming is the

space of all possible computer programs composed

• f functions and terminals appropriate to the

problem domain

3

Genetic Programming - Steps

In applying genetic programming to a

problem, there are five major preparing steps:

The set of terminals The set of primitive functions

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

The set of primitive functions The fitness measure The parameters for controlling the run The method for designating a result and the

criterion for terminating a run Start with an initial population of randomly

generated computer programs

Genetic Programming - Example

Koza, Rice, and Roughgarden (1992) Foraging strategies of Anolis lizards Questions:

”What makes for an optimal foraging strategy?”

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

What makes for an optimal foraging strategy? ”How can an evolutionary process assemble

strategies that require complex calculations from simple components?”

Genetic Programming - Example

Four variables:

The abundance a of insects The sprint velocity v of the lizard The coordinate x, y of the insect in the lizard’s

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

view A strategy is a function of these variables

that returns 1 or -1

The goal: A function that maximizes food

capture per unit time

4

Genetic Programming - Example

10 x 20 meter viewing area (fig 1a)

Region 1: Insects always escape Region 2: Insects never escape Region 3: Insects escape with probability zero

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

• n the x axis and linearly increasing with the

angle to a maximum of 0.5 on the y axis Result, the best individual at generation

0 (fig 1b) 12 (fig 1c) 46 (fig 1d)

Genetic Algorithms - Schema

How do genetic algorithm’s work? The Schema Theorem (Holland, 1975)

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Genetic Algoritms – Example: Coevolution

Hillis (1990) Host-parasite coevolution Adaptation in a static environment results

in

loss of diversity

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

loss of diversity

• verfit solutions

Problem:

Evolving minimal sorting networks for sorting

lists with a fixed number n of elements

Ex: (3,8), (14,8), (4,9), ... With n = 16, best known solution is 60

comparisons

5

GA – Example: Coevolution

Hillis used a GA Two criteria for networks in the population

Small size, implicitly favored through the

encoding

Correctness, explicitly through the fitness

function

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

The fitness of a network, equal to the

percentage of correctly sorted cases

Spatial implementation, each individual

were placed on a two-dimensional lattice

Result (with static environment):

The GA got stuck on local optima 65 comparisons

GA – Example: Coevolution

Reason:

After a while the test cases were not

challenging enough Solution:

Let the test cases evolve The network’s fitness was the percentage of

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

The network s fitness was the percentage of

test cases in the parasite that it sorted correctly

The fitness of the parasite was the percentage

• f its test cases that the network sorted

incorrectly New result:

61 comparisons

The Blind Watchmaker

40% of all Americans (25% of college-

educated Americans) do not believe in Darwinian evolution (M. Mitchell, 1999)

Only 40% of all Americans accept the

evolution theory (Aftonbladet, 090201)

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

y

Richard Dawkins (1996) ”Biomorphs”

A way to teach how evolution works

Variants

SimLife Creatures

6

Lamarckian Evolution

”... the evolution of traits that are

modified through experience and passed

• n, in their modified form, to the genotype
• f the next generation” – M. Mitchell

Not possible in natural systems

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Not possible in natural systems But artificial systems can use it

Needs a mean for adapting within a generation and a way of passing new gains to the genotype

• f the next generation

Lamarckian Evolution

Often more effective than Darwin

evolution in static environments

Each individual can try out many possibilities in

each generation But, not so effective when the environment

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

, ff is dynamic

The Baldwin Effect

Also known as genetic assimilation ”... If learning or other forms of adaptation during

individuals’ lifetime are available, the desired configuration can arise via these mechanisms, and while the trait itself will not be passed on to

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

while the trait itself will not be passed on to

• ffspring, the genetic background producing it will

be favored. Thus, according to Baldwin, learning and other forms of within-lifetime adaptation can lead to increased survival, which can eventually lead to genetic variation that produces the trait genetically.” – M. Mitchell

7

Classifier Systems

Adaptation

Learning – in the lifetime of the agent Evolution – across generations

What about adaptation in systems between

learning and evolution

Culture

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Culture Social Economic

Classifier systems combine

Genetic algorithms Environmental feedback Simple reinforcement learning

Feedback and Control

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Visible features usually correspond to a subset of

environment

Reinforcement

What differs adaptive systems from non-adaptive

Delayed rewards and punishments How does one find the optimal controller?

Classifier Systems

Rules

if condition then action

Production, Expert, and Classifier systems Classifier systems

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Classifier systems

Are mostly used to control-like problems Almost never ”programmed”

8

Classifier Systems

A classifier system consists of

List of classifiers

• condition : message : strength
• Ex: 1#0#:1001:37

List of messages

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

f g

• Messages describe the ”current” environment
• Temporary storage space
• Actions to take

Detectors

• Sensory organs, post on the message list

Effectors

• Can be used to modify the environment

Classifier Systems

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Classifier Systems

1.

The effectors place messages on the message list

2.

A match set is formed from all suitable classifiers

3.

The classifiers bid against each other. A function

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

g

• f strength and maybe specificity. An action set

is formed from the highest bidders

4.

The classifiers in the action set pay a portion of their bids to the other classifiers (if any) that were responsible for posting the message that matched their condition. The paid classifiers have their strengths increased as a result

9

Classifier Systems

• 5. The message list is erased and a new message list

is formed from the message portions of all the classifiers in the action set

• 6. If any of the new messages in the message list

correspond to a real action then the effectors

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

correspond to a real action, then the effectors process the action appropriately

• 7. If the environment rewards the classifier

system, then the reward is divided among the classifiers in the action set, which increases the strengths of the winning classifiers

Classifier Systems

The ”bucket brigade” algorithm

Payments are passed down a line of classifiers,

reinforcing all in the chain

Represent the basis of a long-term memory

Genetic algorithms

Initially randomly selected classifiers

Af hil l ifi ill b

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10 After a while some classifiers will be strong A GA weed out the weak classifiers and form new ones

from the stronger classifiers Thus,

GA remove bad classifiers and introduces new,

potentially good classifiers

The ”bucket brigade” algorithm strengthening the good

• nes

Summary

Evolutionary computation

Overview

Genetic programming Genetic algorithms

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

Genetic algorithms Aspects of evolution Classifier systems

10

Next time

Modeling and simulation

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

11

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 4/2 - 10 Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

12

ZCS – Wilson’s Woods1

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10

ZCS – Wilson’s Woods7

Emergent Systems, Jonny Pettersson, UmU 4/2 - 10