Towards Efficient Evolutionary Design of Autonomous Robots Peter Kr - - PowerPoint PPT Presentation

Towards Efficient Evolutionary Design of Autonomous Robots

Peter Krˇ cah

Computer Center, Charles University Ovocn´ y Trh 5, 116 36 Prague 1 Czech Republic peter.krcah@ruk.cuni.cz

September 23, 2008

Introduction

◮ Ultimate goal: to automatically design hardware and software

for robots capable of performing given tasks

◮ Previous works: Virtual Creatures (Sims 1994), GOLEM

(Lipson, Pollack 2000), Genobots (Hornby, Pollack, 2002), Miconi (2006)

◮ Fitness evaluation done in simulation ◮ Simulation is slow (only about 10x faster than reality)

◮ Single run takes typically several hours/days

◮ Possible solutions

◮ Increase computational power ◮ Use better algorithms

Introduction

◮ Ultimate goal: to automatically design hardware and software

for robots capable of performing given tasks

◮ Previous works: Virtual Creatures (Sims 1994), GOLEM

(Lipson, Pollack 2000), Genobots (Hornby, Pollack, 2002), Miconi (2006)

◮ Fitness evaluation done in simulation ◮ Simulation is slow (only about 10x faster than reality)

◮ Single run takes typically several hours/days

◮ Possible solutions

◮ Increase computational power ◮ Use better algorithms

Robot Morphology

◮ Morphology represented by a directed graph

◮ Each node represents a body part ◮ Each connection represents a physical joint between two parts

◮ Generative encoding inspired by Sims’ Virtual Creatures

◮ Phenotype is constructed by recursive traversal of the

connections of the genotype

◮ Recursive limit in each node prevents infinite cycles

Robot Control System

◮ Control system is distributed along the robot body ◮ Each body part contains a local feedforward neural

network, sensors and an effector

◮ Each neuron has one of 22 transfer functions ◮ Single independent neural network for central coordination

Problem Definition

◮ Robot is represented by a morphology graph with a nested

neural network in each node:

◮ Problem 1: How to do effective recombination?

◮ Intuition: We need to swap corresponding parts ◮ Existing approaches: Grafting, crossover (Sims), picking (Miconi)

◮ Problem 2: How to do define similarity measure?

◮ We need to find correspondence of structure elements ◮ Useful feature for advanced speciation techniques

Problem Definition

◮ Robot is represented by a morphology graph with a nested

neural network in each node:

◮ Problem 1: How to do effective recombination?

◮ Intuition: We need to swap corresponding parts ◮ Existing approaches: Grafting, crossover (Sims), picking (Miconi)

◮ Problem 2: How to do define similarity measure?

◮ We need to find correspondence of structure elements ◮ Useful feature for advanced speciation techniques

Problem Definition

◮ Robot is represented by a morphology graph with a nested

neural network in each node:

◮ Problem 1: How to do effective recombination?

◮ Intuition: We need to swap corresponding parts ◮ Existing approaches: Grafting, crossover (Sims), picking (Miconi)

◮ Problem 2: How to do define similarity measure?

◮ We need to find correspondence of structure elements ◮ Useful feature for advanced speciation techniques

Historical Markings

◮ Introduced by K. Stanley in 2002 as a part of NEAT

algorithm

◮ Algorithm for evolving neural networks ◮ Main principles:

• 1. Each new structure element created during evolution is

assigned a unique number (a historical marking)

• 2. Markings are inherited

◮ Properties:

• 1. We can track the genes during the evolution
• 2. Markings can be used as an age indicators

Historical Markings

◮ Introduced by K. Stanley in 2002 as a part of NEAT

algorithm

◮ Algorithm for evolving neural networks ◮ Main principles:

• 1. Each new structure element created during evolution is

assigned a unique number (a historical marking)

• 2. Markings are inherited

◮ Properties:

• 1. We can track the genes during the evolution
• 2. Markings can be used as an age indicators

Historical Markings

◮ Introduced by K. Stanley in 2002 as a part of NEAT

algorithm

◮ Algorithm for evolving neural networks ◮ Main principles:

• 1. Each new structure element created during evolution is

assigned a unique number (a historical marking)

• 2. Markings are inherited

◮ Properties:

• 1. We can track the genes during the evolution
• 2. Markings can be used as an age indicators

Historical Markings for Robots

◮ Historical markings used for:

◮ Nodes and connections in the morphology graph ◮ Neurons and neural connections inside each body part

Historical Markings for Robots

◮ Historical markings used for:

◮ Nodes and connections in the morphology graph ◮ Neurons and neural connections inside each body part

Historical Markings for Robots

◮ Historical markings used for:

◮ Nodes and connections in the morphology graph ◮ Neurons and neural connections inside each body part

Historical Markings for Robots

◮ Historical markings used for:

◮ Nodes and connections in the morphology graph ◮ Neurons and neural connections inside each body part

Crossover and Similarity Measure

• 1. Align morphological nodes based on their markings
• 2. Align neurons inside morphological nodes based on their

markings

• 3. Do a regular crossover or compute similarity

Similarity Measure

◮ Similarity measure of A and B

◮ 0 ... if A and B are the same ◮ 1 ... if A and B are completely different

◮ Similarity of two robots

◮ The average similarity of their corresponding nodes and

connections

◮ Similarity of two nodes

◮ The average similarity of their neurons and neural connections

◮ Genes without a counterpart count as if they have a

completely different counterpart

HierarchicalNEAT

◮ Speciation

◮ Explicit fitness sharing (Goldberg, Richardson, 1987) ◮ Robots divided into species based on their similarity ◮ Fitness is shared among all robots in a single species

◮ HierarchicalNEAT (hNEAT) - An algorithm for evolving

morphology and control of robotic organisms

◮ Historical markings for both morphology and neural networks ◮ Crossover and similarity measure based on historical

markings

◮ Speciation using explicit fitness sharing

HierarchicalNEAT

◮ Speciation

◮ Explicit fitness sharing (Goldberg, Richardson, 1987) ◮ Robots divided into species based on their similarity ◮ Fitness is shared among all robots in a single species

◮ HierarchicalNEAT (hNEAT) - An algorithm for evolving

morphology and control of robotic organisms

◮ Historical markings for both morphology and neural networks ◮ Crossover and similarity measure based on historical

markings

◮ Speciation using explicit fitness sharing

Experimental Setup

◮ Fitness functions: walking, jumping, swimming,

light-following

◮ Winning fitness value defined for each fitness function ◮ Performance measure: number of fitness evaluations needed

to find the winner

◮ Evolution fails if winner not found in the first 30 000 fitness

evaluations

◮ Each configuration tested 30 times for 100 generations ◮ Single evolution: 70 minutes on 7 computers ◮ All experiments: 8 months of CPU time (< 4 days using 70

computers)

◮ Statistical significance of the differences verified (p < 0.02)

Experimental Setup

◮ Fitness functions: walking, jumping, swimming,

light-following

◮ Winning fitness value defined for each fitness function ◮ Performance measure: number of fitness evaluations needed

to find the winner

◮ Evolution fails if winner not found in the first 30 000 fitness

evaluations

◮ Each configuration tested 30 times for 100 generations ◮ Single evolution: 70 minutes on 7 computers ◮ All experiments: 8 months of CPU time (< 4 days using 70

computers)

◮ Statistical significance of the differences verified (p < 0.02)

Measuring Performance

◮ Comparison of standard GA with grafting and crossover and

hNEAT

◮ Fitness functions: walking, swimming, jumping,

light-following

◮ hNEAT consistently better than standard GA

Ablation Experiments

◮ Analysis of individual components of the proposed algorithm ◮ Verification of the usefulness of mating and speciation

Configuration Mean Std Failed Full hNEAT 16704.53 6339.75 8% Non-mating hNEAT 22441.38 6146.31 20% Non-speciated hNEAT 20456.40 6313.65 20%

◮ Finding the best method for creating the initial population

Configuration Mean Std Failed Random - 2 nodes 11503.77 5871.19 3% Random - 3 nodes 16704.53 6339.75 8% Random - 4 nodes 29920.69 387.05 93% Uniform - 2 nodes 16811.39 8190.81 21%

Ablation Experiments

◮ Analysis of individual components of the proposed algorithm ◮ Verification of the usefulness of mating and speciation

Configuration Mean Std Failed Full hNEAT 16704.53 6339.75 8% Non-mating hNEAT 22441.38 6146.31 20% Non-speciated hNEAT 20456.40 6313.65 20%

◮ Finding the best method for creating the initial population

Configuration Mean Std Failed Random - 2 nodes 11503.77 5871.19 3% Random - 3 nodes 16704.53 6339.75 8% Random - 4 nodes 29920.69 387.05 93% Uniform - 2 nodes 16811.39 8190.81 21%

Ablation Experiments

◮ Analysis of individual components of the proposed algorithm ◮ Verification of the usefulness of mating and speciation

Configuration Mean Std Failed Full hNEAT 16704.53 6339.75 8% Non-mating hNEAT 22441.38 6146.31 20% Non-speciated hNEAT 20456.40 6313.65 20%

◮ Finding the best method for creating the initial population

Configuration Mean Std Failed Random - 2 nodes 11503.77 5871.19 3% Random - 3 nodes 16704.53 6339.75 8% Random - 4 nodes 29920.69 387.05 93% Uniform - 2 nodes 16811.39 8190.81 21%

Explaining the Performance

◮ Speciation maintains diversity and prevents stagnation in a

local optimum

◮ Crossover based on historical markings performs sensible

recombination of genes

Conclusion

◮ Contributions

◮ Our algorithm evolves robots with given fitness value twice as

fast as standard GA

◮ Crossover and similarity measure for complex structures

(graphs with nested graphs)

◮ Future works

◮ Further analysis and experimentation (complexity of evolved

creatures)

◮ Constructing evolved robots in reality ◮ Large-scale computation on clusters of 1000+ computers

Thank you for your attention.