Genetic Algorithms in Robotics Julius Mayer Universit at Hamburg - - PowerPoint PPT Presentation

▶
genetic algorithms in robotics
SMART_READER_LITE
LIVE PREVIEW

Genetic Algorithms in Robotics Julius Mayer Universit at Hamburg - - PowerPoint PPT Presentation

Mar 15, 2024 228 likes •484 views

MIN-Fakult at Fachbereich Informatik Universit at Hamburg GAs in Robotics Genetic Algorithms in Robotics Julius Mayer Universit at Hamburg Fakult at f ur Mathematik, Informatik und Naturwissenschaften Fachbereich Informatik

slide-1
SLIDE 1

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik GA’s in Robotics

Genetic Algorithms in Robotics

Julius Mayer

Universit¨ at Hamburg Fakult¨ at f¨ ur Mathematik, Informatik und Naturwissenschaften Fachbereich Informatik Technische Aspekte Multimodaler Systeme

October 31, 2016

  • J. Mayer

1

slide-2
SLIDE 2

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik GA’s in Robotics

Outline

  • 1. Introduction

Motivation Classification

  • 2. Algorithm

Overview Phases

  • 3. Application

GA’s in Robotics Neuroevolution

  • 4. Discussion

Evaluation Conclusion

  • J. Mayer

2

slide-3
SLIDE 3

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Introduction - Motivation GA’s in Robotics

Motivation

Problems that are hard to solve with classical optimization because of

◮ too many parameters (intractability), ◮ large search space, ◮ non-differentiable objective functions, ◮ varying numbers of variables within the optimization process, ◮ a lacking mathematical function specification. [1]

  • J. Mayer

3

slide-4
SLIDE 4

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Introduction - Classification GA’s in Robotics

Classification

Biologic: Theory of evolution

◮ Adaptation of population to the

environment

◮ Gradual, hereditary change in

the individuals of a species

◮ Information storage and transfer

through genomes

◮ Optimizing population fitness by

reproduction

[3]

  • J. Mayer

4

slide-5
SLIDE 5

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Introduction - Classification GA’s in Robotics

Classification

Biologic: Theory of evolution

◮ Adaptation of population to the

environment

◮ Gradual, hereditary change in

the individuals of a species

◮ Information storage and transfer

through genomes

◮ Optimizing population fitness by

reproduction

[5]

  • J. Mayer

5

slide-6
SLIDE 6

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Introduction - Classification GA’s in Robotics

Classification

Algorithmic: Stochastic optimization

◮ Can ’solve’ NP-hard problems ◮ Solution approximation ◮ Computational complexity as

prohibiting factor

◮ e.g. ant colony & particle swarm

  • ptimization

[7]

  • J. Mayer

6

slide-7
SLIDE 7

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Algorithm - Overview GA’s in Robotics

Algorithm

  • 1. Initialization
  • 2. Evaluation
  • 3. Selection
  • 4. Mutation
  • 5. Terminate
  • J. Mayer

7

slide-8
SLIDE 8

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Algorithm - Phases GA’s in Robotics

Phase 1. Initialization

Initialize random population

◮ Choose appropriate

representation

◮ Individuals as encoded

problem solutions

◮ e.g. real values or binary

strings 1,67 2,34 1,04 5,83 2,10 101010011011010010101

[12]

  • J. Mayer

8

slide-9
SLIDE 9

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Algorithm - Phases GA’s in Robotics

Phase 2. Evaluation

◮ Choose objective function ◮ Decode & evaluate

chromosome

◮ Find minima in objective

function

◮ Survival of the fittest

[9]

  • J. Mayer

9

slide-10
SLIDE 10

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Algorithm - Phases GA’s in Robotics

Phase 3. Selection

◮ Probabilistic selection

◮ Roulette wheel selection ◮ Tournament Selection

◮ Balance exploration &

exploitation dilemma

  • J. Mayer

10

slide-11
SLIDE 11

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Algorithm - Phases GA’s in Robotics

Phase 4. Mutation

◮ Assign mutation rate ◮ Mutate

◮ Bitflips ◮ Normal distribution

mutation

◮ Recombination

◮ Crossover ◮ Permutation

◮ Mutation drives change

[12]

  • J. Mayer

11

slide-12
SLIDE 12

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Algorithm - Phases GA’s in Robotics

Phase 4. Mutation

◮ Assign mutation rate ◮ Mutate

◮ Bitflips ◮ Normal distribution

mutation

◮ Recombination

◮ Crossover ◮ Permutation

◮ Mutation drives change

[12]

  • J. Mayer

12

slide-13
SLIDE 13

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Algorithm - Phases GA’s in Robotics

Phase 5. Termination

◮ If solution is sufficiently close ◮ Performance is satisfying ◮ Change is stagnant

  • J. Mayer

13

slide-14
SLIDE 14

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Application - GA’s in Robotics GA’s in Robotics

GA’s in Robotics

◮ Evolutionary robotics ◮ Path planing ◮ Multivariate parameter

  • ptimization

◮ Evolving artificial network

architectures & optimizing connection weights (learning)

[6]

  • J. Mayer

14

slide-15
SLIDE 15

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Application - Neuroevolution GA’s in Robotics

Neuroevolution of augmenting topologies (NEAT)

◮ Uses crossover ◮ Uses speciation to protect

structures

◮ Increment growth from

minimal structures

[10]

  • J. Mayer

15

slide-16
SLIDE 16

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Application - Neuroevolution GA’s in Robotics

NEAT: Mutation

◮ Connection weights ◮ Network structure

◮ Add new connection

(random weight)

◮ Add new node (split

connection)

[10]

  • J. Mayer

16

slide-17
SLIDE 17

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Application - Neuroevolution GA’s in Robotics

NEAT: Crossover

◮ Genes are lined up according

to innovation number

◮ Matching Genes are randomly

chosen

◮ Disjoint / excess are taken

from fitter parent

[10]

  • J. Mayer

17

slide-18
SLIDE 18

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Application - Neuroevolution GA’s in Robotics

NEAT for RoboCup Soccer

Keepaway (robot soccer subtask)

◮ Large state space (unable to

explore exhaustively)

◮ Only partial state

information for each agent

◮ Continuous action space ◮ Multiple teammates need to

learn simultaneously

[4]

  • J. Mayer

18

slide-19
SLIDE 19

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Application - Neuroevolution GA’s in Robotics

NEAT vs. Temporal Difference Methods

[11]

  • J. Mayer

19

slide-20
SLIDE 20

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Application - Neuroevolution GA’s in Robotics

NEAT vs. Temporal Difference Methods

[11]

  • J. Mayer

20

slide-21
SLIDE 21

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Discussion - Evaluation GA’s in Robotics

Evaluation

Challenges

◮ Get GA’s parameters right ◮ Solution encoding ◮ Exploration vs. exploitation

◮ Premature convergence

(local minima)

◮ slow convergence ◮ big search space

◮ Find representative fitness

function

Drawbacks

◮ Approximation instead of

exact solution

◮ Computational complexity as

prohibiting factor

◮ No guaranteed convergence

to global optimum

◮ Non-exhaustive coverage of

the complete solution space

  • J. Mayer

21

slide-22
SLIDE 22

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Discussion - Conclusion GA’s in Robotics

Conclusion

◮ Inspired by biological evolution ◮ Stochastic optimization ◮ Population containing encoded

problem solutions (parameters)

◮ Different applications in robotics ◮ e.g. evolving ANN (NEAT) for

RoboCup Soccer

◮ No guarantee for convergence to

global optimum

[2]

  • J. Mayer

22

slide-23
SLIDE 23

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Discussion - Conclusion GA’s in Robotics

References

[1] http://rednuht.org/genetic walkers/, 2016. [2] https://cdn.meme.am/instances/41036988.jpg. 2016. [3] https://i.ytimg.com/vi/3ZJNyScv8to/maxresdefault.jpg, 2016. [4] https://i.ytimg.com/vi/HHlN0TDgllE/maxresdefault.jpg. 2016. [5] http://www.hanskottke.de/wordpress/wp-content/uploads/2012/09/evolutions hip hop.jpg. 2016. [6] http://www.orocos.org. 2016. [7] http://www.turingfinance.com/wp-content/uploads/2014/10/Self-Organizing-Feature-Map-3.png. 2016. [8] John H Holland. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. U Michigan Press, 1975. [9] Melanie Mitchell. Complexity: A Guided Tour. In Oxford, volume 1, chapter Cellular A, pages 145–159. Oxford University Press, Inc, New York, 2009.

  • J. Mayer

23

slide-24
SLIDE 24

Universit¨ at Hamburg

MIN-Fakult¨ at Fachbereich Informatik Discussion - Conclusion GA’s in Robotics

References (cont.)

[10] Kenneth O Stanley and Risto Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary computation, 10(2):99–127, 2002. [11] Matthew E Taylor. Comparing Evolutionary and Temporal Difference Methods in a Reinforcement Learning Domain. pages 1321–1328. [12] Mattias Wahde. Biologically Inspired Optimization Methods: An introduction. WIT Press, Boston, MA, 2008.

  • J. Mayer

24