I t Introduction to d ti t Evolutionary Algorithms Federico - - PDF document

▶
i t introduction to d ti t evolutionary algorithms
SMART_READER_LITE
LIVE PREVIEW

I t Introduction to d ti t Evolutionary Algorithms Federico - - PDF document

Sep 24, 2023 401 likes •569 views

4/17/2020 I t Introduction to d ti t Evolutionary Algorithms Federico Nesti, f.nesti@santannapisa.it Layout Evolutionary Algorithms basics Evolutionary Algorithms for Control CMA-ES 1 4/17/2020 Layout Evolutionary

slide-1
SLIDE 1

4/17/2020 1

I t d ti t Introduction to Evolutionary Algorithms

Federico Nesti, f.nesti@santannapisa.it

  • Evolutionary Algorithms basics

Layout

  • Evolutionary Algorithms for Control
  • CMA-ES
slide-2
SLIDE 2

4/17/2020 2

  • Evolutionary Algorithms basics

Layout

  • Evolutionary Algorithms for Control
  • CMA-ES

Evolutionary algorithms are Black-box, gradient-free

Evolutionary Algorithms

  • ptimization methods.

They are biologically inspired, and rely on the concept

  • f evolution and survival of the fittest.
slide-3
SLIDE 3

4/17/2020 3

Evolutionary Algorithms Evolutionary Algorithms

slide-4
SLIDE 4

4/17/2020 4

Evolutionary Algorithms

3 1 2

Evolutionary Algorithms

3 1 2

slide-5
SLIDE 5

4/17/2020 5

Evolutionary Algorithms

Step 4 – Reproduction/ recom bination: the survived individuals reproduce and their genes are recombined in the offspring to refill the population.

Parents Offsprings

Evolutionary Algorithms

Step 5 – Mutation: stochastically, it could happen that a mutation occurs in one or more of the individuals.

slide-6
SLIDE 6

4/17/2020 6

Evolutionary Algorithms

I terate steps 2 -5 : I nitialization I terate steps 2 5 : a Generation is composed of evaluation, survival, reproduction and mutation Ranking and survival Reproduction mutation. Mutation

Applications of EAs

Com puter-Aided Design p g

https://www.youtube.com/watch?v=aR5N2Jl8k14&t=207s

slide-7
SLIDE 7

4/17/2020 7

Applications of EAs

Evolvable Electronics Evolvable Electronics

Applications of EAs

Molecular Design Molecular Design

slide-8
SLIDE 8

4/17/2020 8

Applications of EAs

…and m any m ore! …and m any m ore!

  • Image Processing
  • Climatology
  • Finance and Economics
  • Social Sciences
  • Quality Control
  • Biological Applications

Evolutionary Algorithms for Control

slide-9
SLIDE 9

4/17/2020 9

EAs for Control

Run for M generations Observations, reward Actions

Survival, Mutation, Recombination

Reward

Ranking

Fitness

There are many evolutionary algorithms, and are

Evolutionary Algorithms

classified in lots of different categories. Some of them are intuitively described in this great blog post. In this lecture we are going to use only CMA-ES (Covariance Matrix Adaptation – Evolutionary Strategy),

  • ne of the most popular Evolutionary Algorithms For full
  • ne of the most popular Evolutionary Algorithms. For full

implementation details, refer to this tutorial.

slide-10
SLIDE 10

4/17/2020 10

Covariance Matrix Adaptation

Iterate for M generations

Covariance Matrix Adaptation

Iterate for M generations

slide-11
SLIDE 11

4/17/2020 11

Covariance Matrix Adaptation

Iterate for M generations

Covariance Matrix Adaptation

Iterate for M generations

slide-12
SLIDE 12

4/17/2020 12

Covariance Matrix Adaptation

Iterate for M generations

Reinforcement Learning Car

Acceleration,

Sh ll NN

Acceleration, Steer

Shallow NN

slide-13
SLIDE 13

4/17/2020 13

CMA-ES for Car Control

No obstacles 60 generations 20 individuals Keep 10 best

CMA-ES for Car Control

With obstacles

Same hyperp.

slide-14
SLIDE 14

4/17/2020 14 Evolutionary Algorithms for RL are promising, since

Conclusions

  • The search for the optimal solution is not done

sequentially, but « in parallel» for each different

  • solution. This allows a much more broader exploration

and could lead to better solutions.

  • There is no need to have a strong mathematical

background to optimize a network! background to optimize a network!