Enhancing a Model-Free Adaptive Controller through Evolutionary - - PowerPoint PPT Presentation

▶
enhancing a model free adaptive controller through
SMART_READER_LITE
LIVE PREVIEW

Enhancing a Model-Free Adaptive Controller through Evolutionary - - PowerPoint PPT Presentation

Feb 14, 2024 110 likes •556 views

Enhancing a Model-Free Adaptive Controller through Evolutionary Computation Anthony Clark, Philip McKinley, and Xiaobo Tan Michigan State University, East Lansing, USA Anthony J. Clark - Adaptive Control - GECCO2015

slide-1
SLIDE 1
slide-2
SLIDE 2

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Enhancing a Model-Free Adaptive Controller through Evolutionary Computation

Anthony Clark, Philip McKinley, and Xiaobo Tan Michigan State University, East Lansing, USA

slide-3
SLIDE 3

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Aquatic Robots

Practical uses

– autonomous mobile sensors – biological studies (elicit natural behaviors)

slide-4
SLIDE 4

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Research platform

Simple physical design (relatively)

– few actuators

Nonlinear environment

– changing currents

Complex dynamics

– flexible fins

slide-5
SLIDE 5

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Focus on Control

We’d like controllers to:

  • 1. match oscillating frequency with material

properties

  • 2. handle changes in the environment
  • 3. handle changes to the robotic device
  • 4. …unknown conditions?

We do not want to account for these by hand

  • Leads us to adaptive control
slide-6
SLIDE 6

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Adaptive Control

Model-based

– require a precise model – perform parameter identification

Data-driven

– model-free – input / output data

slide-7
SLIDE 7

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Model-based Adaptive Control

Controller System

r e u y

+ _ Reference Model Adaptive Law

yp θ

slide-8
SLIDE 8

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Model-based Adaptive Control

Controller System

r e u y

+ _ Reference Model Adaptive Law

yp θ

Reference model:

  • based on complex

system dynamics

  • make simplifying

assumptions

slide-9
SLIDE 9

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Model-free Adaptive Control

For “gray-box” situations

– partial / incomplete information known about the system

MFA Controller System

r e u y

+ _

x d

+ +

slide-10
SLIDE 10

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

MFA Controller

System

r e u y +_ x d + +

Model-free Adaptive Control

slide-11
SLIDE 11

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Model-free Adaptive Control

What do we gain?

  • 1. do not have to create a dynamic model
  • 2. adapts to changing internal dynamics
  • 3. adapts to noisy environment
  • 4. adapts to varying high-level control input

What are the drawbacks?

  • 1. less precise
  • 2. still need to specify a number of parameters
  • ANN topology, learning rate, gain values, error

bounds, activation timing, network bias values

slide-12
SLIDE 12

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

This Study

Exploit EC to Enhance an MFAC

– evolve MFAC parameters – controlling a robotic fish – adapt to:

  • changing fin flexibilities
  • changing fin length
  • changing control demands
slide-13
SLIDE 13

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

MFAC vs. Neural Plasticity

Plastic neural networks

– will generally learn (or transition to) a new behavior – merge high-level logic and low-level control

Adaptive controllers

– regulate a control signal – behaviors are still determined at a higher level

slide-14
SLIDE 14

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Adaptive Neural Network

Network Activation

– feed-forward network – propagated error – sigmoid activation

Network Update

– minimize error

slide-15
SLIDE 15

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Adaptive Neural Network

slide-16
SLIDE 16

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Simulation

Task

  • Swim at a given speeds

Optimize

  • MFAC parameters

Adapt to:

  • different control signals
  • changing fin flexibilities
  • changing fin lengths

Evaluation

  • simulate for 60 seconds
  • mean absolute error
slide-17
SLIDE 17

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Baseline Experiment

MFA Controller System

r e u y

+ _

slide-18
SLIDE 18

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Baseline Experiment

MFA Controller System

r e u y

+ _

slide-19
SLIDE 19

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Baseline Experiment

MFA Controller System

r e u y

+ _

slide-20
SLIDE 20

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Baseline Experiment

MFA Controller System

r e u y

+ _

slide-21
SLIDE 21

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Baseline Experiment

MFA Controller System

r e u y

+ _ Output of the MFAC

  • regulates speed of the robotic fish
  • frequency of oscillation
slide-22
SLIDE 22

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Baseline Experiment

slide-23
SLIDE 23

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Differential Evolution

Evolutionary algorithm for real-valued problems Evolved parameters

– neural network size – learning rate – upper and lower error bounds – controller gain – controller update timing

slide-24
SLIDE 24

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Single Evaluation Experiment

60 120 −5 7.5 20

Time (s) Speed (cm/s)

y r e

  • verfit
slide-25
SLIDE 25

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Multiple Evaluations

Trial Flexibility Length sim1 100 % 100 % sim2 200 % 100 % sim3 50 % 100 % sim4 100 % 110 % sim5 200 % 110 % sim6 50 % 110 % sim7 100 % 90 % sim8 200 % 90 % sim9 50 % 90 %

slide-26
SLIDE 26

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Multiple Evaluations Experiment

60 120 −4 5 14

Time (s) Speed (cm/s)

y r e

slide-27
SLIDE 27

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Goals of the Study

We want to adapt to:

– changing fin flexibilities – changing fin length – changing control signal dynamics – any combination of the above changes

slide-28
SLIDE 28

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Fin Length

9−Evaluations (best replicate) : 80% length

Nominal Damaged Fin (60%) Attached Debris (137%)

4.5 cm 7.6 cm 10.4 cm

slide-29
SLIDE 29

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Control and Flexibility

Flexibility of 150% compared to the nominal value Different speeds Different accelerations Different decelerations

slide-30
SLIDE 30

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Simultaneous Changes

60 120 −4 5 14

9−Evaluations (best replicate) : 80% length, 120% flexibility

Time (s) Speed (cm/s)

y r e

slide-31
SLIDE 31

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Extended Multiple Evaluations

Trial Flexibility Length sim1 100 % 100 % sim2 200 % à 1000 % 100 % sim3 50 % à 10 % 100 % sim4 100 % 110 % à 200 % sim5 200 % à 1000 % 110 % à 200 % sim6 50 % à 10 % 110 % à 200 % sim7 100 % 90 % à 67 % sim8 200 % à 1000 % 90 % à 67 % sim9 50 % à 10 % 90 % à 67 %

slide-32
SLIDE 32

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Increase Simulation Ranges

60 120 −5 7.5 20

9−Evaluations, wide−range (best replicate)

Time (s) Speed (cm/s)

y r e

slide-33
SLIDE 33

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

When Adaptation Breaks-Down

slide-34
SLIDE 34

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

When Adaptation Breaks-Down

slide-35
SLIDE 35

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Key Points

  • 1. Attained adaptability

– to varying parameters for the robotic fish

  • 2. Performance was easily better than expert

chosen values

  • 1. Envelope of adaptability

– for evolution (tested values) – for operation (range of adaptability)

slide-36
SLIDE 36

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Ongoing Work

1. Integrate with high- level control

– self-modeling takes over when adaptation fails

1. Multiple-input, Multiple-output

– regulate speed and direction

1. Physical testing

– perform adaptation online

slide-37
SLIDE 37

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

The authors gratefully acknowledge the contributions and feedback on the work provided by:

  • Jared Moore,
  • Jianxun Wang, and
  • the BEACON Center at Michigan State University.

This work was supported in part by National Science Foundation grants IIS-1319602, CCF-1331852, CNS- 1059373, CNS-0915855, and DBI-0939454, and by a grant from Michigan State University.

slide-38
SLIDE 38

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

References

[Wang 2012] : Dynamic modeling of robotic fish with a flexible caudal fin.

– In Proceedings of the ASME 2012 5th Annual Dynamic Systems and Control Conference, joint with the JSME 2012 11th Motion and Vibration Conference, Ft. Lauderdale, Florida, USA, October 2012.

[Clark 2012] : Evolutionary design and experimental validation of a flexible caudal fin for robotic fish.

– In Proceedings of the Thirteenth International Conference on the Synthesis and Simulation of Living Systems, pages 325–332, East Lansing, Michigan, USA, July 2012.

[Rose 2013] : Just Keep Swimming: Accounting for Uncertainty in Self- Modeling Aquatic Robots

– In Proceedings of the 6th International Workshop on Evolutionary and Reinforcement Learning for Autonomous Robot Systems, Taormina, Italy, September 2013

slide-39
SLIDE 39

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Uncertainty in Robotics

Materials

– materials changing with temperature – flexibility changing due to water absorption

Hardware

– motors becoming less efficient

Environment

– transitioning from smooth to rough terrain

slide-40
SLIDE 40

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Address Uncertainty (1)

Mimicking biology

– biomimetic / bioinspired design

  • soft / flexible materials

– evolutionary design

  • evolutionary robotics and optimization

– evolving / learning behaviors

  • artificial neural networks (ANNs)
  • central pattern generators (CPGs)
slide-41
SLIDE 41

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Address Uncertainty (2)

Complex (feedback) control strategies

– robust control

  • handle a static range of uncertainty
  • robust to a noisy environment

– adaptive control

  • adapting to varying parameters
  • explicitly changes the controller’s dynamics
slide-42
SLIDE 42

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

Our Research

  • Autonomous behaviors
  • Feedback motor control
  • Biomimetic robots

High-level Control Low-level Control Robotic System

Sensor Feedback

slide-43
SLIDE 43

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

High-level Control Low-level Control Robotic System

Our Research

Neural Networks State Machine Adaptive Control Robotic Fish Wheeled Robot Digital Muscles

slide-44
SLIDE 44

Anthony J. Clark ––––- Adaptive Control ––––- GECCO2015

High-level Control Low-level Control Robotic System

Our Research

Neural Networks State Machine Adaptive Control Robotic Fish Wheeled Robot Digital Muscles