SLIDE 1
Migration Ecology of Birds, Ian Newton
Learning How to Soar Learning How to Soar Terrence Sejnowski Salk - - PowerPoint PPT Presentation
Learning How to Soar Learning How to Soar Terrence Sejnowski Salk - - PowerPoint PPT Presentation
Learning How to Soar Learning How to Soar Terrence Sejnowski Salk Institute UCSD Bird Migration Bird Migration Migration Ecology of Birds, Ian Newton Thermal Soaring Thermal Soaring Rayleigh-Bnard Convection Rayleigh-Bnard Convection
SLIDE 2
SLIDE 3
Thermal Soaring Thermal Soaring
SLIDE 4
Rayleigh-Bénard Convection Rayleigh-Bénard Convection
SLIDE 5
Atmospheric Turbulence Atmospheric Turbulence
SLIDE 6
Tracking a Falcon with GPS Tracking a Falcon with GPS
SLIDE 7
Humans Soar Too Humans Soar Too
SLIDE 8
Glider Aerodynamics
Control over bank angle and angle of attack
Bank angle Angle of attack
Shephard & Lambertucci, 2013
1 - male condor 2- female condor 3 - black vulture 4 - caracara 1 2 3 4
SLIDE 9
- What quantities do birds sense?
- Vertical velocities, temperature, gradients, etc?
- How should the bird respond to these cues?
Physics simulations are complex and there are many variables.
How do Birds Find and Navigate Thermals? How do Birds Find and Navigate Thermals?
Experiments are hard to control and strategies are difficult to infer from limited data What should an optimal agent sense?
SLIDE 10
Time is Honey
SLIDE 11
Karl von Frisch
SLIDE 12
Temporal Difference Learning Temporal Difference Learning
) ( ) ( : error
- TD
1 1 t t t t
s V s V r
Sutton and Barto, 1988
t t t t t b b s p a s p t t t
a s p a s p e e s s a a a s
) , ( ) , ( : s preference the Update , Pr ) , ( : s preference by determined are Actions
) , ( ) , ( t t t
s V s V ) ( ) ( : update function value The
SLIDE 13
VUMmx1 - Octopamine
Hammer and Menzel, 1997
SLIDE 14
Montague, Dayan and Sejnowski, 1994
Temporal Difference Learning Temporal Difference Learning
SLIDE 15
SLIDE 16
SLIDE 17
SLIDE 18
Dopamine Neurons Dopamine Neurons Actor Critic Model Actor Critic Model
Environment
Dopamine Reward Prediction Error Cerebral Cortex Basal Ganglia
Montague, Dayan and Sejnowski, 1996
SLIDE 19
Go Defeat, 2017 Go Defeat, 2017 Temporal Difference Learning Temporal Difference Learning
Dopamine Reward Prediction Error Cerebral Cortex Basal Ganglia
Ke Jie Ke Jie
Environment
DeepMind DeepMind
SLIDE 20
Vertical velocity field Temperature field
What Do Thermals Look Like? What Do Thermals Look Like?
Rayleigh-Benard convection Reddy, Vergassola, Sejnowski, 2017
SLIDE 21
Pre-training Post-training
Sink or Soar? Sink or Soar?
SLIDE 22
+5o 0o
- 5o
vz
1-2 meters
az Vertical acceleration
Learned Policy Learned Policy
Vertical velocity gradient
SLIDE 23
temperature climb rate vertical acceleration vz gradients angle of attack
az and vz gradients across wings are useful control over angle of attack is not useful az and vz gradients across wings are useful control over angle of attack is not useful
Conclusions Conclusions
SLIDE 24
Field Experiments Field Experiments
SLIDE 25
GoPro Glider GoPro Glider
SLIDE 26
Field Experiments Field Experiments
Gautam Reddy
SLIDE 27
Field Experiments Field Experiments
Bank angle (o) 30
- 30
50 100 Time (s) desired
- bserved
SLIDE 28
Measuring the Vertical Wind Velocity Measuring the Vertical Wind Velocity
GPS and barometer measurement give vertical ground velocity We need to estimate wind velocity ground vel. = wind vel. + glider’s air vel. GPS/baro modeling
Pitch(o) 20 s
- 8
8
Phugoid
SLIDE 29
Training a Glider in the Field Training a Glider in the Field
Reddy, Vergassola, Sejnowski, 2018
SLIDE 30
Training a Glider in the Field Training a Glider in the Field
SLIDE 31
Field Experiments Field Experiments
SLIDE 32
SLIDE 33