SLIDE 1
Learning to Control Self-Assembling Morphologies
Generalization via Modularity
Deepak Pathak* Chris Lu* Trevor Darrell Phillip Isola Alyosha Efros
* equal contribution
Generalization via Modularity Deepak Chris Trevor Phillip - - PowerPoint PPT Presentation
Generalization via Modularity Deepak Chris Trevor Phillip - - PowerPoint PPT Presentation
Learning to Control Self-Assembling Morphologies Generalization via Modularity Deepak Chris Trevor Phillip Alyosha Pathak* Lu* Darrell Isola Efros * equal contribution NeurIPS 2019 How do we train a robot? Multiple tasks
SLIDE 2
SLIDE 3
SLIDE 4
- Multiple tasks
- Expert demonstrations
- Rewards, labels
- …
SLIDE 5
- Multiple tasks
- Expert demonstrations
- Rewards, labels
- …
- Self-supervision
- Curious exploration
- Learning “common sense”
- …
SLIDE 6
. . .
SLIDE 7
. . .
… even earlier?
SLIDE 8
Single to Multicellular
SLIDE 9
Single to Multicellular competition collaboration
SLIDE 10
Single to Multicellular shared objective competition collaboration
SLIDE 11
Compositionality has been useful in language …
[Andreas et. al. 2016]
SLIDE 12
How to implement compositionality in hardware?
SLIDE 13
Modular Co-evolution of Control and Morphology
SLIDE 14
Modular Co-evolution of Control and Morphology
Cylindrical Limb
SLIDE 15
Modular Co-evolution of Control and Morphology
Cylindrical Limb Configurable Motor Joint
SLIDE 16
Modular Co-evolution of Control and Morphology
SLIDE 17
Modular Co-evolution of Control and Morphology
SLIDE 18
Modular Co-evolution of Control and Morphology
Potential Magnetic Joint
SLIDE 19
Modular Co-evolution of Control and Morphology
Potential Magnetic Joint
SLIDE 20
Modular Co-evolution of Control and Morphology
Potential Magnetic Joint
Acts as single agent upon joining Rewards are shared!
SLIDE 21
Modular Co-evolution of Control and Morphology
Potential Magnetic Joint
Input = Local Sensory State Output = Torques, Link, Unlink Acts as single agent upon joining Rewards are shared!
SLIDE 22
Modular Co-evolution of Control and Morphology
Potential Magnetic Joint
Input = Local Sensory State Output = Torques, Link, Unlink Acts as single agent upon joining Rewards are shared!
SLIDE 23
Consider the task of “standing up” …
SLIDE 24
SLIDE 25
How to learn compositional controllers?
SLIDE 26
Idea: Shared policy network across limbs
Node Node Node Node Node Node Node Node Node Node Node Node
Nod in
SLIDE 27
Idea: Shared policy network across limbs
Node Node Node Node Node Node Node Node Node Node Node Node
Nod in
𝜌𝜄
shared policy
- utput
input
SLIDE 28
How to adapt when morphology changes?
SLIDE 29
How to adapt when morphology changes?
SLIDE 30
Network as reusable LEGO Blocks
SLIDE 31
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
SLIDE 32
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
SLIDE 33
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
same dimension
SLIDE 34
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
SLIDE 35
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
SLIDE 36
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
𝜌𝜄 𝜌𝜄 𝜌𝜄
SLIDE 37
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
𝜌𝜄 𝜌𝜄 𝜌𝜄
SLIDE 38
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
𝜌𝜄 𝜌𝜄 𝜌𝜄
SLIDE 39
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
𝜌𝜄 𝜌𝜄 𝜌𝜄
cut
SLIDE 40
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
𝜌𝜄 𝜌𝜄 𝜌𝜄 𝜌𝜄 𝜌𝜄 𝜌𝜄
and paste
cut
SLIDE 41
Network as reusable LEGO Blocks 𝜌𝜄
shared policy
- utput
input
message
- utput
message input
𝜌𝜄 𝜌𝜄 𝜌𝜄 𝜌𝜄 𝜌𝜄 𝜌𝜄
and paste
adaptation by conditioning cut
SLIDE 42
Dynamic Graph Networks
SLIDE 43
BTW, basically curriculum learning but in hardware
SLIDE 44
How well does it generalize?
SLIDE 45
SLIDE 46
. . .
SLIDE 47
a bit crazy… is it even possible in real world?
. . .
SLIDE 48
Self-Assembling Robots in the Real World
[Mark Yim’s Lab at UPenn] [Daniela Rus's Lab at MIT]
Also: [Modular Snake Robot – Howie Choset’s Lab at CMU]
SLIDE 49