SLIDE 1
PUBLIC
Deep Reinforcement Learning for Robotics Using DIANNE
Tim Verbelen, Steven Bohez, Elias De Coninck, Sam Leroux, Pieter Van Molle Bert VanKeirsbilck, Pieter Simoens, Bart Dhoedt
sam.leroux@ugent.be
Deep Reinforcement Learning for Robotics Using DIANNE Tim Verbelen, - - PowerPoint PPT Presentation
Deep Reinforcement Learning for Robotics Using DIANNE Tim Verbelen, - - PowerPoint PPT Presentation
Deep Reinforcement Learning for Robotics Using DIANNE Tim Verbelen, Steven Bohez, Elias De Coninck, Sam Leroux, Pieter Van Molle Bert VanKeirsbilck, Pieter Simoens, Bart Dhoedt sam.leroux@ugent.be PUBLIC How can we build robots that are able to
SLIDE 2
SLIDE 3
Kuka Youbot
3
5 axis arm Length: 66 cm Gripper Omnidirectional wheels Max speed: 0.8 m/s Battery operated Embedded PC
SLIDE 4
Hokuyo Laser rangefinder Kuka soft gripper
SLIDE 5
Reinforcement learning
5
Environment Agent
SLIDE 6
Reinforcement learning
6
Environment Observation Agent
SLIDE 7
Reinforcement learning
7
Environment Action Agent
SLIDE 8
Reinforcement learning
8
Environment Reward Agent
SLIDE 9
Deep Reinforcement learning
9
- The actor needs to process high dimensional observations to determine the next action.
- Our favorite processing block: deep neural networks
Observation Action
SLIDE 10
How can we train without destroying our robot ?
SLIDE 11
11
SLIDE 12
V-REP simulator
12
SLIDE 13
13
Multiple simulator instances gathering experience on CPU
SLIDE 14
14
Multiple simulator instances gathering experience on CPU GPU system training the model
SLIDE 15
15
Abstraction layer with ROS
Base Sensor Arm
SLIDE 16
How can we evaluate our models on the robot ?
SLIDE 17
17
Brain transplantation !
SLIDE 18
How can we connect the different components ?
SLIDE 19
19
SLIDE 20
20
Dianne
- Modular software framework for designing, training and evaluating neural networks.
- Distributed training and evaluation
- Java based
- Easy integration (service based architecture)
- GUI
- Open source (AGPL 3)
SLIDE 21
21
Deployed agent Deployed agent
SLIDE 22
22
Experience Pool Deployed agent Deployed agent
SLIDE 23
23
Experience Pool Repository Training Deployed agent Deployed agent
SLIDE 24
24
Experience Pool Repository Training Deployed agent Deployed agent
SLIDE 25
Deep Reinforcement learning algorithms
SLIDE 26
DQN
26
“Playing Atari with Deep Reinforcement Learning” (Mnih et al, 2013) Expected future return for each possible action raw laser scanner measurements (512 values) Q Values
SLIDE 27
27
SLIDE 28
DDPG
28
Continuous control with Deep Reinforcement Learning (Lillicrap, et al. 2015) Actor network Critic network raw laser scanner measurements (512 values) Continuous action Expected future return
SLIDE 29
29
SLIDE 30
30
Visit dianne.intec.ugent.be for more information
SLIDE 31
PUBLIC