SLIDE 1
DeepMDP
Learning Latent Space Continuous Models for Representation Learning
Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, Marc G. Bellemare
DeepMDP Learning Latent Space Continuous Models for Representation - - PowerPoint PPT Presentation
DeepMDP Learning Latent Space Continuous Models for Representation - - PowerPoint PPT Presentation
DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, Marc G. Bellemare Simple Representations for RL 2 12 DeepMDP Latent Space Model: Neural networks MDP:
SLIDE 2
SLIDE 3
Neural networks
DeepMDP
Latent Space Model: MDP: & trained via the following two losses:
SLIDE 4
Reward Loss
SLIDE 5
Transition Loss
SLIDE 6
Tractable Losses
SLIDE 7
Deep Policies
SLIDE 8
Representation Quality
SLIDE 9
Only Discards:
Ferns, N., Panangaden, P., and Precup, D. Metrics for Finite Markov Decision
- Processes. In Proceedings of the 20th Conference on Uncertainty in Artificial
Intelligence, UAI ’04, pp. 162–169, 2004.
SLIDE 10
Phi as a Representation
SLIDE 11
Donut World
SLIDE 12
DeepMDP on Donut World
2D latent space + DeepMDP losses
SLIDE 13
DeepMDP on Donut World
Visualization of latent distance
SLIDE 14
DeepMDP Auxiliary Task
Base C51 agent + DeepMDP losses
SLIDE 15
DeepMDP Auxiliary Task
Base C51 agent + DeepMDP losses
SLIDE 16
- DeepMDPs as Models of the Environment
- Norm-MMD Metrics and their Associated Smoothness
SLIDE 17