Learning Latent Dynamics for Planning from Pixels Danijar Hafner, - PowerPoint PPT Presentation

Jul 03, 2023 •223 likes •446 views

Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson @danijar h danijar.com/planet Planning with Learned Models Watter et al., 2015, Banijamali

Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson @danijar h danijar.com/planet
Planning with Learned Models Watter et al., 2015, Banijamali et al. 2017, Zhang et al. 2017 Agrawal et al., 2016; Finn & Levine, 2016; Ebert et al., 2018
Visual Control Tasks partially many sparse contacts balance observable joints reward Some model-free methods can solve these tasks but need up to 100,000 episodes
Visual Control Tasks partially many sparse contacts balance observable joints reward Some model-free methods can solve these tasks but need up to 100,000 episodes
We introduce PlaNet Recipe for scalable model-based reinforcement learning 1 Efficient planning in latent space with large batch size 2 Reaches top performance using 200X fewer episodes 3
Latent Dynamics Model encode images
Latent Dynamics Model encode images predict states
Latent Dynamics Model encode images predict states decode images
Latent Dynamics Model encode images predict states decode images decode rewards
Recurrent State Space Model deterministic stochastic h 1 h 2 h 3 h 1 h 2 h 3 z 1 s 1 z 2 s 2 z 3 s 3 z 1 z 2 z 3 Recurrent Neural Network State Space Model Recurrent State Space Model
Unguided Video Predictions by Single Agent 5 frames context and 45 frames predicted
Unguided Video Predictions by Single Agent 5 frames context and 45 frames predicted
Planning in Latent Space
Planning in Latent Space
Planning in Latent Space
Planning in Latent Space
Planning in Latent Space
Planning in Latent Space
Comparison to Model-Free Agents Training time 1 day on a single GPU
Enabling More Model-Based RL Research Explore dynamics Distill the planner to save Value function to extend without supervision computation planning horizon
Learning Latent Dynamics for Planning from Pixels Website with code, videos, blog post, animated paper: danijar.com/planet 33

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer,

Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson @danijarh danijar.com/planet Planning with Learned Models Watter et al., 2015, Banijamali

519 views • 31 slides

Deep Learning Helicopter Dynamics Models Ali Punjani Pieter Abbeel UC Berkeley EECS Latent

Deep Learning Helicopter Dynamics Models Ali Punjani Pieter Abbeel UC Berkeley EECS Latent State: Airflow, Flexibility, Engine Dynamics etc. Similar trajectories have similar dynamics acceler acceleration ation stat state-contr e-control

372 views • 12 slides

Dream to Control Learning Behaviors by Latent Imagination Danijar Hafner, Timothy Lillicrap,

Dream to Control Learning Behaviors by Latent Imagination Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi Google Brain DeepMind @danijarh danijar.com/dreamer We introduce Dreamer Scalable reinforcement learning from pixels

428 views • 30 slides

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330 Student Presentation Table of Contents Motivation & problem Method overview Experiments Takeaways Discussion (strengths

653 views • 24 slides

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada,

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, Marc G. Bellemare Simple Representations for RL 2 12 DeepMDP Latent Space Model: Neural networks MDP:

573 views • 17 slides

Pixels Pixels Row and column indicates a PIXEL not a POINT. A pixel can theoretically contain

Pixels Pixels Row and column indicates a PIXEL not a POINT. A pixel can theoretically contain infinitely many points Drawing lines with pixels: fill pixels along the trajectory of a line from point p1 to p2 Aliasing Antialiasing

498 views • 15 slides

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine Joint work with Majid Janzamin Rong Ge UC Irvine Microsoft Research Latent Variable Probabilistic Models Latent (hidden) variable h R k ,

769 views • 47 slides

La Latent-sp space Dynam Dynamics ics for r Re Reduced Deformable Simulation Lawson Fulton

La Latent-sp space Dynam Dynamics ics for r Re Reduced Deformable Simulation Lawson Fulton 1,2 , Vismay Modi 1 , David Duvenaud 1 , David I.W. Levin 1 , Alec Jacobson 1 1 University of Toronto, Canada 2 MESH Consultants, Canada The 40

772 views • 59 slides

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges in Unsupervised Learning Learn a latent variable model without labeled examples. E.g. topic models, hidden Markov models, Gaussian mixtures,

763 views • 61 slides

Model-Free Control (Reinforcement Learning) and Deep Learning M ARC G. B ELLEMARE Google Brain

Model-Free Control (Reinforcement Learning) and Deep Learning M ARC G. B ELLEMARE Google Brain (Montral) 1956 1992 2016 2015 T HE A RCADE L EARNING E NVIRONMENT (B ELLEMARE ET AL ., 2013) 160 pixels Reward: change in score 210 pixels

793 views • 55 slides

STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in

STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in machine learning? Training continuous latent- variable models (VAEs, GANs) to produce large images Training large supervised models with

1.26k views • 61 slides

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine Joint work with Anima Anandkumar Rong Ge UC Irvine Microsoft Research Latent Variable Modeling Goal: Discover hidden effects from observed

1.27k views • 115 slides

Decoding in Latent Conditional Models: A Practically Fast Solution for an NP-hard Problem Xu Sun (

Latent dynamics workshop 2010 Decoding in Latent Conditional Models: A Practically Fast Solution for an NP-hard Problem Xu Sun ( ) University of Tokyo 2010.06.16 Outline Introduction Related Work & Motivations Our

741 views • 49 slides

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of Maryland furongh@cs.umd.edu ACM SIGMETRICS Tutorial 2018 1 / 75 Tutorial Topic Learning algorithms for latent variable models based on

1.96k views • 180 slides

1 Latent variable models In the next section we will discuss latent variable models for

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 16 notes: Latent variable models and EM Tues, 4.10 1 Latent variable models In the next section we will discuss latent

271 views • 4 slides

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes Mauricio A. Zhenwen Dai Alvarez Neil D. Lawrence Gaussian Process Approximation Workshop, 2017 Motivation I Machine learning has been very

685 views • 21 slides

Entropy minimization in emergent languages Eugene Kharitonov , Rahma Chaabouni, Diane Bouchacourt,

Entropy minimization in emergent languages Eugene Kharitonov , Rahma Chaabouni, Diane Bouchacourt, Marco Baroni Setup: signalling game (Lewis, 1969) Two deterministic neural agents, Sender sends a discrete message (one- Sender and

253 views • 24 slides

DReX: A Declarative Language for Efficiently Evaluating Regular String Transformations Rajeev

DReX: A Declarative Language for Efficiently Evaluating Regular String Transformations Rajeev Alur Loris DAntoni Mukund Raghothaman POPL 2015 1 DReX is a DSL for String Transformations align-bibtex ... ... @book{Book1 , @book{Book1 ,

436 views • 39 slides

on learned visual embedding patrick prez Allegro Workshop Inria Rhnes-Alpes 22 July 2015

on learned visual embedding patrick prez Allegro Workshop Inria Rhnes-Alpes 22 July 2015 Vector visual representation Fixed-size image representation High-dim ( 100 100,000 ) Generic, unsupervised: BoW, FV, VLAD / DBM, SAE

547 views • 32 slides

SLAM: COMPARATIVE APPROACH Khooshal Saurty 1 OUTLINE Introduction - What is SLAM? EKF SLAM

Intelligent Robotics Seminar - 31 October 2016 SLAM: COMPARATIVE APPROACH Khooshal Saurty 1 OUTLINE Introduction - What is SLAM? EKF SLAM FAST SLAM Comparison Cartographer Conclusion and References 2 INTRODUCTION - WHAT IS SLAM?

211 views • 20 slides

Latent Variable Models Volodymyr Kuleshov Cornell Tech Lecture 5 Volodymyr Kuleshov (Cornell

Latent Variable Models Volodymyr Kuleshov Cornell Tech Lecture 5 Volodymyr Kuleshov (Cornell Tech) Deep Generative Models Lecture 5 1 / 35 Announcements Glitches with Google Hangout link should be resolved. Will be checking email at the

352 views • 34 slides

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 3 is extended by one week, to Wednesday after next Todays Lecture 1. Managing overfitting in model-based RL

686 views • 40 slides

Meta-Reinforcement Learning of Structured Exploration Strategies Abhishek Gupta , Russell

Meta-Reinforcement Learning of Structured Exploration Strategies Abhishek Gupta , Russell Mendonca, YuXuan Liu, Pieter Abbeel, Sergey Levine Human Exploration vs Robot Exploration Human Exploration vs Robot Exploration Human Exploration vs Robot

1.2k views • 30 slides

Dream to Control: Learning Behaviors by Latent Imagination Danijar Hafner, Timothy Lillicrap,

Dream to Control: Learning Behaviors by Latent Imagination Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi Topic: Model Based RL Presenter: Haotian Cui Motivation and Main Problem 1-4 slides Should capture - High level

786 views • 26 slides