LunarLander-v2 using Deep Reinforcement Learning A project - PowerPoint PPT Presentation

Oct 08, 2022 •183 likes •300 views

LunarLander-v2 using Deep Reinforcement Learning A project developed for Autonomous Agents Course PLH513 Portokalakis Petros February 2020 Simple Game 8-Dimensional state space 4 actions per state +100 points for landing

LunarLander-v2 using Deep Reinforcement Learning A project developed for Autonomous Agents Course PLH513 Portokalakis Petros February 2020
Simple Game 8-Dimensional state space ● 4 actions per state ● +100 points for landing ● -100 points when crashed ● Infinite fuel, but -0.3 points per ● frame when firing main engine +10 for each leg ground contact (to ● encourage smooth landing)
Deep Reinforcement Learning Objective: approximate the optimal Q-Function (which satisfies the Bellman Equation) Neural network: 8 node input layer - dimensionality of state space ● 150 node fully connected 1st hidden layer ● 128 node fully connected 2nd hidden layer ● 4 node output layer - q-values for actions ● 4 layer approach works well with a variety of hidden layer node number 5 layers prove insufficient to even train the agent
Deep Reinforcement Learning: Advancing performance Experience replay: Every tuple(s,a,r,s’,done) is stored in a replay buffer (maxlength=1M) ● Randomly sample a batch of previous experiences (64). Break correlation ● between consecutive samples Predict best action for all items in the batch via the NN ● Update neural network weights ● Generate episodes via exploration or exploitation ●
Deep Reinforcement Learning: Advancing performance Calculating loss between output Q-value and target Q-value requires a seconds ● pass to the network for the next state s and s’ share the same network and have one step difference ● Optimization becomes unstable ● Target network: Use an identical network to the policy network, but update target network weight’s every C iterations (C is a hyperparameter) First pass occures with the policy network Second pass occures with the target network
Deep Reinforcement Learning: Advancing performance Abstract version of the agent algorithm implemented
Deep Reinforcement Learning: Performance of Lunar Lander
Deep Reinforcement Learning: Performance of Lunar Lander Adding a third hidden layer
Deep Reinforcement Learning: Hyperparameter Tuning Hyperparameter Value Starting epsilon 1 Minimum epsilon 0.01 Decay factor of epsilon 0.99 Discount factor gamma 0.99 Learning rate 0.001 Batch size 64 Replay buffer 1000000
Thank you Questions? Contact: pportokalakis@gmail.com

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

540 views • 15 slides

1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement

1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement learning? Agent/Actor + Action + Environment + State + Reward How does reinforcement learning work?

792 views • 31 slides

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature 2015] CS 486/686 University of Waterloo Lecture 20: July 10, 2017 Outline Value Function Approximation Linear approximation Neural

706 views • 19 slides

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning arXiv:1702.08165 Reinforcement Learning with Deep Energy-Based Policies 1 / 25 Reinforcement Learning Environment Action Reward Interpreter State

531 views • 25 slides

Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1.

Introduction to Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 1 is due next Wednesday! Remember that Monday is a holiday, so no office hours 2. Remember to start forming final project

877 views • 48 slides

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 3 is extended by one week, to Wednesday after next Todays Lecture 1. Managing overfitting in model-based RL

686 views • 40 slides

Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning

Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning for robots? slide from V. Vanhoucke End-to-end Deep Learning for robots? 2010 : Speech Recognition Audio Acoustic Model Phonetic Model

1.05k views • 65 slides

Inverse Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Todays

Inverse Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Todays Lecture 1. So far: manually design reward function to define a task 2. What if we want to learn the reward function from observing an expert, and

650 views • 33 slides

CMP784 DEEP LEARNING Lecture #12 Deep Reinforcement Learning Aykut Erdem // Hacettepe

DeepLoco by X. B. Peng, G. Berseth & M. van de Panne CMP784 DEEP LEARNING Lecture #12 Deep Reinforcement Learning Aykut Erdem // Hacettepe University // Spring 2018 Neural Face by Taehoon Kim Previously on CMP784 Generative

895 views • 71 slides

Deep Reinforcement Learning for Robotics:

Deep Reinforcement Learning for Robotics: Frontiers and Beyond Shixiang (Shane) Gu ( ) 2018.5.27 01 Deep RL: successes and limitations Computation-Constrained Data-Constrained

634 views • 21 slides

Introduction to Deep Reinforcement Learning and Control Spring 2019, CMU 10-403 Katerina

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Introduction to Deep Reinforcement Learning and Control Spring 2019, CMU 10-403 Katerina Fragkiadaki Course Logistics Course website : all you need to

1.13k views • 88 slides

Reinforcement Learning Timothy Chou Charlie Tong Vincent Zhuang April 19, 2016 Reinforcement

Reinforcement Learning Q-Learning Deep Q-Learning on Atari Reinforcement Learning Timothy Chou Charlie Tong Vincent Zhuang April 19, 2016 Reinforcement Learning Q-Learning Deep Q-Learning on Atari Table of Contents Reinforcement Learning

938 views • 63 slides

Deep Reinforcement Learning 1 Outline 1. Overview of Reinforcement Learning 2. Policy Search 3.

Deep Reinforcement Learning 1 Outline 1. Overview of Reinforcement Learning 2. Policy Search 3. Policy Gradient and Gradient Estimators 4. Q-prop: Sample Efficient Policy Gradient and an Off-policy Critic 5. Model Based Planning in Discrete

769 views • 53 slides

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning Haarnoja, Tang et al. (2017) Reinforcement Learning with Deep Energy Based Policies, ICML . Haarnoja, Zhou et al. (2018) Soft Actor-Critic: Off-Policy

682 views • 24 slides

ML MLOp Ops CI CI/CD CD for or Ma Machine Le Learn rning SASHA ROSENBAUM Sasha Rosenbaum

ML MLOp Ops CI CI/CD CD for or Ma Machine Le Learn rning SASHA ROSENBAUM Sasha Rosenbaum Sr. Program Manager @DivineOps https://www.deliveryconf.com/ Agenda Machine Learning 101 ML CI/CD Pipeline Overview Potential

800 views • 67 slides

Fast and Easy Hyper-Parameter Grid Search for Deep Learning GTC 2016 Mark Whitney Rescale

Fast and Easy Hyper-Parameter Grid Search for Deep Learning GTC 2016 Mark Whitney Rescale Overview Hyper-parameter optimization intro Intro to training on Rescale Random sampling demo Advanced optimization workflows Image

445 views • 33 slides

Expected Nodes : a quality function for the detection of link communities e Gaumont , Fran cois

u n i v e r s i t e p i e r r e e t m a r i e c u r i e - l i p 6 Expected Nodes : a quality function for the detection of link communities e Gaumont , Fran cois Queyroi, Cl emence Magnien and No Matthieu Latapy LIP6 - CNRS & UPMC ,

1.89k views • 26 slides

Cleaning Up the Neighborhood: Duplicate Ryan de Vera, Anna Ma, Daniel Moyer, Brendan Detection

Cleaning Up the Neighborhood: Duplicate Detection and Community Analysis of Hollenbeck Gangs Cleaning Up the Neighborhood: Duplicate Ryan de Vera, Anna Ma, Daniel Moyer, Brendan Detection and Community Analysis of Schneiderman

677 views • 26 slides

Trust Region Policy Optimization Yixin Lin Duke University yixin.lin@duke.edu March 28, 2017

Trust Region Policy Optimization Yixin Lin Duke University yixin.lin@duke.edu March 28, 2017 Yixin Lin (Duke) TRPO March 28, 2017 1 / 21 Overview Preliminaries 1 Markov Decision Processes Policy iteration Policy gradients TRPO 2

726 views • 21 slides

VGG/MOBILENET SSD Michael Sun July September, 2019 DATASETS FOR FINE-TUNING BelgaLogos

LOGO DETECTION WITH VGG/MOBILENET SSD Michael Sun July September, 2019 DATASETS FOR FINE-TUNING BelgaLogos FlickrLogos Combined LogosInTheWild DATASETS FOR FINE-TUNING (POST-CLEANING) BelgaLogos BelgaLogos-top LogosInTheWild

670 views • 32 slides

Using Deep Learning to Detect Galaxy Mergers Jonas Arilho Levy Supervisor: Mateus Espadoto [

Using Deep Learning to Detect Galaxy Mergers Jonas Arilho Levy Supervisor: Mateus Espadoto [ Co-supervisor: Prof. Dr. Roberto Hirata Junior ] Instituto de Matemtica e Estatstica da Universidade de So Paulo Contents: Objectives 3

470 views • 26 slides

Network Anomaly Detection in Modbus TCP Industrial Control Systems RP1 #52: Industrial Control

Network Anomaly Detection in Modbus TCP Industrial Control Systems RP1 #52: Industrial Control Systems Research Philipp Mieden & Rutger Beltman, 2020 Supervisor: Bartosz Czaszynski, Deloitte Industrial Network VS Corporate Network 2

379 views • 35 slides