10703 deep reinforcement learning
play

10703 Deep Reinforcement Learning Reinforcement Learning in Humans - PowerPoint PPT Presentation

Feb 09, 2024 •237 likes •448 views

10703 Deep Reinforcement Learning Reinforcement Learning in Humans and Animals Tom Mitchell October 29, 2018 Reading: Barto & Sutton Chapter 15 Tom Mitchell, October 2018 Outline RL in primates RL in humans Error signals and

  1. 10703 Deep Reinforcement Learning Reinforcement Learning in Humans and Animals Tom Mitchell October 29, 2018 Reading: Barto & Sutton Chapter 15 Tom Mitchell, October 2018

  2. Outline • RL in primates • RL in humans • Error signals and predictive coding Tom Mitchell, October 2018

  3. Reward based learning in primates Tom Mitchell, October 2018

  4. Dopamine As Reward Signal t [Schultz et al., Science , 1997] Tom Mitchell, October 2018

  5. Dopamine As Reward Signal t [Schultz et al., Science , 1997] Tom Mitchell, October 2018

  6. Dopamine As Reward Signal t [Schultz et al., Science , 1997] error r γ V(s ) V(s ) = + − t t 1 t + � 6 Tom Mitchell, October 2018

  7. Reward based learning in humans Tom Mitchell, October 2018

  8. RL Models for Human Learning [Seymore et al., Nature 2004] Tom Mitchell, October 2018

  9. [Seymore et al., Nature 2004] � 9 Tom Mitchell, October 2018

  10. One Theory of RL in the Brain from [Nieuwenhuis et al.] • Basal ganglia monitor events, predict future rewards • When prediction revised upward (downward), causes increase (decrease) in activity of midbrain dopaminergic neurons, influencing ACC • This dopamine-based activation somehow results in revising the reward prediction function. Possibly through direct influence on Basal ganglia, and via prefrontal cortex Tom Mitchell, October 2018

  11. Tom Mitchell, October 2018

  12. Tom Mitchell, October 2018

  13. Neuron Level Learning Mechanisms • Hebbian learning – fire together � wire together • Spike Timing Dependent Plasticity 
 (STDP) – if incoming neuron fires before outgoing 
 then strengthen connection – if incoming neuron fires after outgoing 
 then weaken connection • Reward modulated STDP – less understood – in some neurons, appears STDP occurs only if neuromodulator (e.g., dopamine) activity follows firing within time up to 10 sec Tom Mitchell, October 2018

  14. Tom Mitchell, October 2018

  15. Summary: Temporal Difference ML Model 
 Predicts Dopaminergic Neuron Acitivity during Learning • Evidence now of neural reward signals from – Direct neural recordings in monkeys – fMRI in humans (1 mm spatial resolution) – EEG in humans (1-10 msec temporal resolution) • Dopaminergic responses encode Temporal Difference error • Some differences, and efforts to refine the model – How/where is the value function encoded in the brain? – Study timing (e.g., basal ganglia learns faster than PFC ?) – Role of prior knowledge, rehearsal of experience, multi-task learning? Tom Mitchell, October 2018

  16. Predictive Coding Tom Mitchell, October 2018

  17. [Rao & Ballard, Nature, 1999] Tom Mitchell, October 2018

  18. [Rao & Ballard, 1999] Tom Mitchell, October 2018

  19. Tom Mitchell, October 2018

  20. Tom Mitchell, October 2018

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018

10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018

10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018 Recommended readings: Used Materials Much of the material and slides for this lecture were borrowed from Katerina Fragkiadaki, and Ruslan

1.25k views • 49 slides
10703 Deep Reinforcement Learning Policy Gradient Methods Part 3 Tom Mitchell October 8, 2018

10703 Deep Reinforcement Learning Policy Gradient Methods Part 3 Tom Mitchell October 8, 2018

10703 Deep Reinforcement Learning Policy Gradient Methods Part 3 Tom Mitchell October 8, 2018 Recommended readings: next slide. (not covered in Barto & Sutton) Used Materials Disclaimer : Much of the material and slides for this

781 views • 26 slides
10703 Deep Reinforcement Learning Exploration vs. Exploitation Tom Mitchell October 22, 2018

10703 Deep Reinforcement Learning Exploration vs. Exploitation Tom Mitchell October 22, 2018

10703 Deep Reinforcement Learning Exploration vs. Exploitation Tom Mitchell October 22, 2018 Reading: Barto & Sutton, Chapter 2 Used Materials Some of the material and slides for this lecture were taken from Chapter 2 of Barto &

785 views • 37 slides
10703 Deep Reinforcement Learning Solving known MDPs Tom Mitchell September 10, 2018 Many

10703 Deep Reinforcement Learning Solving known MDPs Tom Mitchell September 10, 2018 Many

10703 Deep Reinforcement Learning Solving known MDPs Tom Mitchell September 10, 2018 Many slides borrowed from Katerina Fragkiadaki Russ Salakhutdinov Markov Decision Process (MDP) A Markov Decision Process is a tuple is a

1.05k views • 36 slides
Asynchronous RL CMU 10703 Katerina Fragkiadaki Non-stationary data problem for Deep RL

Asynchronous RL CMU 10703 Katerina Fragkiadaki Non-stationary data problem for Deep RL

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Asynchronous RL CMU 10703 Katerina Fragkiadaki Non-stationary data problem for Deep RL Stability of training neural networks requires the gradient updates

933 views • 69 slides
10703 Deep Reinforcement Learning Tom Mitchell September 5, 2018 Solving known MDPs Many slides

10703 Deep Reinforcement Learning Tom Mitchell September 5, 2018 Solving known MDPs Many slides

9/7/18 10703 Deep Reinforcement Learning Tom Mitchell September 5, 2018 Solving known MDPs Many slides borrowed from Katerina Fragkiadaki Russ Salakhutdinov Markov Decision Process (MDP) A Markov Decision Process is a tuple is a

501 views • 14 slides
10703 Deep Reinforcement Learning Policy Gradient Methods Tom Mitchell October 1, 2018 Reading:

10703 Deep Reinforcement Learning Policy Gradient Methods Tom Mitchell October 1, 2018 Reading:

10703 Deep Reinforcement Learning Policy Gradient Methods Tom Mitchell October 1, 2018 Reading: Barto & Sutton, Chapter 13 Used Materials Much of the material and slides for this lecture were taken from Chapter 13 of Barto & Sutton

586 views • 35 slides
Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

540 views • 15 slides
1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement

1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement

1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement learning? Agent/Actor + Action + Environment + State + Reward How does reinforcement learning work?

792 views • 31 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Exploration and Function Approximation CMU 10703 Katerina Fragkiadaki This lecture Exploration

Exploration and Function Approximation CMU 10703 Katerina Fragkiadaki This lecture Exploration

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Exploration and Function Approximation CMU 10703 Katerina Fragkiadaki This lecture Exploration in Large Continuous State Spaces Exploration-Exploitation

1.21k views • 86 slides
Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature 2015] CS 486/686 University of Waterloo Lecture 20: July 10, 2017 Outline Value Function Approximation Linear approximation Neural

706 views • 19 slides
Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides
Natural Policy Gradients, TRPO, PPO CMU 10703 Katerina Fragkiadaki Part of the slides adapted

Natural Policy Gradients, TRPO, PPO CMU 10703 Katerina Fragkiadaki Part of the slides adapted

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Natural Policy Gradients, TRPO, PPO CMU 10703 Katerina Fragkiadaki Part of the slides adapted from John Shulman and Joshua Achiam Stochastic policies

758 views • 44 slides
Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning arXiv:1702.08165 Reinforcement Learning with Deep Energy-Based Policies 1 / 25 Reinforcement Learning Environment Action Reward Interpreter State

531 views • 25 slides
Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1.

Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1.

Introduction to Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 1 is due next Wednesday! Remember that Monday is a holiday, so no office hours 2. Remember to start forming final project

877 views • 48 slides
Disclosures Evidence Based Medicine: None Concomitant or Sequential Phlebectomy for

Disclosures Evidence Based Medicine: None Concomitant or Sequential Phlebectomy for

4/18/2015 Disclosures Evidence Based Medicine: None Concomitant or Sequential Phlebectomy for Varicosities with Venous Ablation? 2015 UCSF Vascular Symposium Warren Gasper, MD Assistant Professor of Surgery UCSF Division of Vascular

325 views • 4 slides
Inspection changes for 2019 Jacqui Perfect, Assistant Manager, GP Support www.lmc.org.uk Good

Inspection changes for 2019 Jacqui Perfect, Assistant Manager, GP Support www.lmc.org.uk Good

Care Quality Commission (CQC) Inspection changes for 2019 Jacqui Perfect, Assistant Manager, GP Support www.lmc.org.uk Good & Outstanding Practices (i) Approximately 10% of practices will continue to receive a face to face

418 views • 19 slides
Distributions Marc H. Mehlman marcmehlman@yahoo.com University of New Haven Marc Mehlman Marc

Distributions Marc H. Mehlman marcmehlman@yahoo.com University of New Haven Marc Mehlman Marc

Distributions Marc H. Mehlman marcmehlman@yahoo.com University of New Haven Marc Mehlman Marc Mehlman (University of New Haven) Distributions 1 / 49 Table of Contents Distributions 1 Discrete Random Variables 2 Common Discrete

871 views • 49 slides
Accu curacy acy of of Sol olar ar Radi adius us D Det eter erminat nation ons s from

Accu curacy acy of of Sol olar ar Radi adius us D Det eter erminat nation ons s from

Accu curacy acy of of Sol olar ar Radi adius us D Det eter erminat nation ons s from om S Sol olar ar Eclipse O pse Obs bser ervat ations, ons, and and Compar omparison w son with h SOHO D Dat ata David W. Dunham,

631 views • 24 slides
From SPARC_LAB to EuPRAXIA Massimo.Ferrario@LNF.INFN.IT Courtesy BELLA LAL Orsay, 7 March

From SPARC_LAB to EuPRAXIA Massimo.Ferrario@LNF.INFN.IT Courtesy BELLA LAL Orsay, 7 March

From SPARC_LAB to EuPRAXIA Massimo.Ferrario@LNF.INFN.IT Courtesy BELLA LAL Orsay, 7 March 2017 Hawking: the Solartron Towards the Planck scale: 1.22 10 19 GeV Without further novel technology, we will eventually need an accelerator as

1.09k views • 61 slides
The Super Flavour Factory

The Super Flavour Factory

The Super Flavour Factory

724 views • 60 slides
Probability and Statistics for Computer Science The weak law of large numbers gives us a

Probability and Statistics for Computer Science The weak law of large numbers gives us a

Probability and Statistics for Computer Science The weak law of large numbers gives us a very valuable way of thinking about expecta:ons. ---Prof. Forsythe Credit: wikipedia Hongye Liu, Teaching Assistant Prof, CS361, UIUC,

643 views • 61 slides
r.r.fi use indicator others 0 IT In PrEIj Xn I In e EE In E EE Ide c In Xn Iie E ECTS

r.r.fi use indicator others 0 IT In PrEIj Xn I In e EE In E EE Ide c In Xn Iie E ECTS

variance and covariance July 23,2020 f Applications of of Expectation Linearity Variance 2 Covariance and correlation 3 Last time Distributions p if Xsl Bernoulli CP x PREM ipso p i f pin pj I Xu PHX Bincngp Linearity of

518 views • 13 slides

More recommend