geometrically coupled monte carlo sampling
play

Geometrically Coupled Monte Carlo Sampling Mark Rowland Krzysztof - PowerPoint PPT Presentation

Aug 19, 2022 •43 likes •273 views

Geometrically Coupled Monte Carlo Sampling Mark Rowland Krzysztof Choromanski Franois Chalus Aldo Pacchiano Tamas Sarlos Richard E. Turner Adrian Weller Geometrically Coupled Monte Carlo Sampling Central goal: Unbiased Monte Carlo

  1. Geometrically Coupled Monte Carlo Sampling Mark Rowland Krzysztof Choromanski François Chalus Aldo Pacchiano Tamas Sarlos Richard E. Turner Adrian Weller

  2. Geometrically Coupled Monte Carlo Sampling Central goal: Unbiased Monte Carlo estimation: Can we do better than i.i.d.? Key contribution: K -optimality. Optimise the objective below over the joint distribution of INNOVATION + ASSISTANCE This leads to a multi-marginal transport problem, which is often analytically solvable. 2

  3. GCMC in Robotics - Policy Search - An Overview isotropic distribution INNOVATION + ASSISTANCE 3

  4. GCMC in Robotics - Policy Search - An Overview isotropic distribution INNOVATION + ASSISTANCE 4

  5. GCMC in Robotics - Policy Search - An Overview isotropic distribution INNOVATION + ASSISTANCE 5

  6. GCMC in Robotics - Policy Search - An Overview isotropic distribution INNOVATION + ASSISTANCE 6

  7. GCMC in Robotics - Policy Search - An Overview antithetic pair isotropic distribution INNOVATION + ASSISTANCE 7

  8. GCMC in Robotics - Policy Search - An Overview antithetic pair isotropic distribution Typical approach to Monte Carlo Sampling: INNOVATION + ASSISTANCE ● Independent Antithetic Pairs ● Coupled Samples of Equal Lengths 8

  9. GCMC in Robotics - Policy Search - An Overview antithetic pair isotropic distribution Typical approach to Monte Carlo Sampling: INNOVATION + ASSISTANCE ● Independent Antithetic Pairs ● Coupled Samples of Equal Lengths 9

  10. GCMC in Robotics - Policy Search - An Overview antithetic pair isotropic distribution Typical approach to Monte Carlo Sampling: GCMC: INNOVATION + ASSISTANCE ● Independent Antithetic Pairs ● orthogonal directions ● Coupled Samples of Equal Lengths of different antithetic pairs ● correlated unequal lengths within a pair ● variance reduction 10

  11. GCMC for Policy Search - Details INNOVATION + ASSISTANCE 11

  12. GCMC for Policy Search INNOVATION + ASSISTANCE 12

  13. GCMC for Policy Search INNOVATION + ASSISTANCE 13

  14. GCMC for Policy Search Towards smooth relaxations INNOVATION + ASSISTANCE Gaussian smoothings 14

  15. GCMC for Policy Search Towards smooth relaxations INNOVATION + ASSISTANCE Gaussian smoothing gradient 15

  16. GCMC for Policy Search Towards smooth relaxations INNOVATION + ASSISTANCE Gaussian smoothing gradient 16

  17. Coupled antithetic pairs for Monte Carlo gradient estimation Baseline gradient estimator with antithetic pairs (Salimans et al. 2017): INNOVATION + ASSISTANCE 17

  18. Coupled antithetic pairs for Monte Carlo gradient estimation Baseline gradient estimator with antithetic pairs (Salimans et al. 2017): INNOVATION + ASSISTANCE 18

  19. Coupled antithetic pairs for Monte Carlo gradient estimation Baseline gradient estimator with antithetic pairs (Salimans et al. 2017): Antithetic inverse lengths coupling estimator (Rowland, Choromanski et al. 2018): INNOVATION + ASSISTANCE 19

  20. Coupled antithetic pairs for Monte Carlo gradient estimation Baseline gradient estimator with antithetic pairs (Salimans et al. 2017): Antithetic inverse lengths coupling estimator (Rowland, Choromanski et al. 2018): INNOVATION + ASSISTANCE coupled lengths coupled lengths 20

  21. Coupled antithetic pairs for Monte Carlo gradient estimation Baseline gradient estimator with antithetic pairs (Salimans et al. 2017): Antithetic inverse lengths coupling estimator (Rowland, Choromanski et al. 2018): INNOVATION + ASSISTANCE coupled lengths 21

  22. Experimental results: Minitaur Learning How to Walk with antithetic coupled samples + linear policies N=8 N=16 N=48 INNOVATION + ASSISTANCE 22 N=54 N=64 N=96

  23. Thank you !!! 23 INNOVATION + ASSISTANCE

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

The Monte Carlo Method Estimating through sampling (estimating , p -value, integrals,...)

The Monte Carlo Method Estimating through sampling (estimating , p -value, integrals,...)

The Monte Carlo Method Estimating through sampling (estimating , p -value, integrals,...) The main difficulty - sampling sparse events The general sampling to counting reduction The Markov Chain Monte Carlo (MCMC) method -

1.39k views • 73 slides
Monte Carlo Integration Monte Carlo methods use random numbers for sampling Although

Monte Carlo Integration Monte Carlo methods use random numbers for sampling Although

Monte Carlo Integration Monte Carlo methods use random numbers for sampling Although somewhat less glamorous than gambling devices (dice, roulette, cards, etc.) random number generators on the computer are more efficient (>10 8 random

536 views • 11 slides
The Coupled Electron-Ion Monte Carlo study of hydrogen under extreme conditions Carlo Pierleoni

The Coupled Electron-Ion Monte Carlo study of hydrogen under extreme conditions Carlo Pierleoni

The Coupled Electron-Ion Monte Carlo study of hydrogen under extreme conditions Carlo Pierleoni Maison de la Simulation, CEA-Saclay Paris, France and Dept. of Physical and Chemical Sciences University of LAquila, Italy OUTLINE

690 views • 46 slides
Linear Regression: Prior Linear Regression: Posterior 4 4 2 2 0 0 2 2 Prior P ( )

Linear Regression: Prior Linear Regression: Posterior 4 4 2 2 0 0 2 2 Prior P ( )

Monte Carlo Monte Carlo and Insomnia Enrico Fermi (19011954) took great delight in astonishing his colleagues with his remarkably accurate Monte Carlo basics and Motivation predictions of experimental Rejection sampling results. .

343 views • 7 slides
Monte Carlo Estimation 7 January 2019 OSU CSE 1 Monte Carlo Methods Class of computational

Monte Carlo Estimation 7 January 2019 OSU CSE 1 Monte Carlo Methods Class of computational

Monte Carlo Estimation 7 January 2019 OSU CSE 1 Monte Carlo Methods Class of computational methods that use random sampling to estimate results Named after the famous Monte Carlo Casino 7 January 2019 OSU CSE 2 Throwing Darts 3 5

391 views • 12 slides
Sampling Methods CMSC 678 UMBC Outline Recap Monte Carlo methods Sampling Techniques Uniform

Sampling Methods CMSC 678 UMBC Outline Recap Monte Carlo methods Sampling Techniques Uniform

Approximate Inference: Sampling Methods CMSC 678 UMBC Outline Recap Monte Carlo methods Sampling Techniques Uniform sampling Importance Sampling Rejection Sampling Metropolis-Hastings Gibbs sampling Example: Collapsed Gibbs Sampler for

1.03k views • 71 slides
Monte Carlo simulations of the 2D Ising model Stochastic sampling of spin configurations to

Monte Carlo simulations of the 2D Ising model Stochastic sampling of spin configurations to

Monte Carlo simulations of the 2D Ising model Stochastic sampling of spin configurations to estimate Spin configurations configurations; can sample very small fraction for large N Trivial Monte Carlo sampling fails at low T, because the sum is

332 views • 15 slides
Chapter 11: Sampling Methods Lei Tang Department of CSE Arizona State University Dec. 18th,

Chapter 11: Sampling Methods Lei Tang Department of CSE Arizona State University Dec. 18th,

Chapter 11: Sampling Methods Lei Tang Department of CSE Arizona State University Dec. 18th, 2007 1 / 37 Outline 1 Introduction 2 Basic Sampling Algorithms 3 Markov Chain Monte Carlo (MCMC) 4 Gibbs Sampling 5 Slice Sampling 6 Hybrid Monte Carlo

881 views • 52 slides
Chapter 11: Sampling Methods Lei Tang Department of CSE Arizona State University Dec. 18th,

Chapter 11: Sampling Methods Lei Tang Department of CSE Arizona State University Dec. 18th,

Chapter 11: Sampling Methods Lei Tang Department of CSE Arizona State University Dec. 18th, 2007 1 / 35 Outline 1 Introduction 2 Basic Sampling Algorithms 3 Markov Chain Monte Carlo (MCMC) 4 Gibbs Sampling 5 Slice Sampling 6 Hybrid Monte Carlo

1.01k views • 85 slides
Monte Carlo Control CMPUT 366: Intelligent Systems S&B 5.3-5.5, 5.7 Lecture Outline 1.

Monte Carlo Control CMPUT 366: Intelligent Systems S&B 5.3-5.5, 5.7 Lecture Outline 1.

Monte Carlo Control CMPUT 366: Intelligent Systems S&B 5.3-5.5, 5.7 Lecture Outline 1. Recap 2. Estimating Action Values 3. Monte Carlo Control 4. Importance Sampling 5. Off-Policy Monte Carlo Control Recap: Monte Carlo vs.

263 views • 22 slides
Monte Carlo methods for sampling-based Stochastic Optimization Gersende FORT LTCI CNRS &

Monte Carlo methods for sampling-based Stochastic Optimization Gersende FORT LTCI CNRS &

Monte Carlo methods for sampling-based Stochastic Optimization Monte Carlo methods for sampling-based Stochastic Optimization Gersende FORT LTCI CNRS & Telecom ParisTech Paris, France Joint works with B. Jourdain, T. Leli` evre, G.

608 views • 36 slides
Monte Carlo Simulations and PcNaive Heino Bohn Nielsen 1 of 21 Monte Carlo Simulations MC

Monte Carlo Simulations and PcNaive Heino Bohn Nielsen 1 of 21 Monte Carlo Simulations MC

Econometrics 2 Fall 2005 Monte Carlo Simulations and PcNaive Heino Bohn Nielsen 1 of 21 Monte Carlo Simulations MC simulations were introduced in Econometrics 1. Formalizing the thought experiment underlying the data sampling. In

348 views • 11 slides
Convergence Analysis for Anisotropic Monte Carlo Sampling Spectra Gurprit Singh

Convergence Analysis for Anisotropic Monte Carlo Sampling Spectra Gurprit Singh

Convergence Analysis for Anisotropic Monte Carlo Sampling Spectra Gurprit Singh Wojciech Jarosz Thank you for being here. Today, I will be presenting our work on convergence analysis for Monte Carlo integration that is done in

2.26k views • 172 slides
Lecture 5: Sampling (Monte Carlo Methods) Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel

Lecture 5: Sampling (Monte Carlo Methods) Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel

CS598JHM: Advanced NLP (Spring 2013) http://courses.engr.illinois.edu/cs598jhm/ Lecture 5: Sampling (Monte Carlo Methods) Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Office hours: by appointment The goal of Monte Carlo

413 views • 17 slides
Monte Carlo Approximation of Monte Carlo Filters Adam M. Johansen et al. Collaborators Include:

Monte Carlo Approximation of Monte Carlo Filters Adam M. Johansen et al. Collaborators Include:

Monte Carlo Approximation of Monte Carlo Filters Adam M. Johansen et al. Collaborators Include: Arnaud Doucet, Axel Finke, Anthony Lee, Nick Whiteley 7th January 2014 Introduction Monte Carlo Approximationof Monte Carlo Filters Approximating

1.46k views • 31 slides
Sampling Methods Oliver Schulte - CMPT 419/726 Bishop PRML Ch. 11 Sampling Rejection Sampling

Sampling Methods Oliver Schulte - CMPT 419/726 Bishop PRML Ch. 11 Sampling Rejection Sampling

Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo Sampling Methods Oliver Schulte - CMPT 419/726 Bishop PRML Ch. 11 Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo Recall Inference For

573 views • 41 slides
Thompson Sampling based Monte-Carlo Planning in POMDPs Aijun Bai 1 Feng Wu 2 Zongzhang Zhang 3

Thompson Sampling based Monte-Carlo Planning in POMDPs Aijun Bai 1 Feng Wu 2 Zongzhang Zhang 3

The 24th International Conference on Automated Planning and Scheduling, ICAPS 2014 Thompson Sampling based Monte-Carlo Planning in POMDPs Aijun Bai 1 Feng Wu 2 Zongzhang Zhang 3 Xiaoping Chen 1 1 University of Science & Technology of China 2

475 views • 22 slides
Conformal Supergravity, 4D Scattering Equations (and Monte Carlo Methods) Joe Farrow Based on

Conformal Supergravity, 4D Scattering Equations (and Monte Carlo Methods) Joe Farrow Based on

Conformal Supergravity, 4D Scattering Equations (and Monte Carlo Methods) Joe Farrow Based on Farrow, A Monte Carlo Approach to the 4D Scattering Equations, 1806.02732 Farrow & Lipstein, New Worldsheet Formulae for Conformal

740 views • 37 slides
Monte Carlo Methods CS60077: Reinforcement Learning Abir Das IIT Kharagpur Oct 05 and 06, 2020

Monte Carlo Methods CS60077: Reinforcement Learning Abir Das IIT Kharagpur Oct 05 and 06, 2020

Monte Carlo Methods CS60077: Reinforcement Learning Abir Das IIT Kharagpur Oct 05 and 06, 2020 Agenda Introduction MC Evaluation MC Control Agenda Understand how to evaluate policies in model-free setting using Monte Carlo methods

762 views • 53 slides
Sequential Monte Carlo Methods Particle Filter Martin Ulmke Head of Research Group

Sequential Monte Carlo Methods Particle Filter Martin Ulmke Head of Research Group

Sequential Monte Carlo Methods Particle Filter Martin Ulmke Head of Research Group Distributed Sensor Systems Sensor Data and Information Fusion Fraunhofer FKIE Lecture: Introduction to Sensor Data Fusion 12 December 2018 Inhalt

1.27k views • 74 slides
RSA Parameter Generation Bob needs to: - find 2 large primes p,q - find e s.t. gcd(e, (pq))=1

RSA Parameter Generation Bob needs to: - find 2 large primes p,q - find e s.t. gcd(e, (pq))=1

RSA Parameter Generation Bob needs to: - find 2 large primes p,q - find e s.t. gcd(e, (pq))=1 Good news: - primes are fairly common: there are about N/ln N primes N Exercise: If looking for a 512-bit prime, how many randomly generated

358 views • 10 slides
Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk Axel Gandy

Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk Axel Gandy

Sequential Implementation of Monte Carlo Tests with Uniformly Bounded Resampling Risk Axel Gandy Department of Mathematics Imperial College London a.gandy@imperial.ac.uk useR! 2009, Rennes July 8-10, 2009 Introduction Test statistic T ,

624 views • 40 slides
RAL Report for DUNE-UK meeting Fergus Wilson Chris Brew Raja Nandakumar 1 Workload management

RAL Report for DUNE-UK meeting Fergus Wilson Chris Brew Raja Nandakumar 1 Workload management

RAL Report for DUNE-UK meeting Fergus Wilson Chris Brew Raja Nandakumar 1 Workload management system (WMS) DUNE currently uses fife / poms as WMS Aim : Test out DIRAC as an alternative WMS for DUNE For now, only proof-of-concept

294 views • 6 slides
Introduction to Mobile Robotics Bayes Filter Particle Filter and Monte Carlo Localization

Introduction to Mobile Robotics Bayes Filter Particle Filter and Monte Carlo Localization

Introduction to Mobile Robotics Bayes Filter Particle Filter and Monte Carlo Localization Wolfram Burgard, Cyrill Stachniss, Maren Bennewitz, Kai Arras 1 Motivation Recall: Discrete filter Discretize the continuous state space

1.2k views • 61 slides

More recommend