Geometrically Coupled Monte Carlo Sampling Mark Rowland Krzysztof - - PowerPoint PPT Presentation

▶
geometrically coupled monte carlo sampling
SMART_READER_LITE
LIVE PREVIEW

Geometrically Coupled Monte Carlo Sampling Mark Rowland Krzysztof - - PowerPoint PPT Presentation

Aug 19, 2022 43 likes •277 views

Geometrically Coupled Monte Carlo Sampling Mark Rowland Krzysztof Choromanski Franois Chalus Aldo Pacchiano Tamas Sarlos Richard E. Turner Adrian Weller Geometrically Coupled Monte Carlo Sampling Central goal: Unbiased Monte Carlo

slide-1
SLIDE 1

Mark Rowland Krzysztof Choromanski François Chalus Aldo Pacchiano Tamas Sarlos Richard E. Turner Adrian Weller

Geometrically Coupled Monte Carlo Sampling

slide-2
SLIDE 2

INNOVATION + ASSISTANCE 2

Geometrically Coupled Monte Carlo Sampling

Central goal: Can we do better than i.i.d.? Key contribution: K-optimality. Optimise the objective below over the joint distribution of Unbiased Monte Carlo estimation:

This leads to a multi-marginal transport problem, which is often analytically solvable.

slide-3
SLIDE 3

INNOVATION + ASSISTANCE 3

GCMC in Robotics - Policy Search - An Overview

isotropic distribution

slide-4
SLIDE 4

INNOVATION + ASSISTANCE 4

isotropic distribution

GCMC in Robotics - Policy Search - An Overview

slide-5
SLIDE 5

INNOVATION + ASSISTANCE 5

isotropic distribution

GCMC in Robotics - Policy Search - An Overview

slide-6
SLIDE 6

INNOVATION + ASSISTANCE 6

isotropic distribution

GCMC in Robotics - Policy Search - An Overview

slide-7
SLIDE 7

INNOVATION + ASSISTANCE 7

isotropic distribution antithetic pair

GCMC in Robotics - Policy Search - An Overview

slide-8
SLIDE 8

INNOVATION + ASSISTANCE 8

isotropic distribution antithetic pair

  • Independent Antithetic Pairs
  • Coupled Samples of Equal Lengths

Typical approach to Monte Carlo Sampling:

GCMC in Robotics - Policy Search - An Overview

slide-9
SLIDE 9

INNOVATION + ASSISTANCE 9

isotropic distribution antithetic pair

  • Independent Antithetic Pairs
  • Coupled Samples of Equal Lengths

Typical approach to Monte Carlo Sampling:

GCMC in Robotics - Policy Search - An Overview

slide-10
SLIDE 10

INNOVATION + ASSISTANCE 10

isotropic distribution antithetic pair

  • Independent Antithetic Pairs
  • Coupled Samples of Equal Lengths

Typical approach to Monte Carlo Sampling:

GCMC:

  • rthogonal directions
  • f different antithetic pairs
  • correlated unequal lengths

within a pair

  • variance reduction

GCMC in Robotics - Policy Search - An Overview

slide-11
SLIDE 11

INNOVATION + ASSISTANCE 11

GCMC for Policy Search - Details

slide-12
SLIDE 12

INNOVATION + ASSISTANCE 12

GCMC for Policy Search

slide-13
SLIDE 13

INNOVATION + ASSISTANCE 13

GCMC for Policy Search

slide-14
SLIDE 14

INNOVATION + ASSISTANCE 14

Towards smooth relaxations Gaussian smoothings

GCMC for Policy Search

slide-15
SLIDE 15

INNOVATION + ASSISTANCE 15

Towards smooth relaxations Gaussian smoothing gradient

GCMC for Policy Search

slide-16
SLIDE 16

INNOVATION + ASSISTANCE 16

Towards smooth relaxations Gaussian smoothing gradient

GCMC for Policy Search

slide-17
SLIDE 17

INNOVATION + ASSISTANCE 17

Baseline gradient estimator with antithetic pairs (Salimans et al. 2017):

Coupled antithetic pairs for Monte Carlo gradient estimation

slide-18
SLIDE 18

INNOVATION + ASSISTANCE 18

Baseline gradient estimator with antithetic pairs (Salimans et al. 2017):

Coupled antithetic pairs for Monte Carlo gradient estimation

slide-19
SLIDE 19

INNOVATION + ASSISTANCE 19

Baseline gradient estimator with antithetic pairs (Salimans et al. 2017): Antithetic inverse lengths coupling estimator (Rowland, Choromanski et al. 2018):

Coupled antithetic pairs for Monte Carlo gradient estimation

slide-20
SLIDE 20

INNOVATION + ASSISTANCE 20

Baseline gradient estimator with antithetic pairs (Salimans et al. 2017): Antithetic inverse lengths coupling estimator (Rowland, Choromanski et al. 2018):

coupled lengths coupled lengths

Coupled antithetic pairs for Monte Carlo gradient estimation

slide-21
SLIDE 21

INNOVATION + ASSISTANCE 21

Baseline gradient estimator with antithetic pairs (Salimans et al. 2017): Antithetic inverse lengths coupling estimator (Rowland, Choromanski et al. 2018):

coupled lengths

Coupled antithetic pairs for Monte Carlo gradient estimation

slide-22
SLIDE 22

INNOVATION + ASSISTANCE 22

Experimental results: Minitaur Learning How to Walk with antithetic coupled samples + linear policies

N=8 N=16 N=48 N=54 N=64 N=96

slide-23
SLIDE 23

INNOVATION + ASSISTANCE 23

Thank you !!!