Unifying Orthogonal Monte Carlo Methods From Kacs Random Walks To - PowerPoint PPT Presentation

Unifying Orthogonal Monte Carlo Methods From Kac’s Random Walks To Hadamard Multi Rademachers Krzysztof Choromanski, Mark Rowland Wenyu Chen, Adrian Weller

The Phenomenon of Orthogonal Monte Carlo Estimators Estimation task: Applications: ● dimensionality reduction (JLT-mechanisms) isotropic distribution ● scaling kernel methods (e.g. Gaussian) (random feature maps) ● hashing algorithms (e.g. LSH) ● (sliced) Wasserstein Standard MC approach: distances (WGANs, autoencoders...) ● reinforcement learning (ES algorithms) ● and many, many more...

The Phenomenon of Orthogonal Monte Carlo Estimators Sampling from the Estimation task: Haar measure on the O(d) group isotropic distribution (e.g. Gaussian) Expensive: O(n^3 time) The Orthogonal Trick: guarantees unbiasedness # of samples of the often implies better MC estimator <= dim accuracy

Towards Computational Efficiency: The Zoo of Approximate MCs

Towards Computational Efficiency: The Zoo of Approximate MCs ...

Towards Computational Efficiency: The Zoo of Approximate MCs ... ...

Towards Computational Efficiency: The Zoo of Approximate MCs ... size size N x N N/2 x N/2 ... Constraints: ● ●

On the Hunt for the Unifying Theory: The World of Givens Reflections and Rotations Kac’s random walk matrices Givens rotations Hadamard-Rademacher Chains Givens reflections reflection in the jth coordinate

On the Hunt for the Unifying Theory: The World of Givens Reflections and Rotations Kac’s random walk matrices Hadamard-Rademacher Chains

On the Hunt for the Unifying Theory: The World of Givens Reflections and Rotations Hadamard-MultiRademachers Butterfly Matrices

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime Still more accurate estimator than unstructured MC baseline

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime Log-Linear Time Complexity (unstructured MC baseline has quadratic)

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime Analysis of the Total Variation Distance between Haar measure on d-sphere and measure induced by standard Kac’s random walk on d-sphere estimator estimated value Pillai, Smith 2016 Kac’s random walk on d-sphere mixes in O(d log d) steps

First Theoretical Results for Free-Lunch Phenomenon in the Nonlinear Regime Analysis of the Total Variation Distance between Haar measure on d-sphere and measure induced by standard Kac’s random walk on d-sphere estimator estimated value Pillai, Smith 2016 Kac’s random walk on d-sphere mixes in O(d log d) steps More careful analysis of the LHS

Maximum Mean Discrepancy Experiment How Does It Work In Practice ? Kernel Approximation via Random Features Reinforcement Learning via ES-methods Accuracy Computational Efficiency

Thank you for your attention !

Unifying Orthogonal Monte Carlo Methods From Kacs Random Walks To - PowerPoint PPT Presentation

Unifying Orthogonal Monte Carlo Methods From Kacs Random Walks To Hadamard Multi Rademachers Krzysztof Choromanski, Mark Rowland Wenyu Chen, Adrian Weller The Phenomenon of Orthogonal Monte Carlo Estimators Estimation task: Applications:

Monte Carlo Methods An introduction to Monte Carlo (MC) methods How to use MC methods

Chapter 5: Monte Carlo Methods Monte Carlo methods are learning methods Experience

Eligibility Traces Unifying Monte Carlo and TD key algorithms: TD( ), Sarsa( ), Q( )

Monte Carlo Methods Guojin Chen Christopher Cprek Chris Rambicure Monte Carlo Methods 1.

Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods

Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods

Monte Carlo Estimation 7 January 2019 OSU CSE 1 Monte Carlo Methods Class of computational

Monte Carlo Methods and Area Estimates CS3220 - Summer 2008 Jonathan Kaldor Monte Carlo Methods

Monte Carlo Integration Monte Carlo methods use random numbers for sampling Although

Monte Carlo Methods Prof. Kuan-Ting Lai 2020/4/17 Monte Carlo Methods Learn directly from

Monte Carlo Methods Why Monte Carlo? Computation of number ( 1) n +1 = 1 1 2 + 1

Course on Inverse Problems Albert Tarantola Lesson VIII: Monte Carlo Methods Monte Carlo

An introduction to Monte Carlo methods in XRF analysis Tom Schoonjans Joint ICTP-IAEA school,

Planning and Optimization G4. Asymptotically Suboptimal Monte-Carlo Methods Gabriele R oger

Monte Carlo methods for magnetic systems Zoltn Nda Babe-Bolyai University Dept of

Kinetic Monte Carlo Methods C n e n o t i t e a r t f u o p r CC DC Kinetic

(machine) learning jet substructure Machine Learning for Jet Physics Workshop, 2017 Eric M.

Expanding the Zoo We have snakes and armadillos. Let's add ants. An ant has a weight a

Merging time series MAN IP U L ATIN G TIME SE R IE S DATA W ITH XTS AN D ZOO IN R Je re y

New Directions in Randomness Jason Rute Pennsylvania State University Computability, Complexity,

Galaxy Zoo Challenge CLASSIFYING THE MORPHOLOGIES OF DISTANT GALAXIES IN OUR UNIVERSE Is the

Automatic Identification of Locative Expressions from Social Media Text: A Comparative Analysis

Finding Predictors: Nearest Neighbor Modern Motivations: Be Lazy! Classification Regression

Weihrauch degrees of numerical problems comparison with arithmetic Keita Yokoyama joint