PAC Identification of Many Good Arms in Stochastic Multi-Armed - PowerPoint PPT Presentation

PAC Identification of Many Good Arms in Stochastic Multi-Armed Bandits Arghya Roy Chaudhuri under the guidance of Prof. Shivaram Kalyanakrishnan Indian Institute of Technology Bombay, India 1 / 8

What Is It All About? 2 / 8

What Is It All About? 3 / 8

What Is a Multi-Armed Bandit? 1.0 0.9 0.5 0.5 0.2 0.0 Mean (Unknown) Bandits: Slot machines Mean: Pr[Reward = 1] 4 / 8

What Is a Multi-Armed Bandit? To identify the best arm: � n 1.0 ǫ 2 log 1 � E [SC] = Ω 0.9 δ To identify the best subset of size 0.5 0.5 m : � n ǫ 2 log m � E [SC] = Ω 0.2 δ 0.0 Mean (Unknown) Bandits: Slot machines Mean: Pr[Reward = 1] 4 / 8

What Is a Multi-Armed Bandit? To identify the best arm: � n 1.0 ǫ 2 log 1 � E [SC] = Ω 0.9 δ To identify the best subset of size 0.5 0.5 m : � n ǫ 2 log m � E [SC] = Ω 0.2 δ 0.0 Mean (Unknown) We need an alternative. Bandits: Slot machines Mean: Pr[Reward = 1] 4 / 8

Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . 5 / 8

Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. 5 / 8

Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. Redefine the problem to identify 1 from the best m arms. Defining ρ = m n , generalise the problem. What if we n is relatively small? 5 / 8

Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. 6 / 8

Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. k = 1 : Any 1 arm out of the best subset of size m . 6 / 8

Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. k = m : Best subset identification. 6 / 8

Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. k = m = 1 : Best arm identification. 6 / 8

Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. k = 1 : Any 1 arm out of the best subset of size m . k = m : Best subset identification. k = m = 1 : Best arm identification. Contributions: LUCB -k-m (Fully sequential + Adaptive). Worst case upper and lower bound. 6 / 8

Infinite-Armed Bandit Instances ( k , ρ ): To identify any distinct k arms from the best ρ fraction of arms. 7 / 8

Thank You! Poster: #54 Email: arghya@cse.iitb.ac.in 8 / 8

PAC Identification of Many Good Arms in Stochastic Multi-Armed - PowerPoint PPT Presentation

PAC Identification of Many Good Arms in Stochastic Multi-Armed Bandits Arghya Roy Chaudhuri under the guidance of Prof. Shivaram Kalyanakrishnan Indian Institute of Technology Bombay, India 1 / 8 What Is It All About? 2 / 8 What Is It All

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I S ebastien

Multi-Armed Bandits: Non-adaptive and Adaptive Sampling Instructor: Sham Kakade 1 The

On parameter identification in linear stochastic differential equations by Gaussian statistics

Multi-level stochastic approximation algorithms Noufel Frikha Universit e Paris Diderot, LPMA

Multi-Index Monte Carlo and Multi-Index Stochastic Collocation Ra ul Tempone Alexander von

A parameter identification problem in stochastic homogenization William Minvielle CERMICS,

Upper confidence bound strategy on stochastical bandits Multiarmed bandit: K arms, at each step we

SHARK-FV 2014 The MOOD Ideas MOOD for multi-mat. flows The good The not-yet-good-enough

Large Deviations for Multi-valued Stochastic Differential Equations Large Deviations for

Sergeant at Arms Training Session www.toastmasters.org Sergeant at Arms TRAINING DISTRICT 37

CS 5150 So(ware Engineering 2. Steps in the so(ware development process William Y. Arms So(ware

Multi-resolution Inference of Stochastic Models from Partially Observed Data Samuel Kou

A Calculus for Stochastic Interventions: Causal Effect Identification and Surrogate Experiments

Multi-level Stochastic Local Search for SAT Camilo Rostoker and Chris Dabrowski

RIG THE PERFECT ANGLE EVERY TIME Two picking arms each with five adjustment holes make

Identification Multi-Echelles par Ondelettes Continues de la Signature des Etats de Surface H.

M 3 : INTEGRATING ARBITRARY COMPUTE UNITS AS FIRST-CLASS CITIZENS OS: Nils Asmussen, Hermann H

ARM memory generator Arm Memory generator Make sure you create a folder similar to what you

ARM EDITION Matt Spisak REcon 2016, Montreal RECON 2016 ABOUT Offense-based approach to

Probabilis)c Reasoning for Assembly-Based 3D Modeling

NEVE: Nested Virtualization Extensions for ARM Jin Tack Lim, Christo ff er Dall, Shih-Wei Li, Jason

Extending the swsusp Hibernation Framework to ARM Russell Dill 1 2 Introduction Russ Dill

HIGHLEVELMANIPULATION PRIMITIVESFORAROBOTARM Supported by National

ARM Exception Handling CS2253 Owen Kaser, UNBSJ Overview Warning: hardest parts of CS2253.