one armed bandit
play

One Armed Bandit source: http://dogbeforewicket.blogspot.ca EECS - PowerPoint PPT Presentation

Dec 31, 2022 •43 likes •623 views

One Armed Bandit source: http://dogbeforewicket.blogspot.ca EECS 1030 moodle.yorku.ca One Armed Bandit Utility /** * Returns the winnings from one pull of the one armed * bandit. * * @param coin the coin deposited in the one armed bandit.

  1. One Armed Bandit source: http://dogbeforewicket.blogspot.ca EECS 1030 moodle.yorku.ca

  2. One Armed Bandit Utility /** * Returns the winnings from one pull of the one armed * bandit. * * @param coin the coin deposited in the one armed bandit. * @return the payoff from one pull of the lever. */ public static List<Coin> pull(Coin coin) EECS 1030 moodle.yorku.ca

  3. Casino App List<Coin> wallet = new ArrayList<Coin>(); wallet.add(new Coin()); final int TRIES = 10; int tries = 0; while (tries < TRIES && !wallet.isEmpty()) { Coin coin = wallet.remove(0); List<Coin> winnings = OneArmedBandit.pull(coin); wallet.addAll(winnings); tries++; } System.out.printf("After %d tries, %d coins left%n", tries, wallet.size()); EECS 1030 moodle.yorku.ca

  4. The Coin Class Problem Implement the Coin class. We only need to be able to create Coin objects, using the default constructor. We do not need any methods of the Coin class. EECS 1030 moodle.yorku.ca

  5. The Coin Class Problem Implement the Coin class. We only need to be able to create Coin objects, using the default constructor. We do not need any methods of the Coin class. Now that we have the Coin class, we can run the Casino app. EECS 1030 moodle.yorku.ca

  6. How Many Coins? Question You win 37 coins in 10 tries. How many Coin objects are there stored in memory? EECS 1030 moodle.yorku.ca

  7. How Many Coins? Question You win 37 coins in 10 tries. How many Coin objects are there stored in memory? Answer At least 37. EECS 1030 moodle.yorku.ca

  8. How Many Coins? Question You win 37 coins in 10 tries. How many Coin objects are there stored in memory? Answer At least 37. Question Instead of storing Coin objects in your wallet, could you store aliases of a single Coin object in your wallet? EECS 1030 moodle.yorku.ca

  9. How Many Coins? Question You win 37 coins in 10 tries. How many Coin objects are there stored in memory? Answer At least 37. Question Instead of storing Coin objects in your wallet, could you store aliases of a single Coin object in your wallet? Answer Yes. EECS 1030 moodle.yorku.ca

  10. A Single Coin Problem Modify the Coin class so that the client can create at most one Coin object. EECS 1030 moodle.yorku.ca

  11. A Single Coin Question Can the client use a constructor to create the Coin object? EECS 1030 moodle.yorku.ca

  12. A Single Coin Question Can the client use a constructor to create the Coin object? Answer No. EECS 1030 moodle.yorku.ca

  13. A Single Coin Question Can the client use a constructor to create the Coin object? Answer No. Question Why not? EECS 1030 moodle.yorku.ca

  14. A Single Coin Question Can the client use a constructor to create the Coin object? Answer No. Question Why not? Answer If we provide a public constructor, clients can invoke it as many times as they want and, hence, create as many Coin objects as they want. EECS 1030 moodle.yorku.ca

  15. A Single Coin Question Since the client cannot use a constructor, what other options does the client have? EECS 1030 moodle.yorku.ca

  16. A Single Coin Question Since the client cannot use a constructor, what other options does the client have? Answer Methods. EECS 1030 moodle.yorku.ca

  17. A Single Coin Question Since the client cannot use a constructor, what other options does the client have? Answer Methods. Question Can the method be non-static? EECS 1030 moodle.yorku.ca

  18. A Single Coin Question Since the client cannot use a constructor, what other options does the client have? Answer Methods. Question Can the method be non-static? Answer No, because you would need a Coin object to invoke it on (and we are trying to create a Coin object). So the method is static. EECS 1030 moodle.yorku.ca

  19. A Single Coin Question What is the return type of this static method? EECS 1030 moodle.yorku.ca

  20. A Single Coin Question What is the return type of this static method? Answer Coin . EECS 1030 moodle.yorku.ca

  21. A Single Coin Question What is the return type of this static method? Answer Coin . Question Does it have any parameters? EECS 1030 moodle.yorku.ca

  22. A Single Coin Question What is the return type of this static method? Answer Coin . Question Does it have any parameters? Answer No. EECS 1030 moodle.yorku.ca

  23. A Single Coin public static Coin getInstance() Question The method name suggests that the Coin class has a static attribute. What is its name and type? EECS 1030 moodle.yorku.ca

  24. A Single Coin public static Coin getInstance() Question The method name suggests that the Coin class has a static attribute. What is its name and type? Answer instance and Coin . EECS 1030 moodle.yorku.ca

  25. A Single Coin Question Where do we initialize the attribute? EECS 1030 moodle.yorku.ca

  26. A Single Coin Question Where do we initialize the attribute? Answer In the declaration. EECS 1030 moodle.yorku.ca

  27. A Single Coin Question Where do we initialize the attribute? Answer In the declaration. Question How? EECS 1030 moodle.yorku.ca

  28. A Single Coin Question Where do we initialize the attribute? Answer In the declaration. Question How? Answer private static Coin instance = new Coin(); EECS 1030 moodle.yorku.ca

  29. A Single Coin Question But as we argued earlier, we cannot provide a public constructor. So, how can we use new Coin() in the Coin class? EECS 1030 moodle.yorku.ca

  30. A Single Coin Question But as we argued earlier, we cannot provide a public constructor. So, how can we use new Coin() in the Coin class? Answer By adding a private default constructor. EECS 1030 moodle.yorku.ca

  31. A Single Coin Question But as we argued earlier, we cannot provide a public constructor. So, how can we use new Coin() in the Coin class? Answer By adding a private default constructor. Question Now that we have declared and initialized the instance attribute, how do we implement the getInstance method? EECS 1030 moodle.yorku.ca

  32. A Single Coin Question But as we argued earlier, we cannot provide a public constructor. So, how can we use new Coin() in the Coin class? Answer By adding a private default constructor. Question Now that we have declared and initialized the instance attribute, how do we implement the getInstance method? Answer return Coin.instance; EECS 1030 moodle.yorku.ca

  33. Singleton Design Pattern The pattern to ensure that at most one object of a particular class can be created is known as the singleton design pattern. The example we presented is contrived. In case object represent physical entities, such as a connection to a database, the singleton design pattern comes in handy. EECS 1030 moodle.yorku.ca

  34. Coin with Values Problem Modify the Coin class some that each coin has a value (of type int). Make the class immutable. EECS 1030 moodle.yorku.ca

  35. Coin with Values Problem Modify the Coin class so that the client can create at most one Coin object for each value. That is, the client can create different Coin objects but only if they all have different values. EECS 1030 moodle.yorku.ca

  36. Coin with Values Question Can the client use a constructor to create the Coin object? EECS 1030 moodle.yorku.ca

  37. Coin with Values Question Can the client use a constructor to create the Coin object? Answer No. EECS 1030 moodle.yorku.ca

  38. Coin with Values Question Can the client use a constructor to create the Coin object? Answer No. Question Why not? EECS 1030 moodle.yorku.ca

  39. Coin with Values Question Can the client use a constructor to create the Coin object? Answer No. Question Why not? Answer If we provide a public constructor, clients can invoke it as many times as they want and, hence, create as many Coin objects with the same value as they want. EECS 1030 moodle.yorku.ca

  40. Coin with Values Question Since the client cannot use a constructor, what other options does the client have? EECS 1030 moodle.yorku.ca

  41. Coin with Values Question Since the client cannot use a constructor, what other options does the client have? Answer Methods. EECS 1030 moodle.yorku.ca

  42. Coin with Values Question Since the client cannot use a constructor, what other options does the client have? Answer Methods. Question Can the method be non-static? EECS 1030 moodle.yorku.ca

  43. Coin with Values Question Since the client cannot use a constructor, what other options does the client have? Answer Methods. Question Can the method be non-static? Answer No, because you would need a Coin object to invoke it on (and we are trying to create a Coin object). So the method is static. EECS 1030 moodle.yorku.ca

  44. Coin with Values Question What is the return type of this static method? EECS 1030 moodle.yorku.ca

  45. Coin with Values Question What is the return type of this static method? Answer Coin . EECS 1030 moodle.yorku.ca

  46. Coin with Values Question What is the return type of this static method? Answer Coin . Question Does it have any parameters? EECS 1030 moodle.yorku.ca

  47. Coin with Values Question What is the return type of this static method? Answer Coin . Question Does it have any parameters? Answer Yes, the value of the coin. EECS 1030 moodle.yorku.ca

  48. Coin with Values public static Coin getInstance(int value) EECS 1030 moodle.yorku.ca

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Reinforcement Learning n-armed bandit Kevin Spiteri April 21, 2015 n-armed bandit n-armed

Reinforcement Learning n-armed bandit Kevin Spiteri April 21, 2015 n-armed bandit n-armed

Reinforcement Learning n-armed bandit Kevin Spiteri April 21, 2015 n-armed bandit n-armed bandit 0.9 0.5 0.1 0.9 0.5 0.1 0.0 0.0 0.0 0.0 estimate n-armed bandit n-armed bandit 0.9 0.5 0.1 0.9 0.5 0.1 0 0.0 0.0 0.0 0.0

676 views • 21 slides
Multi-armed Bandits Prof. Kuan-Ting Lai 2020/3/12 k-armed Bandit Problem Playing k armed

Multi-armed Bandits Prof. Kuan-Ting Lai 2020/3/12 k-armed Bandit Problem Playing k armed

Multi-armed Bandits Prof. Kuan-Ting Lai 2020/3/12 k-armed Bandit Problem Playing k armed bandit machines and find a way to win most money! Note: assume you have unlimited money and never go bankrupt!

342 views • 21 slides
Reinforcement Learning Kevin Spiteri April 21, 2015 n-armed bandit n-armed bandit 0.9 0.5

Reinforcement Learning Kevin Spiteri April 21, 2015 n-armed bandit n-armed bandit 0.9 0.5

Reinforcement Learning Kevin Spiteri April 21, 2015 n-armed bandit n-armed bandit 0.9 0.5 0.1 n-armed bandit 0.9 0.5 0.1 0.0 0.0 0.0 0.0 estimate n-armed bandit 0.9 0.5 0.1 0 0.0 0.0 0.0 0.0 estimate 0 0 0 0.0 0

994 views • 84 slides
On the Complexity of Best Arm Identification in Multi-Armed Bandit Models Aurlien Garivier

On the Complexity of Best Arm Identification in Multi-Armed Bandit Models Aurlien Garivier

On the Complexity of Best Arm Identification in Multi-Armed Bandit Models Aurlien Garivier Institut de Mathmatiques de Toulouse Information Theory, Learning and Big Data Simons Institute, Berkeley, March 2015 Simple Multi-Armed Bandit

514 views • 29 slides
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I S ebastien

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I S ebastien

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I S ebastien Bubeck Theory Group i.i.d. multi-armed bandit, Robbins [1952] i.i.d. multi-armed bandit, Robbins [1952] Known parameters: number of arms n and

809 views • 67 slides
About this class An example Bandit problems in general Two-armed bandits Multi-armed bandits

About this class An example Bandit problems in general Two-armed bandits Multi-armed bandits

About this class An example Bandit problems in general Two-armed bandits Multi-armed bandits and Gittins indices 1 An Example [Most of this lecture from Berry &amp; Fristedt] You want to maximize the sum of two observa- tions. The process

407 views • 13 slides
Contextual Multi-armed Bandit Algorithm for Semiparametric Reward Model Gi-Soo Kim, Myunghee Cho

Contextual Multi-armed Bandit Algorithm for Semiparametric Reward Model Gi-Soo Kim, Myunghee Cho

Contextual Multi-armed Bandit Algorithm for Semiparametric Reward Model Gi-Soo Kim, Myunghee Cho Paik Seoul National University June 13, 2019 Introduction We propose a new contextual multi-armed bandit (MAB) algorithm for the nonstationary

209 views • 20 slides
The Nonstochastic Multi Armed Bandit Problem Part 2 and counting... Shahaf Nacson TAU Nov 15,

The Nonstochastic Multi Armed Bandit Problem Part 2 and counting... Shahaf Nacson TAU Nov 15,

Reminder from last week Goals Lower bounds on the weak regret The Nonstochastic Multi Armed Bandit Problem Part 2 and counting... Shahaf Nacson TAU Nov 15, 2017 Shahaf Nacson The Nonstochastic Multi Armed Bandit Problem Reminder from last

1.05k views • 79 slides
The Multi-Armed Bandit Problem Nicol` o Cesa-Bianchi Universit` a degli Studi di Milano Nicol`

The Multi-Armed Bandit Problem Nicol` o Cesa-Bianchi Universit` a degli Studi di Milano Nicol`

The Multi-Armed Bandit Problem Nicol` o Cesa-Bianchi Universit` a degli Studi di Milano Nicol` o Cesa-Bianchi The Multi-Armed Bandit Problem The bandit problem [Robbins, 1952] . . . K slot machines Rewards X i ,1 , X i ,2 , . . . of

591 views • 15 slides
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part 2 S ebastien

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part 2 S ebastien

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part 2 S ebastien Bubeck Theory Group The linear bandit problem, Auer [2002] Known parameters: compact action set A R n , adversarys action set L R n ,

948 views • 69 slides
The multi armed-bandit problem (with covariates if we have time) Vianney Perchet &amp; Philippe

The multi armed-bandit problem (with covariates if we have time) Vianney Perchet &amp; Philippe

The multi armed-bandit problem (with covariates if we have time) Vianney Perchet &amp; Philippe Rigollet LPMA ORFE Universit Paris Diderot Princeton University Algorithms and Dynamics for Games and Optimization October, 14-18th 2013

1.51k views • 76 slides
Multi-armed bandit problem and its applications in reinforcement learning Pietro Lovato Ph.D.

Multi-armed bandit problem and its applications in reinforcement learning Pietro Lovato Ph.D.

University of Verona 28/01/2013 Multi-armed bandit problem and its applications in reinforcement learning Pietro Lovato Ph.D. Course on Special Topics in AI: Intelligent Agents and Multi-Agent Systems Overview Introduction: Reinforcement

628 views • 25 slides
Efficient Algorithms for Infinite-Armed Bandit Arghya Roy Chaudhuri under the guidance of Prof.

Efficient Algorithms for Infinite-Armed Bandit Arghya Roy Chaudhuri under the guidance of Prof.

Efficient Algorithms for Infinite-Armed Bandit Arghya Roy Chaudhuri under the guidance of Prof. Shivaram Kalyanakrishnan Department of Computer Science and Engineering Indian Institute of Technology Bombay Arghya Efficient Algorithms for

349 views • 17 slides
Multi-Armed Bandit: Learning in Dynamic Systems with Unknown Models Qing Zhao Department of

Multi-Armed Bandit: Learning in Dynamic Systems with Unknown Models Qing Zhao Department of

1 Multi-Armed Bandit: Learning in Dynamic Systems with Unknown Models Qing Zhao Department of Electrical and Computer Engineering University of California, Davis, CA 95616 Supported by NSF, ARO. c Qing Zhao. Talk at UMD, October, 2011. 2

963 views • 30 slides
A Multi-Armed Bandit Framework for Recommendations at Netflix Jaya Kawale Elliot Chow

A Multi-Armed Bandit Framework for Recommendations at Netflix Jaya Kawale Elliot Chow

A Multi-Armed Bandit Framework for Recommendations at Netflix Jaya Kawale Elliot Chow Recommendations at Netflix Personalized Homepage for each member Goal : Quickly help members find content theyd like to watch Risk : Member may

1.03k views • 44 slides
REINFORCEMENT LEARNING BANDIT PROBLEMS How does it work? N-armed bandits as in slot machines

REINFORCEMENT LEARNING BANDIT PROBLEMS How does it work? N-armed bandits as in slot machines

REINFORCEMENT LEARNING BANDIT PROBLEMS How does it work? N-armed bandits as in slot machines We detect a state action selection We choose an action evaluation We get a reward Our aim is to learn a policy what action to choose in

428 views • 3 slides
Automated Formal Verification of Smart Contracts Florian Hubert Dana Drachsler- Andrei Arthur

Automated Formal Verification of Smart Contracts Florian Hubert Dana Drachsler- Andrei Arthur

Automated Formal Verification of Smart Contracts Florian Hubert Dana Drachsler- Andrei Arthur Quentin Petar Martin Buenzli Ritzdorf Cohen Dan Gervais Hibon Tsankov Vechev Growth of the Ethereum Ecosystem $27B Billions of USD are

180 views • 16 slides
3-509

3-509

3-509 liuzhen@sjtu.edu.cn 1 1. Wallet How to store and spend your bitcoins? Each coin is a (address, value) pair on the (public) blockchain Store

604 views • 16 slides
Securify: Practical Security Analysis of Smart Contracts https://securify.ch Dr. Petar Tsankov

Securify: Practical Security Analysis of Smart Contracts https://securify.ch Dr. Petar Tsankov

Securify: Practical Security Analysis of Smart Contracts https://securify.ch Dr. Petar Tsankov Scientific Researcher, ICE center, ETH Zurich Co-founder and Chief Scientist, ChainSecurity AG http://www.ptsankov.com/ @ptsankov

407 views • 39 slides
A full node for every Alice By Nicolas Dorier Main maintainer of NBitcoin and BTCPay .NET

A full node for every Alice By Nicolas Dorier Main maintainer of NBitcoin and BTCPay .NET

A full node for every Alice By Nicolas Dorier Main maintainer of NBitcoin and BTCPay .NET fanboy Foreword No ICO Not a company No funding Need Open Source developers, not money BTCPay is started as a reaction to

469 views • 36 slides
Do Investors Care About Corporate Externalities? Experimental Evidence Jean-Franois Bonnefon

Do Investors Care About Corporate Externalities? Experimental Evidence Jean-Franois Bonnefon

Do Investors Care About Corporate Externalities? Experimental Evidence Jean-Franois Bonnefon Augustin Landier TSE &amp; IAST HEC Pari Sastry David Thesmar MIT MIT, NBER, CEPR October 17, 2020 1/21 The Question Calls for firms to

1.19k views • 31 slides
V ERI S OLID : Correct-by-Design Smart Contracts for Ethereum Anastasia Mavridou 1 , Aron

V ERI S OLID : Correct-by-Design Smart Contracts for Ethereum Anastasia Mavridou 1 , Aron

V ERI S OLID : Correct-by-Design Smart Contracts for Ethereum Anastasia Mavridou 1 , Aron Laszka 2 , Emmanouela Stachtiari 3 , Abhishek Dubey 4 1 NASA Ames 2 University of Houston 3 Aristotle University of Thessaloniki 4 Vanderbilt University

555 views • 18 slides
Lecture 1.3: Groups in science, art, and mathematics Matthew Macauley Department of Mathematical

Lecture 1.3: Groups in science, art, and mathematics Matthew Macauley Department of Mathematical

Lecture 1.3: Groups in science, art, and mathematics Matthew Macauley Department of Mathematical Sciences Clemson University http://www.math.clemson.edu/~macaule/ Math 4120, Modern Algebra M. Macauley (Clemson) Lecture 1.3: Groups in science,

470 views • 14 slides
Hyperbolic Ornaments Drawing in Non-Euclidean Crystallographic Groups Martin von Gagern joint

Hyperbolic Ornaments Drawing in Non-Euclidean Crystallographic Groups Martin von Gagern joint

Basics Program Hyperbolic Ornaments Drawing in Non-Euclidean Crystallographic Groups Martin von Gagern joint work with Jrgen Richter-Gebert Technische Universitt Mnchen Second International Congress on Mathematical Software, September 1

2.08k views • 65 slides

More recommend