mastering the game of go with deep
play

Mastering the game of Go with deep neural networks and tree search - PowerPoint PPT Presentation

Nov 06, 2023 •46 likes •556 views

Mastering the game of Go with deep neural networks and tree search Nature, Jan, 2016 Roadmap What this paper is about? Deep Learning Search problem How to explore a huge tree (graph) AlphaGo Video

  1. Mastering the game of Go with deep neural networks and tree search Nature, Jan, 2016

  2. Roadmap What this paper is about? • Deep Learning • Search problem • How to explore a huge tree (graph)

  3. AlphaGo Video https://www.youtube.com/watch?v=53YLZBSS0cc https://www.youtube.com/watch?v=g-dKXOlsf98

  4. rank Al AlphaGo vs vs*European*Champion*(Fan*Hui 27Da Dan) * October$5$– 9,$2015 <Official$match> I Time+limit:+1+hour I AlphaGo Wins (5:0)

  5. Al AlphaGo vs vs*Wo World*Champion*(Lee*Se Sedol 97Da Dan) March$9$– 15,$2016 <Official$match> I Time+limit:+2+hours Venue :+ Seoul ,+Four+Seasons+Hotel

  7. Lee*Sedol wiki Photo+source: Maeil+Economics 2013/04

  8. Lee Sedol

  9. =$multiple$machines European$champion

  10. The Game

  15. Go Elo Ranking http://www.goratings.org/history/

  16. Lee Sedol VS Ke Jie

  17. How about Other Games?

  18. Tic Tac Toe

  19. Chess

  20. Chess (1996)

  21. Deep Blue (1996)

  23. AlphaGo is the Skynet?

  24. Go Game

  25. Simple Rules

  26. High Complexity

  27. High Complexity

  29. Different Games

  30. Search Problem (the search space)

  31. Tic Tac Toe

  32. Tic Tac Toe

  33. The “Tree” in Tic Tac Toe

  34. The “Tree” of Chess

  35. The “Tree” of Go Game

  36. Search Problem (how to search)

  37. MiniMax in Tic Tac Toe

  38. Adversarial"Search"–"MiniMax"" 1" 1" 1" 0" 0" J1" J1" 0" J1" 0" J1" J1" 5"

  39. Adversarial"Search"–"MiniMax"" J1" 1" J1" 0" 1" 1" 0" J1" J1" J1" 1" 1" 0" 1" 0" J1" J1" 0" J1" 0" J1" J1" 6"

  40. What is the problem? 1. Generate the Search Tree 2. use MinMax Search

  41. The Size of the Tree Tic Tac Toe: b = 9, d =9 Chess: b = 35, d =80 Go: b = 250, d =150 b : number of legal move per position d : its depth (game length)

  42. One Grain of Rice https://www.youtube.com/watch?v=byk3pA1GPgU

  43. The “Space” of GO Game

  44. How about other Games? • Flappy bird? • Angry Bird? Tic Tac Toe: • Starcraft? b = 9, d =9 • learning a language Chess: b = 35, d =80 • Write a paper Go: • Get a MS/PhD degree b = 250, d =150 • Finding a job • Life

  45. How to solve?

  46. Chess (1996)

  47. Monte Carlo

  48. Las Vegas

  50. Monte"Carlo"Tree"Search" Tree"search" ……." ……." ……." ……." Monte"Carlo"search" ……." ……." ……." ……." ……." 7"

  51. Monte"Carlo"Tree"Search" • Tree"Search"+"Monte"Carlo"Method"" – SelecIon" white"wins"/"total" 3/5" – Expansion" – SimulaIon" 2/3" 1/2" – BackJPropagaIon" 1/1" 1/2" 1/1" 0/1" 1/1" 0/1" 8"

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

540 views • 15 slides
Mastering the game of Go with deep neural networks and tree search Article overview by

Mastering the game of Go with deep neural networks and tree search Article overview by

David Silver et al. from Google DeepMind Mastering the game of Go with deep neural networks and tree search Article overview by Reinforcement Learning Seminar Ilya Kuzovkin University of Tartu, 2016 T HE G AME OF G O B OARD B OARD S TONES B

907 views • 66 slides
THE MOD METHOD with VESPERS MASTERING In this Module What mastering can do &amp; what it

THE MOD METHOD with VESPERS MASTERING In this Module What mastering can do &amp; what it

THE MOD METHOD with VESPERS MASTERING In this Module What mastering can do &amp; what it cant Self-mastering vs. third-party mastering Picking a mastering engineer Mastering work fl ow Audio artifacts &amp; fi delity:

520 views • 16 slides
Mastering the Game of Go without Human Knowledge 06/15/18 Presented by: Henry Chen CS885

Mastering the Game of Go without Human Knowledge 06/15/18 Presented by: Henry Chen CS885

Mastering the Game of Go without Human Knowledge 06/15/18 Presented by: Henry Chen CS885 Reinforcement Learning Introduction Image source: https://medium.com/syncedreview/alphago-zero-approaching-perfection-d8170e2b4e48 PAGE 2 Introduction

455 views • 22 slides
Analogs of Linguistic Structure in Deep Representations Jacob Andreas and Dan Klein A game for

Analogs of Linguistic Structure in Deep Representations Jacob Andreas and Dan Klein A game for

Analogs of Linguistic Structure in Deep Representations Jacob Andreas and Dan Klein A game for humans everything but the blue shapes orange square and non-squares [FitzGerald et al. 2013] 2 A game for RNNs 1.0 2.3 -0.3 0.4

898 views • 44 slides
Mastering the Gospel P resentation Welcome to the CMF Training page on Mastering a Gospel

Mastering the Gospel P resentation Welcome to the CMF Training page on Mastering a Gospel

Mastering the Gospel P resentation Welcome to the CMF Training page on Mastering a Gospel Presentation. This may be the most important thing that you do and you need to be able to do this as you grow in your Christian walk. You may be the only

290 views • 6 slides
From Deep Blue to Monte Carlo: An Update on Game

From Deep Blue to Monte Carlo: An Update on Game

From Deep Blue to Monte Carlo: An Update on Game Tree Research Akihiro Kishimoto and Mar0n Mller AAAI-14 Tutorial 6: State of

737 views • 29 slides
e-Bug Junior Game Junior Game Game Style Game Process Demo Game Mechanics and

e-Bug Junior Game Junior Game Game Style Game Process Demo Game Mechanics and

e-Bug Junior Game Junior Game Game Style Game Process Demo Game Mechanics and Learning Outcomes Feedback to date and evaluation plans Game Style Game style encompasses two genres of game o Platform game fast action

575 views • 22 slides
e-Bug Senior Game Senior Game Game Style Game Process Demo Game Puzzles and

e-Bug Senior Game Senior Game Game Style Game Process Demo Game Puzzles and

e-Bug Senior Game Senior Game Game Style Game Process Demo Game Puzzles and Learning Outcomes Feedback to date and evaluation plans Game Style Game is a detective game where the player investigates microbial problems.

572 views • 19 slides
Mastering Your Mindset Mastering Your Money Focus: How to Focus on Earning More Income, and

Mastering Your Mindset Mastering Your Money Focus: How to Focus on Earning More Income, and

Mastering Your Mindset Mastering Your Money Focus: How to Focus on Earning More Income, and Managing &amp; Investing Your Money with John Assaraf What Is Money? Money Is A VERY Powerful Idea Money Is A Means Of Exchange Money Is

955 views • 40 slides
Mastering Great Performances in Public Presentation The need to personally communicate is

Mastering Great Performances in Public Presentation The need to personally communicate is

Mastering Great Performances in Public Presentation The need to personally communicate is ever-increasing. Mastering personal appearances on a whole spectrum of channels and formats - TV, radio, YouTube, presentations or speeches - is vital in

272 views • 5 slides
Mastering Data with Spark and ML Strata London 2019 About Me IIT Delhi, 1998 Founder and CEO,

Mastering Data with Spark and ML Strata London 2019 About Me IIT Delhi, 1998 Founder and CEO,

Mastering Data with Spark and ML Strata London 2019 About Me IIT Delhi, 1998 Founder and CEO, Nube Technologies Strata Data San Jose Program Committee Speaker at Spark Summit, Strata, GIDS etc Nube India based startup Deep technical

484 views • 32 slides
Deep Learning for Video Game Playing Authors: Niels Justesen, Philip Bontrager, Julian Togelius,

Deep Learning for Video Game Playing Authors: Niels Justesen, Philip Bontrager, Julian Togelius,

Deep Learning for Video Game Playing Authors: Niels Justesen, Philip Bontrager, Julian Togelius, Sebastian Risi Presented by: Runsheng (Benson) Guo 1 Outline Background Methods History Open Challenges Recent Advances 2

667 views • 21 slides
Towards a Foundation of Deep Learning: SGD, Overparametrization, and Generalization Jason D. Lee

Towards a Foundation of Deep Learning: SGD, Overparametrization, and Generalization Jason D. Lee

Towards a Foundation of Deep Learning: SGD, Overparametrization, and Generalization Jason D. Lee University of Southern California January 29, 2019 Jason Lee Successes of Deep Learning Game-playing (AlphaGo, DOTA, King of Glory) Computer

665 views • 48 slides
The Composition Method Wolfgang Thomas Francqui Lecture, Mons, April 2013 Mastering compositions

The Composition Method Wolfgang Thomas Francqui Lecture, Mons, April 2013 Mastering compositions

The Composition Method Wolfgang Thomas Francqui Lecture, Mons, April 2013 Mastering compositions Wolfgang Thomas Overview 1. Motivation 2. m -equivalence and the EF-game 3. Applications 4. m -types 5. Monadic types Wolfgang Thomas

1.06k views • 40 slides
1 What a Game is Not Outline A bunch of cool features What is a Game? Necessary, but

1 What a Game is Not Outline A bunch of cool features What is a Game? Necessary, but

Outline What is a Game? The Game Development Genres Process: Genres What is a Game? (1 of 4) What is a Game? (2 of 4) Movie? Movie? No interaction , outcome fixed Toy? Toy? Puzzle? No goal , but still fun!

512 views • 4 slides
The natural emergence of the SFR-H2 surface density relation in galaxy simulations Alessandro

The natural emergence of the SFR-H2 surface density relation in galaxy simulations Alessandro

The natural emergence of the SFR-H2 surface density relation in galaxy simulations Alessandro Lupi (Institut dAstrophysique de Paris) THE ROLE OF GAS IN GALAXY DYNAMICS with: S. Bovino, P. R. Capelo, M. Volonteri, J. Silk October 2nd, 2017

542 views • 14 slides
Energy-Efficient Management of Virtual Machines in Data Centers for Cloud Computing Anton

Energy-Efficient Management of Virtual Machines in Data Centers for Cloud Computing Anton

H EURISTICS M ARKOV H OST O VERLOAD D ETECTION I MPLEMENTATION C ONCLUSIONS Energy-Efficient Management of Virtual Machines in Data Centers for Cloud Computing Anton Beloglazov Supervisor: Prof. Rajkumar Buyya The Cloud Computing and

995 views • 87 slides
Inference and Optimalities in Estimation of Gaussian Graphical Model Harrison H. Zhou Department

Inference and Optimalities in Estimation of Gaussian Graphical Model Harrison H. Zhou Department

Inference and Optimalities in Estimation of Gaussian Graphical Model Harrison H. Zhou Department of Statistics Yale University Jointly with Zhao Ren, Tingni Sun and Cun-Hui Zhang 1 Outline Introduction Main Results Asymptotic

509 views • 27 slides
Zero forcing, propagation time, and throttling on a graph Leslie Hogben Iowa State University

Zero forcing, propagation time, and throttling on a graph Leslie Hogben Iowa State University

Zero forcing, propagation time, and throttling on a graph Leslie Hogben Iowa State University and American Institute of Mathematics New York Combinatorics Seminar August 28, 2020 Leslie Hogben (Iowa State University and American Institute of

752 views • 53 slides
Ada, or How to Enforce Safety Rules at Compile Time Jean-Pierre Rosen Adalog www.adalog.fr

Ada, or How to Enforce Safety Rules at Compile Time Jean-Pierre Rosen Adalog www.adalog.fr

Ada, or How to Enforce Safety Rules at Compile Time Jean-Pierre Rosen Adalog www.adalog.fr Safety Integrity Levels and Segregation Railway systems: EN-50128 defines 5 integrity levels From SIL0 (not critical) to SIL4 (highest

321 views • 8 slides
CPU Scheduling Schedulers in the OS Structure of a CPU Scheduler Scheduling =

CPU Scheduling Schedulers in the OS Structure of a CPU Scheduler Scheduling =

CPSC 410 / 611 : Operating Systems CPU Scheduling Schedulers in the OS Structure of a CPU Scheduler Scheduling = Selection + Dispatching Criteria for scheduling Scheduling Algorithms FIFO/FCFS SPF / SRTF

342 views • 16 slides
In-medium QQ potential from lattice QCD &amp; the generalized Gauss-law Alexander Rothkopf

In-medium QQ potential from lattice QCD &amp; the generalized Gauss-law Alexander Rothkopf

In-medium QQ potential from lattice QCD &amp; the generalized Gauss-law Alexander Rothkopf Faculty of Science and Technology Department of Mathematics and Physics University of Stavanger References : P. Petreczky, A.R., J. Weber, NPA982

955 views • 66 slides
production environment Yahoo! Chiebukuro (a CQA service of Yahoo! Japan) Task Given a

production environment Yahoo! Chiebukuro (a CQA service of Yahoo! Japan) Task Given a

Overview of the NTCIR-13 O penLive Q Task Makoto P. Kato, Takehiro Yamamoto (Kyoto University) , Sumio Fujita, Akiomi Nishida, Tomohiro Manabe (Yahoo Japan Corporation) Agenda Task Design (3 slides) Data (5 slides) Evaluation

531 views • 30 slides

More recommend