Mastering the game of Go with deep neural networks and tree search - - PowerPoint PPT Presentation

▶
mastering the game of go with deep
SMART_READER_LITE
LIVE PREVIEW

Mastering the game of Go with deep neural networks and tree search - - PowerPoint PPT Presentation

Nov 06, 2023 46 likes •557 views

Mastering the game of Go with deep neural networks and tree search Nature, Jan, 2016 Roadmap What this paper is about? Deep Learning Search problem How to explore a huge tree (graph) AlphaGo Video

slide-1
SLIDE 1

Mastering the game of Go with deep neural networks and tree search

Nature, Jan, 2016

slide-2
SLIDE 2

Roadmap

  • Deep Learning
  • Search problem
  • How to explore a huge tree (graph)

What this paper is about?

slide-3
SLIDE 3

AlphaGo Video

https://www.youtube.com/watch?v=g-dKXOlsf98 https://www.youtube.com/watch?v=53YLZBSS0cc

slide-4
SLIDE 4

Al AlphaGo vs vs*European*Champion*(Fan*Hui 27Da Dan)

October$5$– 9,$2015

<Official$match> I Time+limit:+1+hour I AlphaGo Wins (5:0) * rank

slide-5
SLIDE 5

Al AlphaGo vs vs*Wo World*Champion*(Lee*Se Sedol 97Da Dan)

March$9$– 15,$2016

<Official$match> I Time+limit:+2+hours Venue:+Seoul,+Four+Seasons+Hotel

slide-6
SLIDE 6
slide-7
SLIDE 7

Lee*Sedol

Photo+source: Maeil+Economics 2013/04

wiki

slide-8
SLIDE 8

Lee Sedol

slide-9
SLIDE 9

=$multiple$machines European$champion

slide-10
SLIDE 10

The Game

slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13
slide-14
SLIDE 14
slide-15
SLIDE 15

Go Elo Ranking

http://www.goratings.org/history/

slide-16
SLIDE 16

Lee Sedol VS Ke Jie

slide-17
SLIDE 17

How about Other Games?

slide-18
SLIDE 18

Tic Tac Toe

slide-19
SLIDE 19

Chess

slide-20
SLIDE 20

Chess (1996)

slide-21
SLIDE 21

Deep Blue (1996)

slide-22
SLIDE 22
slide-23
SLIDE 23

AlphaGo is the Skynet?

slide-24
SLIDE 24

Go Game

slide-25
SLIDE 25

Simple Rules

slide-26
SLIDE 26

High Complexity

slide-27
SLIDE 27

High Complexity

slide-28
SLIDE 28
slide-29
SLIDE 29

Different Games

slide-30
SLIDE 30

Search Problem (the search space)

slide-31
SLIDE 31

Tic Tac Toe

slide-32
SLIDE 32

Tic Tac Toe

slide-33
SLIDE 33

The “Tree” in Tic Tac Toe

slide-34
SLIDE 34

The “Tree” of Chess

slide-35
SLIDE 35

The “Tree” of Go Game

slide-36
SLIDE 36

Search Problem (how to search)

slide-37
SLIDE 37

MiniMax in Tic Tac Toe

slide-38
SLIDE 38

Adversarial"Search"–"MiniMax""

J1" 0" 1" 0" 0" 0" J1" J1" J1" J1" 1" 1"

5"

slide-39
SLIDE 39

Adversarial"Search"–"MiniMax""

J1" 0" 1" 0" 0" 0" J1" J1" J1" J1" 1" 1" J1" 0" J1" J1" 1" 1" 0" J1" 1" J1"

6"

slide-40
SLIDE 40

What is the problem?

1. Generate the Search Tree 2. use MinMax Search

slide-41
SLIDE 41

The Size of the Tree

Chess: b = 35, d =80 Go: b = 250, d =150 b : number of legal move per position d : its depth (game length) Tic Tac Toe: b = 9, d =9

slide-42
SLIDE 42

One Grain of Rice

https://www.youtube.com/watch?v=byk3pA1GPgU

slide-43
SLIDE 43

The “Space” of GO Game

slide-44
SLIDE 44

How about other Games?

  • Flappy bird?
  • Angry Bird?
  • Starcraft?
  • learning a language
  • Write a paper
  • Get a MS/PhD degree
  • Finding a job
  • Life

Chess: b = 35, d =80 Go: b = 250, d =150 Tic Tac Toe: b = 9, d =9

slide-45
SLIDE 45

How to solve?

slide-46
SLIDE 46

Chess (1996)

slide-47
SLIDE 47

Monte Carlo

slide-48
SLIDE 48

Las Vegas

slide-49
SLIDE 49
slide-50
SLIDE 50

Monte"Carlo"Tree"Search"

……." ……." ……." ……." ……." ……." ……." ……."

Tree"search" Monte"Carlo"search"

……."

7"

slide-51
SLIDE 51

Monte"Carlo"Tree"Search"

  • Tree"Search"+"Monte"Carlo"Method""

– SelecIon" – Expansion" – SimulaIon" – BackJPropagaIon"

3/5" 1/2" 2/3" 0/1"

white"wins"/"total"

1/2" 1/1" 1/1" 1/1" 0/1"

8"