Adversarial Search Robert Platt Northeastern University Some - - PowerPoint PPT Presentation

▶
adversarial search
SMART_READER_LITE
LIVE PREVIEW

Adversarial Search Robert Platt Northeastern University Some - - PowerPoint PPT Presentation

Dec 11, 2022 346 likes •1.19k views

Adversarial Search Robert Platt Northeastern University Some images and slides are used from: 1. CS188 UC Berkeley 2. RN, AIMA What is adversarial search? Adversarial search: planning used to play a game such as chess or checkers

slide-1
SLIDE 1

Adversarial Search

Robert Platt Northeastern University Some images and slides are used from:

  • 1. CS188 UC Berkeley
  • 2. RN, AIMA
slide-2
SLIDE 2

What is adversarial search?

Adversarial search: planning used to play a game such as chess or checkers – algorithms are similar to graph search except that we plan under the assumption that our opponent will maximize his own advantage...

slide-3
SLIDE 3

Examples of adversarial search

Chess Checkers Tic-tac-toe Go

slide-4
SLIDE 4

Examples of adversarial search

Chess Checkers Tic-tac-toe Go Solved/unsolved? Solved/unsolved? Solved/unsolved? Solved/unsolved? Outcome of game can be predicted from any initial state assuming both players play perfectly

slide-5
SLIDE 5

Examples of adversarial search

Chess Checkers Tic-tac-toe Go Outcome of game can be predicted from any initial state assuming both players play perfectly Unsolved Solved Solved Unsolved

slide-6
SLIDE 6

Examples of adversarial search

Chess Checkers Tic-tac-toe Go Outcome of game can be predicted from any initial state assuming both players play perfectly Unsolved Solved Solved Unsolved ~10^40 states ~10^20 states Less than 9!=362k states ?

slide-7
SLIDE 7

Different types of games

Deterministic / stochastic Two player / multi player? Zero-sum / non zero-sum Fully observable / partially observable

slide-8
SLIDE 8

What is a zero-sum game?

Zero-sum:

  • Sum of utilities is zero
  • In the case of a two player game:
  • Pure competition

Not zero-sum:

  • Agents have arbitrary utilities
  • Might induce cooperation or competition
slide-9
SLIDE 9

A formal definition of a deterministic game

Problem: State set: S (start at s0) Players: P={1...N} (usually take turns) Action set: A Transition Function: SxA -> S Terminal Test: S -> {t,f} Terminal Utilities: SxP -> R Solution: Policy, S -> A Objective: Find an optimal policy – a policy that maximizes utility assuming that adversary acts optimally.

slide-10
SLIDE 10

A formal definition of a deterministic game

Problem: State set: S (start at s0) Players: P={1...N} (usually take turns) Action set: A Transition Function: SxA -> S Terminal Test: S -> {t,f} Terminal Utilities: SxP -> R Solution: Policy, S -> A Objective: Find an optimal policy – a policy that maximizes utility assuming that adversary acts optimally.

How is this similar/different to the def'n of a standard search problem?

slide-11
SLIDE 11

A formal definition of a deterministic game

Problem: State set: S (start at s0) Players: P={1...N} (usually take turns) Action set: A Transition Function: SxA -> S Terminal Test: S -> {t,f} Terminal Utilities: SxP -> R Solution: Policy, S -> A Objective: Find an optimal policy – a policy that maximizes utility assuming that adversary acts optimally.

How do we solve this problem?

slide-12
SLIDE 12

Adversarial search

Image: Berkeley CS188 course notes (downloaded Summer 2015)

slide-13
SLIDE 13

This is a game tree for tic-tac-toe

Images: AIMA, Berkeley CS188 course notes (downloaded Summer 2015)

slide-14
SLIDE 14

This is a game tree for tic-tac-toe

Images: AIMA, Berkeley CS188 course notes (downloaded Summer 2015)

You

slide-15
SLIDE 15

This is a game tree for tic-tac-toe

Images: AIMA, Berkeley CS188 course notes (downloaded Summer 2015)

You Them

slide-16
SLIDE 16

This is a game tree for tic-tac-toe

Images: AIMA, Berkeley CS188 course notes (downloaded Summer 2015)

You Them You

slide-17
SLIDE 17

This is a game tree for tic-tac-toe

Images: AIMA, Berkeley CS188 course notes (downloaded Summer 2015)

You Them Them You

slide-18
SLIDE 18

This is a game tree for tic-tac-toe

Images: AIMA, Berkeley CS188 course notes (downloaded Summer 2015)

You Them Them You Utility

slide-19
SLIDE 19

What is Minimax?

Consider a simple game:

  • 1. you make a move
  • 2. your opponent makes a move
  • 3. game ends
slide-20
SLIDE 20

What is Minimax?

Consider a simple game:

  • 1. you make a move
  • 2. your opponent makes a move
  • 3. game ends

What does the minimax tree look like in this case?

slide-21
SLIDE 21

What is Minimax?

3 8 12 2 6 4 14 2 5

Max (you) Min (them) Max (you)

Consider a simple game:

  • 1. you make a move
  • 2. your opponent makes a move
  • 3. game ends

What does the minimax tree look like in this case?

slide-22
SLIDE 22

What is Minimax?

3 8 12 2 6 4 14 2 5

Max (you) Min (them) Max (you)

These are terminal utilities – assume we know what these values are

slide-23
SLIDE 23

What is Minimax?

3 8 12 2 6 4 14 2 5 3 2 2

Max (you) Min (them) Max (you)

slide-24
SLIDE 24

What is Minimax?

3 8 12 2 6 4 14 2 5 3 2 2 3

Max (you) Min (them) Max (you) Max (you) Min (them)

slide-25
SLIDE 25

What is Minimax?

3 8 12 2 6 4 14 2 5 3 2 2 3

Max (you) Min (them) Max (you)

This is called “backing up” the values

slide-26
SLIDE 26

What is Minimax?

3 8 12 2 6 4 14 2 5

Okay – so we know how to back up values ... … but, how do we construct the tree?

This tree is already built...

slide-27
SLIDE 27

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

slide-28
SLIDE 28

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

slide-29
SLIDE 29

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

3

slide-30
SLIDE 30

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

3 12

slide-31
SLIDE 31

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

3 8 12

slide-32
SLIDE 32

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

3 8 12 3

slide-33
SLIDE 33

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

3 8 12 3

slide-34
SLIDE 34

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

3 8 12 2 6 4 3 2

slide-35
SLIDE 35

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense.

3 8 12 2 6 4 14 2 5 3 2 2 3

slide-36
SLIDE 36

What is Minimax?

Notice that we only get utilities at the bottom of the tree … – therefore, DFS makes sense. – since most games have forward progress, the distinction between tree search and graph search is less important

slide-37
SLIDE 37

What is Minimax?

slide-38
SLIDE 38

Is it always correct to assume your opponent plays optimally?

Minimax properties

Slide: Berkeley CS188 course notes (downloaded Summer 2015)

10 10 9 100 max min

slide-39
SLIDE 39

Minimax vs “expectimax”

Slide: Berkeley CS188 course notes (downloaded Summer 2015)

slide-40
SLIDE 40

Minimax vs “expectimax”

Slide: Berkeley CS188 course notes (downloaded Summer 2015)

slide-41
SLIDE 41

Is minimax optimal? Is it complete?

Minimax properties

slide-42
SLIDE 42

Is minimax optimal? Is it complete? Time complexity = ? Space complexity = ?

Minimax properties

slide-43
SLIDE 43

Is minimax optimal? Is it complete? Time complexity = Space complexity =

Minimax properties

slide-44
SLIDE 44

Is minimax optimal? Is it complete? Time complexity = Space complexity = Is it practical? In chess, b=35, d=100

Minimax properties

slide-45
SLIDE 45

Is minimax optimal? Is it complete? Time complexity = Space complexity = Is it practical? In chess, b=35, d=100

Minimax properties

is a big number...

slide-46
SLIDE 46

Is minimax optimal? Is it complete? Time complexity = Space complexity = Is it practical? In chess, b=35, d=100

Minimax properties

is a big number...

So what can we do?

slide-47
SLIDE 47

Key idea: cut off search at a certain depth and give the corresponding nodes an estimated value.

Evaluation functions

? ? ? ?

  • 1
  • 2

4 9 4

  • 2

4 Image: Berkeley CS188 course notes (downloaded Summer 2015)

Cut it off here

slide-48
SLIDE 48

Key idea: cut off search at a certain depth and give the corresponding nodes an estimated value.

Evaluation functions

? ? ? ?

  • 1
  • 2

4 9 4

  • 2

4 Image: Berkeley CS188 course notes (downloaded Summer 2015)

Cut it off here the evaluation function makes this estimate.

slide-49
SLIDE 49

Evaluation functions

How does the evaluation function make the estimate? – depends upon domain For example, in chess, the value of a state might equal the sum of piece values. – a pawn counts for 1 – a rook counts for 5 – a knight counts for 3 ...

slide-50
SLIDE 50

A weighted linear evaluation function

number of pawns on the board number of knights on the board A pawn counts for 1 A knight counts for 3

slide-51
SLIDE 51

At what depth do you run the evaluation function?

? ? ? ?

  • 1
  • 2

4 9 4

  • 2

4

Option 1: cut off search at a fixed depth Option 2: cut off search at quiescient states deeper than a certain threshold Option 3: ? The deeper your threshold, the less the quality of the evaluation function matters...

slide-52
SLIDE 52

At what depth do you run the evaluation function?

Slide: Berkeley CS188 course notes (downloaded Summer 2015)

Search depth=2

slide-53
SLIDE 53

At what depth do you run the evaluation function?

Slide: Berkeley CS188 course notes (downloaded Summer 2015)

Search depth=10

slide-54
SLIDE 54

Alpha/Beta pruning

Image: Berkeley CS188 course notes (downloaded Summer 2015)

slide-55
SLIDE 55

Alpha/Beta pruning

3 8 12 3

slide-56
SLIDE 56

Alpha/Beta pruning

3 8 12 3

slide-57
SLIDE 57

Alpha/Beta pruning

3 8 12 2 3

slide-58
SLIDE 58

Alpha/Beta pruning

3 8 12 2 4 3

slide-59
SLIDE 59

Alpha/Beta pruning

3 8 12 2 4 3 We don't need to expand this node!

slide-60
SLIDE 60

Alpha/Beta pruning

3 8 12 2 4 3 We don't need to expand this node! Why?

slide-61
SLIDE 61

Alpha/Beta pruning

3 8 12 2 4 3 We don't need to expand this node! Why?

Max Min

slide-62
SLIDE 62

Alpha/Beta pruning

Max Min

3 8 12 2 14 2 5 3 2 2 3

slide-63
SLIDE 63

Alpha/Beta pruning

Max Min

3 8 12 2 14 2 5 3 2 2 3 So, we don't need to expand these nodes in order to back up correct values!

slide-64
SLIDE 64

Alpha/Beta pruning

Max Min

3 8 12 2 14 2 5 3 2 2 3 So, we don't need to expand these nodes in order to back up correct values! That's alpha-beta pruning.

slide-65
SLIDE 65

Alpha/Beta pruning: algorithm idea

  • General confjguration (MIN version)
  • We’re computing the MIN-VALUE at

some node n

  • We’re looping over n’s children
  • n’s estimate of the childrens’ min is

dropping

  • Who cares about n’s value? MAX
  • Let a be the best value that MAX can

get at any choice point along the current path from the root

  • If n becomes worse than a, MAX will

avoid it, so we can stop considering n’s

  • ther children (it’s already bad enough

that it won’t be played)

  • MAX version is symmetric

MAX MIN MAX MIN

a n

Slide: Berkeley CS188 course notes (downloaded Summer 2015)

slide-66
SLIDE 66

Alpha/Beta pruning: algorithm

Slide: adapted from Berkeley CS188 course notes (downloaded Summer 2015)

def min-value(state , α, β): initialize v = +∞ for each successor of state: v = min(v, value(successor, α, β)) if v ≤ α return v β = min(β, v) return v def max-value(state, α, β): initialize v = -∞ for each successor of state: v = max(v, value(successor, α, β)) if v ≥ β return v α = max(α, v) return v α: best value so far for MAX along path to root β: best value so far for MIN along path to root

slide-67
SLIDE 67

Alpha/Beta pruning

(-inf,+inf)

slide-68
SLIDE 68

Alpha/Beta pruning

(-inf,+inf) (-inf,+inf)

slide-69
SLIDE 69

Alpha/Beta pruning

3 3 (-inf,+inf) (-inf,3) Best value for far for MIN along path to root

slide-70
SLIDE 70

Alpha/Beta pruning

3 12 3 (-inf,+inf) (-inf,3) Best value for far for MIN along path to root

slide-71
SLIDE 71

Alpha/Beta pruning

3 8 12 3 (-inf,+inf) (-inf,3) Best value for far for MIN along path to root

slide-72
SLIDE 72

Alpha/Beta pruning

3 8 12 3 (3,+inf) (-inf,3) Best value for far for MAX along path to root

slide-73
SLIDE 73

Alpha/Beta pruning

3 8 12 3 (3,+inf) (-inf,3) (3,+inf)

slide-74
SLIDE 74

Alpha/Beta pruning

3 8 12 3 2 (3,+inf) (-inf,3) (3,+inf) 2

slide-75
SLIDE 75

Alpha/Beta pruning

3 8 12 3 2 (3,+inf) (-inf,3) (3,+inf) 2 Prune because value (2) is out of alpha-beta range

slide-76
SLIDE 76

Alpha/Beta pruning

3 8 12 3 2 (3,+inf) (-inf,3) (3,+inf) 2 (3,+inf)

slide-77
SLIDE 77

Alpha/Beta pruning

3 8 12 3 2 (3,+inf) (-inf,3) (3,+inf) 2 14 (3,14) 14

slide-78
SLIDE 78

Alpha/Beta pruning

3 8 12 3 2 (3,+inf) (-inf,3) (3,+inf) 2 5 (3,5) 14 5

slide-79
SLIDE 79

Alpha/Beta pruning

3 8 12 3 2 (3,+inf) (-inf,3) (3,+inf) 2 2 (3,5) 14 5 2

slide-80
SLIDE 80

Alpha/Beta properties

Is it complete?

slide-81
SLIDE 81

Alpha/Beta properties

Is it complete? How much does alpha/beta help relative to minimax? Minimax time complexity = Alpha/beta time complexity >= – the improvement w/ alpha/beta depends upon move ordering...

slide-82
SLIDE 82

Alpha/Beta properties

Is it complete? How much does alpha/beta help relative to minimax? Minimax time complexity = Alpha/beta time complexity >= – the improvement w/ alpha/beta depends upon move ordering... 3 8 12 2 6 4 14 2 5 3 2 2 3 The order in which we expand a node.

slide-83
SLIDE 83

Alpha/Beta properties

Is it complete? How much does alpha/beta help relative to minimax? Minimax time complexity = Alpha/beta time complexity >= – the improvement w/ alpha/beta depends upon move ordering... 3 8 12 2 6 4 14 2 5 3 2 2 3 The order in which we expand a node. How to choose move ordering? Use IDS. – on each iteration of IDS, use prior run to inform ordering of next node expansions.