Adversarial Search Volker Sorge Intro to AI: Problem of Games - - PowerPoint PPT Presentation

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Adversarial Search

Volker Sorge

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Problem of Games

◮ Discrete games, that is, games with clearly defined

single actions or moves, can be represented as search problems.

◮ Examples: Chess, Backgammon, Poker ◮ Search tree needs to contain moves for both protagonist

and antagonist.

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Basic Notions

◮ Protagonist called Max ◮ Antagonist called Min ◮ States are valid positions of a game with start state as

the initial position in the game

◮ Actions are the legal moves in the game. ◮ Terminal nodes are final states of the game (goal states

are usually winning states of Max).

◮ Utility function u:V → Z: Assigns a numeric value to

each state s. u(s)    > 0 if Max wins, < 0 if Min wins, = 0 in case of a draw. Z

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

MiniMax Idea

◮ Generate the entire search tree for the game. ◮ We have two types of layers in the tree:

◮ Max layer: It is Max’s turn. ◮ Min layer: It is Min’s turn.

◮ Compute utility function for all terminal nodes. ◮ Propagate utility values to the root:

◮ In Max layer: take maximum utility value of children. ◮ In Min layer: take minimum utility value of children.

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Algorithm: MiniMax

Minimax; Input: Start state s, Set of terminal states T, Utility function u, Flag MAX or MIN Output: Utility value u(s)

1 begin 2

if s ∈ T then

3

return u(s)

4

end

5

;

6

let {v1, . . . , vn} be children of s;

7

if MAX then

8

return max{MiniMax(v1, MIN), . . . , MiniMax(vn, MIN)}

9

end

10

else

11

return min{MiniMax(v1, MAX), . . . , MiniMax(vn, MAX)}

12

end

13 end

min max

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Algorithm: MiniMax

Observe that

◮ the algorithm computes the utility value for the start

• node. If we want to make a decision for the next move

in a particular state s we need to run it on all children

• f s as MIN nodes.

◮ often it is easier to implement a test for a terminal

instead of providing the set of goal states. For example, consider Chess vs. Tic-Tac-Toe

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: Tic-Tac-Toe

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: Tic-Tac-Toe

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: Tic-Tac-Toe

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: Tic-Tac-Toe

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: Tic-Tac-Toe

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: Tic-Tac-Toe

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Analysis

◮ We do not need to consider completeness and optimality

as these would depend on the oponents strategy.

◮ Time complexity is O(bm), where m is the maximum

depth of the tree.

◮ Space complexity is O(bm) as we effectively use a

depth first approach.

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Pruning

◮ In any interesting game MiniMax is unrealistic. ◮ Try to restrict computation by pruning the search tree. ◮ Determine early that a particular branch will not

contribute to a winning strategy

◮ In particular, stop descending into branches if it is

• bvious that

◮ Max would never do a move, or ◮ Min would never do a move.

◮ This assumes that Min plays rational!

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Alpha-Beta Pruning

◮ We define two values α and β to store information

about the potential of a branch for us or the opponent. Alpha-cutoff Terminate branch because we already have a better opportunity elsewhere. Beta-cutoff Terminate branch because opponent already has better opportunity elsewhere.

◮ α value of the best choice so far at node along the path

to the root for Max.

◮ β value of the best choice so far at node along the path

to the root for Min.

◮ We can prune a branch

◮ in a Max node if value is > β ◮ in Min node if value is < α

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Alpha Computation

Basic Idea: If a Max node in a descendant branch has a lower value than α, then we can prune that branch in the parent Min node, as it would propagate at most this value.

◮ α stores the highest Max value already computed along

a given branch to the root.

◮ Initially α is −∞. ◮ As soon as a node is evaluated with value v, set α at

the next Max node to α = max(α, v).

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Beta Computation

Basic Idea: Similar to the α case we can prune at descendant Max nodes, if some Min node below has a value greater than β.

◮ β stores the highest Min value already computed along

a given branch.

◮ Initially β is +∞. ◮ As soon as a node is evaluated with value v, set β at

the next Min node to β = min(β, v).

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: α-β-Pruning (Step 0)

4 5 6 7 3 4 1

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: α-β-Pruning (Step 1)

4

α=−∞ β=+∞

5

α=−∞ β=+∞

6 7

α=−∞ β=+∞

3 4 1

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: α-β-Pruning (Step 2)

4 4 5

4>β? α=4 β=+∞ α=−∞ β=+∞

6 7

α=−∞ β=+∞

3 4 1

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: α-β-Pruning (Step 3)

5 5 4 5 6

α=−∞ β=5

7

5<α? α=−∞ β=5 α=−∞ β=+∞

3 4 1

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: α-β-Pruning (Step 4)

5 5 4 5 6 6 7

6>β? α=−∞ β=5 α=−∞ β=5 α=−∞ β=+∞

3 4 1

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: α-β-Pruning (Step 5)

5 5 5 4 5 6 6 7 3

α=5 β=+∞

4

α=5 β=+∞

1

5>β? α=5 β=+∞

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: α-β-Pruning (Step 6)

5 5 5 4 5 6 6 7 3 3 4

3>β? α=5 β=+∞ α=5 β=+∞

1

α=5 β=+∞

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Example: α-β-Pruning (Step 7)

5 5 5 4 5 6 6 7 4 4 3 4 1

4<α? α=5 β=+∞ α=5 β=+∞

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm

Example: Tic-Tac-Toe

Alpha-Beta Pruning

Additional Complications

Multiplayers Extend tree to one layer per player. Calculate and propagate utility vectors that reflect the point of view of each player. Incomplete Knowledge Use expected values, rather than precise values, by calculating possibilities for the next moves. Chance Use stochastic layers, that is after each step we branch by propabilities of the next moves.