Adversarial Search Volker Sorge Intro to AI: Problem of Games - PowerPoint PPT Presentation

Intro to AI: Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning Adversarial Search Volker Sorge

Intro to AI: Problem of Games Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning ◮ Discrete games, that is, games with clearly defined single actions or moves, can be represented as search problems. ◮ Examples: Chess, Backgammon, Poker ◮ Search tree needs to contain moves for both protagonist and antagonist.

Intro to AI: Basic Notions Lecture 4 Volker Sorge ◮ Protagonist called Max MiniMax Algorithm ◮ Antagonist called Min Example: Tic-Tac-Toe Alpha-Beta ◮ States are valid positions of a game with start state as Pruning the initial position in the game ◮ Actions are the legal moves in the game. ◮ Terminal nodes are final states of the game (goal states are usually winning states of Max). ◮ Utility function u : V → Z : Assigns a numeric value to each state s .  > 0 if Max wins,  u ( s ) < 0 if Min wins, = 0 in case of a draw.  Z

Intro to AI: MiniMax Idea Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta ◮ Generate the entire search tree for the game. Pruning ◮ We have two types of layers in the tree: ◮ Max layer: It is Max’s turn. ◮ Min layer: It is Min’s turn. ◮ Compute utility function for all terminal nodes. ◮ Propagate utility values to the root: ◮ In Max layer: take maximum utility value of children. ◮ In Min layer: take minimum utility value of children.

Intro to AI: Algorithm: MiniMax Lecture 4 Volker Sorge Minimax ; Input : Start state s , Set of terminal states T , Utility function u , MiniMax Algorithm Flag MAX or MIN Example: Tic-Tac-Toe Output : Utility value u ( s ) Alpha-Beta 1 begin Pruning if s ∈ T then 2 return u ( s ) 3 end 4 ; 5 let { v 1 , . . . , v n } be children of s ; 6 if MAX then 7 return 8 max { MiniMax ( v 1 , MIN ) , . . . , MiniMax ( v n , MIN ) } end 9 else 10 return 11 min { MiniMax ( v 1 , MAX ) , . . . , MiniMax ( v n , MAX ) } end 12 13 end min max

Intro to AI: Algorithm: MiniMax Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Observe that Pruning ◮ the algorithm computes the utility value for the start node. If we want to make a decision for the next move in a particular state s we need to run it on all children of s as MIN nodes. ◮ often it is easier to implement a test for a terminal instead of providing the set of goal states. For example, consider Chess vs. Tic-Tac-Toe

Intro to AI: Example: Tic-Tac-Toe Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning

Intro to AI: Example: Tic-Tac-Toe Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning

Intro to AI: Example: Tic-Tac-Toe Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning

Intro to AI: Example: Tic-Tac-Toe Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning

Intro to AI: Example: Tic-Tac-Toe Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning

Intro to AI: Example: Tic-Tac-Toe Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning

Intro to AI: Analysis Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning ◮ We do not need to consider completeness and optimality as these would depend on the oponents strategy. ◮ Time complexity is O ( b m ), where m is the maximum depth of the tree. ◮ Space complexity is O ( bm ) as we effectively use a depth first approach.

Intro to AI: Pruning Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe ◮ In any interesting game MiniMax is unrealistic. Alpha-Beta Pruning ◮ Try to restrict computation by pruning the search tree. ◮ Determine early that a particular branch will not contribute to a winning strategy ◮ In particular, stop descending into branches if it is obvious that ◮ Max would never do a move, or ◮ Min would never do a move. ◮ This assumes that Min plays rational!

Intro to AI: Alpha-Beta Pruning Lecture 4 Volker Sorge MiniMax ◮ We define two values α and β to store information Algorithm Example: about the potential of a branch for us or the opponent. Tic-Tac-Toe Alpha-Beta Alpha-cutoff Terminate branch because we already Pruning have a better opportunity elsewhere. Beta-cutoff Terminate branch because opponent already has better opportunity elsewhere. ◮ α value of the best choice so far at node along the path to the root for Max. ◮ β value of the best choice so far at node along the path to the root for Min. ◮ We can prune a branch ◮ in a Max node if value is > β ◮ in Min node if value is < α

Intro to AI: Alpha Computation Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Basic Idea: If a Max node in a descendant branch has a Alpha-Beta Pruning lower value than α , then we can prune that branch in the parent Min node, as it would propagate at most this value. ◮ α stores the highest Max value already computed along a given branch to the root. ◮ Initially α is −∞ . ◮ As soon as a node is evaluated with value v , set α at the next Max node to α = max( α, v ).

Intro to AI: Beta Computation Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Basic Idea: Similar to the α case we can prune at Alpha-Beta Pruning descendant Max nodes, if some Min node below has a value greater than β . ◮ β stores the highest Min value already computed along a given branch. ◮ Initially β is + ∞ . ◮ As soon as a node is evaluated with value v , set β at the next Min node to β = min( β, v ).

Intro to AI: Example: α - β -Pruning (Step 0) Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Pruning 4 5 6 7 3 4 0 1

Intro to AI: Example: α - β -Pruning (Step 1) Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe α = −∞ Alpha-Beta β =+ ∞ Pruning α = −∞ β =+ ∞ α = −∞ β =+ ∞ 4 5 6 7 3 4 0 1

Intro to AI: Example: α - β -Pruning (Step 2) Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe α = −∞ Alpha-Beta β =+ ∞ Pruning α = −∞ β =+ ∞ 4 4 >β ? α =4 β =+ ∞ 4 5 6 7 3 4 0 1

Intro to AI: Example: α - β -Pruning (Step 3) Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe α = −∞ Alpha-Beta β =+ ∞ Pruning 5 5 <α ? α = −∞ β =5 5 α = −∞ β =5 4 5 6 7 3 4 0 1

Intro to AI: Example: α - β -Pruning (Step 4) Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe α = −∞ Alpha-Beta β =+ ∞ Pruning 5 α = −∞ β =5 5 6 6 >β ? α = −∞ β =5 4 5 6 7 3 4 0 1

Intro to AI: Example: α - β -Pruning (Step 5) Lecture 4 Volker Sorge MiniMax Algorithm 5 Example: 5 >β ? Tic-Tac-Toe α =5 Alpha-Beta β =+ ∞ Pruning 5 α =5 β =+ ∞ 5 6 α =5 β =+ ∞ 4 5 6 7 3 4 0 1

Intro to AI: Example: α - β -Pruning (Step 6) Lecture 4 Volker Sorge MiniMax Algorithm 5 Example: Tic-Tac-Toe α =5 Alpha-Beta β =+ ∞ Pruning 5 α =5 β =+ ∞ 5 6 3 3 >β ? α =5 β =+ ∞ 4 5 6 7 3 4 0 1

Intro to AI: Example: α - β -Pruning (Step 7) Lecture 4 Volker Sorge MiniMax Algorithm 5 Example: Tic-Tac-Toe α =5 Alpha-Beta β =+ ∞ Pruning 5 4 4 <α ? α =5 β =+ ∞ 5 6 4 4 5 6 7 3 4 0 1

Intro to AI: Additional Complications Lecture 4 Volker Sorge MiniMax Algorithm Example: Tic-Tac-Toe Alpha-Beta Multiplayers Extend tree to one layer per player. Calculate Pruning and propagate utility vectors that reflect the point of view of each player. Incomplete Knowledge Use expected values, rather than precise values, by calculating possibilities for the next moves. Chance Use stochastic layers, that is after each step we branch by propabilities of the next moves.