Adversarial Search CE417: Introduction to Artificial Intelligence Sharif University of Technology Spring 2018 Soleymani “ Artificial Intelligence: A Modern Approach ” , 3 rd Edition, Chapter 5 Most slides have been adopted from Klein and Abdeel, CS188, UC Berkeley.
Outline Game as a search problem Minimax algorithm 𝛽 - 𝛾 Pruning: ignoring a portion of the search tree Time limit problem Cut off & Evaluation function 2
Games as search problems Games Adversarial search problems (goals are in conflict) Competitive multi-agent environments Games in AI are a specialized kind of games (in the game theory) 3
Adversarial Games 4
Types of Games Many different kinds of games! Axes: Deterministic or stochastic? One, two, or more players? Zero sum? Perfect information (can you see the state)? Want algorithms for calculating a strategy (policy) which recommends a move from each state 5
Zero-Sum Games Zero-Sum Games General Games Agents have opposite utilities Agents have independent utilities (values on outcomes) (values on outcomes) Lets us think of a single value that one Cooperation, indifference, competition, maximizes and the other minimizes and more are all possible Adversarial, pure competition More later on non-zero-sum games 6
Primary assumptions We start with these games: T wo-player Turn taking agents act alternately Zero-sum agents ’ goals are in conflict: sum of utility values at the end of the game is zero or constant Deterministic Perfect information fully observable Examples: Tic-tac-toe, chess, checkers 7
Deterministic Games Many possible formalizations, one is: States: S (start at s 0 ) Players: P={1...N} (usually take turns) Actions:A (may depend on player / state) Transition Function: SxA S TerminalTest: S {t,f} Terminal Utilities: SxP R Solution for a player is a policy: S A 8
Single-Agent Trees 8 2 0 … 2 6 … 4 6 9
Value of a State Non-Terminal Value of a state: The States: best achievable outcome (utility) from that state 8 2 0 … 2 6 … 4 6 Terminal States: 10
Adversarial Search 11
Adversarial Game Trees -20 -8 … -18 -5 … -10 +4 -20 +8 12
Minimax Values States Under Agent ’ s Control: States Under Opponent ’ s Control: -8 -5 -10 +8 Terminal States: 13
Tic-Tac-Toe Game Tree 14
Game tree (tic-tac-toe) Two players: 𝑄 1 and 𝑄 2 ( 𝑄 1 is now searching to find a good move) Zero-sum games: 𝑄 1 gets 𝑉(𝑢) , 𝑄 2 gets 𝐷 − 𝑉(𝑢) for terminal node 𝑢 𝑄 𝑄 1 : 𝑌 1 𝑄 2 : 𝑃 1-ply = half move 𝑄 2 𝑄 1 𝑄 2 Utilities from the point of view of 𝑄 1 15
Optimal play Opponent is assumed optimal Minimax function is used to find the utility of each state. MAX/MIN wants to maximize/minimize the terminal payoff MAX gets 𝑉(𝑢) for terminal node 𝑢 16
Adversarial Search (Minimax) Minimax search: A state-space search tree Players alternate turns max 5 Compute each node ’ s minimax value: the best achievable utility against a min 5 2 rational (optimal) adversary 8 2 5 6 Terminal values: part of the game 17
Minimax 𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡, 𝑁𝐵𝑌) 𝑗𝑔 𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡) 𝑛𝑏𝑦 𝑏∈𝐵𝐷𝑈𝐽𝑃𝑂𝑇 𝑡 𝑁𝐽𝑂𝐽𝑁𝐵𝑌(𝑆𝐹𝑇𝑉𝑀𝑈(𝑡, 𝑏)) 𝑄𝑀𝐵𝑍𝐹𝑆 𝑡 = 𝑁𝐵𝑌 𝑁𝐽𝑂𝐽𝑁𝐵𝑌 𝑡 = 𝑛𝑗𝑜 𝑏∈𝐵𝐷𝑈𝐽𝑃𝑂𝑇 𝑡 𝑁𝐽𝑂𝐽𝑁𝐵𝑌(𝑆𝐹𝑇𝑉𝑀𝑈 𝑡, 𝑏 ) 𝑄𝑀𝐵𝑍𝐹𝑆 𝑡 = 𝑁𝐽𝑂 Utility of being in state s 𝑁𝐽𝑂𝐽𝑁𝐵𝑌(𝑡) shows the best achievable outcome of being in state 𝑡 (assumption: optimal opponent) 3 3 2 2 18
Minimax (Cont.) Optimal strategy: move to the state with highest minimax value Best achievable payoff against best play Maximizes the worst-case outcome for MAX It works for zero-sum games 19
Minimax Properties max min 10 10 9 100 Optimal against a perfect player. Otherwise? 20
Minimax Implementation def max-value(state): def min-value(state): initialize v = - ∞ initialize v = + ∞ for each successor of state: for each successor of state: v = max(v, min-value(successor)) v = min(v, max-value(successor)) return v return v 21
Minimax Implementation (Dispatch) def value(state): if the state is a terminal state: return the state ’ s utility if the next agent is MAX: return max-value(state) if the next agent is MIN: return min-value(state) def max-value(state): def min-value(state): initialize v = - ∞ initialize v = + ∞ for each successor of state: for each successor of state: v = max(v, value(successor)) v = min(v, value(successor)) return v return v 22
Minimax algorithm Depth first search function 𝑁𝐽𝑂𝐽𝑁𝐵𝑌_𝐸𝐹𝐷𝐽𝑇𝐽𝑃𝑂(𝑡𝑢𝑏𝑢𝑓) returns 𝑏𝑜 𝑏𝑑𝑢𝑗𝑝𝑜 𝑏∈𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) 𝑁𝐽𝑂_𝑊𝐵𝑀𝑉𝐹(𝑆𝐹𝑇𝑉𝑀𝑈(𝑡𝑢𝑏𝑢𝑓, 𝑏)) max return function 𝑁𝐵𝑌_𝑊𝐵𝑀𝑉𝐹(𝑡𝑢𝑏𝑢𝑓) returns 𝑏 𝑣𝑢𝑗𝑚𝑗𝑢𝑧 𝑤𝑏𝑚𝑣𝑓 if 𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡𝑢𝑏𝑢𝑓) then return 𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡𝑢𝑏𝑢𝑓) 𝑤 ← −∞ for each 𝑏 in 𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) do 𝑤 ← 𝑁𝐵𝑌(𝑤, 𝑁𝐽𝑂_𝑊𝐵𝑀𝑉𝐹(𝑆𝐹𝑇𝑉𝑀𝑈𝑇(𝑡𝑢𝑏𝑢𝑓, 𝑏))) return 𝑤 function 𝑁𝐽𝑂_𝑊𝐵𝑀𝑉𝐹(𝑡𝑢𝑏𝑢𝑓) returns 𝑏 𝑣𝑢𝑗𝑚𝑗𝑢𝑧 𝑤𝑏𝑚𝑣𝑓 if 𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡𝑢𝑏𝑢𝑓) then return 𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡𝑢𝑏𝑢𝑓) 𝑤 ← ∞ for each 𝑏 in 𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) do 𝑤 ← 𝑁𝐽𝑂(𝑤, 𝑁𝐵𝑌_𝑊𝐵𝑀𝑉𝐹(𝑆𝐹𝑇𝑉𝑀𝑈𝑇(𝑡𝑢𝑏𝑢𝑓, 𝑏))) return 𝑤 23
Properties of minimax Complete?Yes (when tree is finite) Optimal?Yes (against an optimal opponent) Time complexity: 𝑃(𝑐 𝑛 ) Space complexity: 𝑃(𝑐𝑛) (depth-first exploration) For chess, 𝑐 ≈ 35 , 𝑛 > 50 for reasonable games Finding exact solution is completely infeasible 24
Game Tree Pruning 25
Pruning Correct minimax decision without looking at every node in the game tree α - β pruning Branch & bound algorithm Prunes away branches that cannot influence the final decision 26
α - β pruning example 27
α - β pruning example 28
α - β pruning example 29
α - β pruning example 30
α - β pruning example 31
α - β pruning Assuming depth-first generation of tree We prune node 𝑜 when player has a better choice 𝑛 at (parent or) any ancestor of 𝑜 Two types of pruning (cuts): pruning of max nodes ( α -cuts) pruning of min nodes ( β -cuts) 32
Alpha-Beta Pruning General configuration (MIN version) We ’ re computing the MIN-VALUE at some node n MAX We ’ re looping over n ’ s children MIN Who cares about n ’ s value? MAX a Let a be the best value that MAX can get at any choice point along the current path from the root If n becomes worse than a , MAX will avoid it, so we MAX can stop considering n ’ s other children (it ’ s already bad enough that it won ’ t be played) MIN n MAX version is symmetric 33
α - β pruning (an other example) 3 ≤ 2 3 2 ≥ 5 5 1 34
Why is it called α - β? α : Value of the best (highest) choice found so far at any choice point along the path for MAX 𝛾 : Value of the best (lowest) choice found so far at any choice point along the path for MIN Updating α and 𝛾 during the search process For a MAX node once the value of this node is known to be more than the current 𝛾 ( v ≥ 𝛾 ), its remaining branches are pruned. For a MIN node once the value of this node is known to be less than the current 𝛽 ( v ≤ 𝛽 ), its remaining branches are pruned. 35
Alpha-Beta Implementation α : MAX ’ s best option on path to root β : MIN ’ s best option on path to root def max-value(state, α , β ): def min-value(state , α , β ): initialize v = - ∞ initialize v = + ∞ for each successor of state: for each successor of state: v = max(v, value(successor, α , β )) v = min(v, value(successor, α , β )) if v ≥ β return v if v ≤ α return v α = max( α , v) β = min( β , v) return v return v 36
Recommend
More recommend
