G AME T HEORY 2 I NSTRUCTOR : G IANNI A. D I C ARO T HE PROFESSOR S - PowerPoint PPT Presentation

15-382 C OLLECTIVE I NTELLIGENCE – S18 L ECTURE 27: G AME T HEORY 2 I NSTRUCTOR : G IANNI A. D I C ARO

T HE PROFESSOR ’ S DILEMMA Class  Simultaneous move Listen Sleep  Non-cooperative game  Complete information Make 10 6 ,10 6 -10,0 Professor  Imperfect information effort  Solution concept: predict how the game will be played Slack with rational agents 0,-10 0,0 off  Prediction ≡ Solution  Nash: Equilibrium concept Dominant strategies? Nope, if Class listen, and Professor slacks off, Sleep provides a higher payoff! No dominant strategy: best strategy it doesn’t matter what other player’s strategy 15781 Fall 2016: Lecture 22 2

N ASH EQUILIBRIUM (1951)  Can we find an equilibrium also in absence of a dominant strategy?  At equilibrium, each player’s strategy is a best response to strategies of others  Formally, a Nash equilibrium is strategy profile 𝑡 = 𝑡 1 … , 𝑡 𝑜 ∈ 𝑇 𝑜 such that: ′ ∈ 𝑇, 𝑣 𝑗 𝑡 ≥ 𝑣 𝑗 (𝑡 𝑗 ′ , 𝑡 −𝑗 ) ∀𝑗 ∈ 𝑂, ∀𝑡 𝑗 John F. Nash, Nobel Prize in Economics, 1994 15781 Fall 2016: Lecture 22 3

N ASH EQUILIBRIUM  In equilibrium, each player is playing the strategy that is a “ best response ” to the strategies of the other players. No one has an incentive to change strategy given the strategy choices of the others  A NE is an equilibrium where each player’s strategy is optimal given the strategies of all other players .  A Nash Equilibrium exists when there is no unilateral profitable deviation from any of the players involved  Nash Equilibria are self-enforcing strategies: when players are at a Nash Equilibrium they have no desire to move because they will be worse off → Equilibrium in the policy space  Dominant strategy ⟹ Nash equilibrium : All solutions in dominant strategies are also Nash equilibria, but the vice versa is not 15781 Fall 2016: Lecture 22 necessarily (and not usually) true 4

N ASH EQUILIBRIUM Equilibrium is not :  The best possible outcome of the game. Equilibrium in the one-shot prisoners’ dilemma is for both players to confess, which is not the best possible outcome (not Pareto optimal)  A situation where players always choose the same action. Sometimes equilibrium will involve changing action choices ( mixed strategy equilibrium). 15781 Fall 2016: Lecture 22 5

N ASH EQUILIBRIUM  How many Nash equilibria does the Professor’s Dilemma have ? Listen Sleep 10 6 ,10 6 -10,0 Make effort Slack off 0,-10 0,0 ML - SS 15781 Fall 2016: Lecture 22 6

N ASH EQUILIBRIA : H OW DO WE FIND THEM ?  Nash equilibrium: A play of the game where each strategy is a best reply to the given strategy of the other.  Let’s examine all the possible pure strategy profiles and check if for a profile (X,Y) one player could improve its payoff, given the strategy of the other  (M, L)? If Prof plays M, then L is the best reply given M. Neither player can increase its the payoff by choosing a different action o (S,L)? If Prof plays S, S is the best reply given S, not L o (M, S)? If Prof plays M, then L is the best reply given M, not S  (S,S)? If Prof plays S, then S is the best reply given S. Neither player can increase its the payoff by choosing a different action 15781 Fall 2016: Lecture 22 7

N ASH EQUILIBRIUM FOR P RISONER ’ S D ILEMMA Prisoner B Don’t Confess confess Confess Don’t Prisoner A -1,-1 -9,0 Confess 0,-9 -6,-6 15781 Fall 2016: Lecture 22 8

C OORDINATION G AME : S TAG HUNT (Originally from J.J. Rousseau)  Two equilibria at ( stag, stag ) and ( rabbit, rabbit ) → Players' optimal strategy depend on their expectation on what the other player may do.  This game has been used as an analogy for social cooperation, and mutual trust  In Prisoner’s dilemma, the Nash equilibrium corresponds to defect, no cooperate! 9

C OMPETITION G AME  Both players simultaneously choose an integer from 0 to 3  They both win the smaller of the two numbers in points.  In addition, if one player chooses a larger number than the other, then it has to give up two points to the other. Does the (unique) NE at (0,0) make sense? 15781 Fall 2016: Lecture 22 10

R OCK - PAPER - SCISSORS R P S R 0,0 -1,1 1,-1 P 1,-1 0,0 -1,1 S -1,1 1,-1 0,0 Nash equilibrium? Is there a pure strategy as best response? 15781 Fall 2016: Lecture 22 11

R OCK -P APER -S CISSORS  For every pure strategy (X,Y), there R P S is a different strategy choice that increases the payoff of a player R 0,0 -1,1 1,-1  E.g., for strategy (P,R), player B can get a higher payoff playing strategy S instead R P 1,-1 0,0 -1,1  E.g., for strategy (S,R), player A can get a higher payoff playing strategy P instead S S -1,1 1,-1 0,0  No strategy equilibrium can be settled, players have the incentive to No (pure) Nash equilibria: keep switching their strategy Best response: randomize! 15781 Fall 2016: Lecture 22 12

M IXED STRATEGIES  Mixed strategy: a probability distribution over ( pure ) strategies  The mixed strategy of player 𝑗 ∈ 𝑂 is 𝑦 𝑗 , where 𝑦 𝑗 (𝑡 𝑗 ) = Pr[𝑗 plays 𝑡 𝑗 ] (e.g., 𝑦 𝑗 𝑆 = 0.3, 𝑦 𝑗 𝑄 = 0.5, 𝑦 𝑗 𝑇 = 0.2)  The (expected) utility of player 𝑗 ∈ 𝑂 is 𝑜 𝑣 𝑗 𝑦 1 , … , 𝑦 𝑜 = ෍ 𝑣 𝑗 𝑡 1 , … , 𝑡 𝑜 ⋅ ෑ 𝑦 𝑘 (𝑡 𝑘 ) (𝑡 1 ,…,𝑡 𝑜 )∈𝑇 𝑜 𝑘=1 Mixed strategy Pure strategy Utility of pure Joint probability of profile profile strategy the pure strategy profile profile given the mixed profile 15781 Fall 2016: Lecture 22 13

E XERCISE : M IXED NE 1 1 R P S  Player 1 plays 2 , 2 , 0 , player 2 1 1 plays 0, 2 , 2 . What is 𝑣 1 ? R 0,0 -1,1 1,-1 1 1 1  Both players play P 1,-1 0,0 -1,1 3 , 3 , 3 . What is 𝑣 1 ? S -1,1 1,-1 0,0 15781 Fall 2016: Lecture 22 14

E XERCISE : M IXED NE 1 1 1 1 2 , 2 , 0 , player 2 plays 0, 2 , 2 . What is 𝑣 1 ? Player 1 plays + + R P S R 0,0 -1,1 1,-1 P 1,-1 0,0 -1,1 S -1,1 1,-1 0,0 In the second case, because of symmetry, the utility is 15781 Fall 2016: Lecture 22 zero: It’s a zero-sum game 15

M IXED S TRATEGIES E QUILIBRIUM IS N ASH  The mixed strategy profile 𝑦 ∗ in a strategic game is a mixed strategy Nash equilibrium if ∗ , 𝑦 −𝑗 ∗ ∗ ∀ 𝑦 𝑗 and 𝑗 𝑣 𝑗 𝑦 𝑗 ≥ 𝑣 𝑗 𝑦 𝑗 , 𝑦 −𝑗  𝑣 𝑗 𝑦 is player 𝑗 ’s expected utility with mixed strategy profile 𝑦  → Same definition as in the case f pure strategies, where 𝑣 𝑗 was the utility of a pure strategy instead of a mixed strategy 15781 Fall 2016: Lecture 22 16

M IXED S TRATEGIES N ASH E QUILIBRIUM ∗ is  Using best response functions, 𝑦 ∗ is a mixed strategy NE iff 𝑦 𝑗 the best response for every player 𝑗 .  If a mixed strategy 𝑦 ∗ is a best response, then each of the pure strategies in the mix must be best response : they must yield the same expected payoff (otherwise it would just make sense to choose the one with the better payoff)  → If a mixed strategy is a best response for player 𝑗 , then the player must be indifferent among the pure strategies in the mix  E.g., in the RPS game, if the mixed strategy of player 𝑗 assigns non-zero probabilities p R for playing R and p P for playing P, then 𝑗 ’s expected utility for playing R or P has to be the same 15781 Fall 2016: Lecture 22 17

E XERCISE : M IXED NE  Which is a NE? R P S 1 1 1 1 1. 2 , 2 , 0 , 2 , 2 , 0 R 0,0 -1,1 1,-1 1 1 1 1 2. 2 , 2 , 0 , 2 , 0, 2 P 1,-1 0,0 -1,1 1 1 1 1 1 1 3. 3 , 3 , 3 , 3 , 3 , 3 S -1,1 1,-1 0,0 1 2 2 1 4. 3 , 3 , 0 , 3 , 0, 3 Any other NE? 15781 Fall 2016: Lecture 22 18

N ASH ’ S T HEOREM  Theorem [Nash, 1950]: In any game with finite number of strategies there exists at least one (possibly mixed) Nash equilibrium Player B Left Right 1,2 0,4 Up Player A 0,5 3,2 Down This game has no pure strategy Nash equilibria but it does have a Nash 15781 Fall 2016: Lecture 22 equilibrium in mixed strategies. How is it computed? 19

C OMPUTATION OF MS NE Player B Left Right 1,2 0,4 Up Player A 0,5 3,2 Down Player A plays Up with probability p U and plays Down with probability 1-p U Player B plays Left with probability p L and plays Right with probability 1-p L. 15781 Fall 2016: Lecture 22 20

C OMPUTATION OF MS NE Player B L, p L R,1- p L U, p U 1,2 0,4 Player A D,1- p U 0,5 3,2 15781 Fall 2016: Lecture 22 21

C OMPUTATION OF MS NE Player B L, p L R,1- p L U, p U 1,2 0,4 Player A D,1- p U 0,5 3,2 If B plays Left, its expected utility is p   p 2 5 1 ( ) U U 15781 Fall 2016: Lecture 22 22

C OMPUTATION OF MS NE Player B L, p L R,1- p L U, p U 1,2 0,4 Player A D,1- p U 0,5 3,2 If B plays Right, its expected utility is p   p 4 2 1 ( ). U U 15781 Fall 2016: Lecture 22 23

C OMPUTATION OF MS NE Player B L, p L R,1- p L U, p U 1,2 0,4 Player A D,1- p U 0,5 3,2 p   p  p   p 2 5 1 ( ) 4 2 1 ( ) If then U U U U B would play only Left, which would be a pure strategy. But there are no (pure) Nash equilibria in which B plays only Left. 15781 Fall 2016: Lecture 22 24