Models of Language Evolution Session 03 : Evolutionary Game Theory: - - PowerPoint PPT Presentation

Models of Language Evolution

Session 03: Evolutionary Game Theory: Games & Stable Outcomes Michael Franke

Seminar f¨ ur Sprachwissenschaft Eberhard Karls Universit¨ at T¨ ubingen

Classical GT Evolutionary GT

Course Overview (tentative) date content 20-4 MoLE: Aims & Challenges 27-4 Evolutionary Game Theory 1: Statics 04-5 Evolutionary Game Theory 2: Macro-Dynamics 11-5 Evolutionary Game Theory 3: Micro-Dynamics 18-5 Multi-Agent Systems: The A-Life Approach to LE 25-5 Communication, Cooperation & Relevance 01-6 Combinatoriality, Compositionality & Recursion 08-6 Evolution of Semantic Meaning & Pragmatic Strategies 15-6 Pentecost — no class 22-6 work on student projects 29-6 work on student projects 06-7 work on student projects 13-7 presentations 20-7 presentations

2 / 20

Classical GT Evolutionary GT

Today’s Session

1 (classical) game theory

• static games
• (strict) Nash equilibria
• in pure strategies
• in mixed strategies

2 games on populations

• symmetric
• asymmetric

3 evolutionarily stable states

• in symmetric populations
• in asymmetric populations
• existence, uniqueness, some properties
• relation to Nash equilibrium

3 / 20

Classical GT Evolutionary GT

Game Theory

• abstract mathematical tools for modeling and analyzing multi-agent

interaction

• since 1940, classical game theory

(von Neumann and Morgenstern)

• (mostly) assumes perfectly rational agents
• initially promised to be a unifying formal foundation for all social sciences
• most central notion: Nash equilibrium
• widely used tool in economics still
• Nobel laureates:
• Nash, Harsanyi & Selten (1994)
• Aumann & Schelling (2006)
• recently, connection to epistemic logic

(Harsanyi, Aumann, Stalnaker)

• recently, connection to linguistics

(Lewis, Skyrms, J¨ ager, van Rooij)

• since 1970: evolutionary game theory

(Maynard-Smith, Prize)

• studies boundedly-rational agents
• many applications in biology
• most central notions: evolutionary stability & replicator dynamics
• since 1990, behavioral game theory

(Selten, Camerer)

• studies interactive decision making in the lab

4 / 20

Classical GT Evolutionary GT

Static Game (Intuition) A static game is the specification of a situation in which two or more agents choose from a fixed set of options, together with an indication how preferable any possible outcome (combination of individual actions) is to each agent. Definition (Static Game (a.k.a. Strategic Game, Normal Form Game)) A static game is a triple N, (A)i∈N, (U)i∈N with:

• N — set of players
• Ai — actions available to player i
• Ui : ×j∈NAj → R — player i’s utility function, with:
• ×j∈NAj — set of action profiles (outcomes).

(Alternatively, we will occasionally consider the Ui to be suitable matrices.)

5 / 20

Classical GT Evolutionary GT

Example (Battle of the Sexes)

• wife & husband want to go to the opera
• N = {1, 2}

(wife ↔ 1)

• there are two operas showing this night
• A1 = A2 =
• aBach, aStravinsky
• wife & husband cannot communicate where to go
• they want to be together,

but:

• wife prefers Bach
• husband Stravinsky

U1 =

• 3

1

• U2 =
• 1

3

• alternative notation in table:

aBach aStrav aBach 3,1 0,0 aStrav 0,0 1,3 (player 1 is assumed to choose the row; her utils are listed first)

6 / 20

Classical GT Evolutionary GT

Example (Prisoner’s Dilemma)

• 2 prisoner’s held captive and charged with a crime
• both can either confess (ac) or deny (ad)
• they cannot communicate, and must make their choice independently
• if both confess, they both go to prison for a short time only
• if both deny, they both go to prison for a longer time
• if one confesses and the other denies, the confesser goes to jail for a

very long time, while the denier goes free U1 =

• 2

3 1

• U2 =
• 2

3 1

• ac

ad ac 2,2 0,3 ad 3,0 1,1

7 / 20

Classical GT Evolutionary GT

Example (Hawks & Doves)

• conflict over a food resource
• hawks fight, doves share

ahawk adove ahawk 1,1 7,2 adove 2,7 3,3 Example (Matching Pennies)

• one’s victory is the other’s defeat

aheads atails aheads 1,0 0,1 atails 0,1 1,0

8 / 20

Classical GT Evolutionary GT

Example (Coordination)

• perfectly aligned interests
• but coordination problem

astay ago astay 1,1 0,0 ago 0,0 1,1 Example (Anti-Coordination)

• perfectly aligned interests
• but coordination problem

astay ago astay 0,0 1,1 ago 1,1 0,0

9 / 20

Classical GT Evolutionary GT

Games vs. Strategies vs. Solutions

• games: are models of a choice situation
• strategies: model actual agent behavior
• solution concepts: capture particular behavior:

good, optimal, rational, stable (. . . ) Different (Classical) Solutions for Static Games Nash equilibrium: steady state Iterated Strict Dominance: step-by-step pruning of game Rationalizability: common belief in rationality (we only look at Nash equilibrium in this course)

10 / 20

Classical GT Evolutionary GT

Nash Equilibrium (Intuition) A Nash equilibrium is an arrangement of strategies, one for each player, such that no player would benefit from unilateral deviation (i.e., no player would be better off doing something else if everybody else keeps doing the same thing). Definition (Nash Equilibrium (in Pure Strategies)) A Nash equilibrium in pure strategies is an action profile a1, . . . , ai, . . . , an such that for all i ∈ N and a′

i ∈ Ai:

Ui(a1, . . . , ai, . . . , an) ≥ Ui(

• a1, . . . , a′

i, . . . , an

) . It is strict if for all i ∈ N and a′

i ∈ Ai, a′ i = ai:

Ui(a1, . . . , ai, . . . , an) > Ui(

• a1, . . . , a′

i, . . . , an

) .

• not all games have pure nes

(e.g. Matching Pennies)

• all games have nes in mixed strategies

11 / 20

Classical GT Evolutionary GT

Definition (Mixed Strategies) A mixed strategy for player i is a vector p ∈ ∆k, where k = |Ai|, and ∆k =

• (p1, p2, . . . , pk) ∈ Rk | pi ≥ 0 &

k

∑

i=1

pi = 1

• .

(∆k — the set of all probability vectors of size k alt.: the (k − 1) unit simplex in Rk )

• pure strategies are equivalently characterized as unit vectors:

(0, . . . , 0, 1, 0, . . . , 0)

• we can think of a mixed strategy either
• as a deliberate randomization, or
• as a probabilistic belief about opponent behavior, or
• as a population aggregate

12 / 20

Classical GT Evolutionary GT

Definition (Expected Utility) Fix a static 2-player game. The expected utility for player i playing p when j is playing q is: EUi( pi, q) = pi · Ui q . Reminder

• matrix multiplication: if A is an m × o matrix, and B is a o × n matrix,

then C = AB is an m × n matrix with: Cij =

p

∑

k=1

Aik × Bkj

• dot product: if

a and b are n-place vectors, then:

• a ·

b =

n

∑

i=1

ai × bi

13 / 20

Classical GT Evolutionary GT

Definition (Best Response) Player i’s best response to player j playing q is any (mixed) strategy p that maximizes player i’s expected utility given q: BRi(

• q) = arg max
• p∈∆(k) EUi(

p, q) . (not playing a best response is irrational) Definition (Nash Equilibrium (in Mixed Strategies)) The (mixed) strategy profile p = p1, p2 is a (mixed) Nash equilibrium if for both players i: EUi( pi, pj) ≥ EUi( x, pj) for all x ∈ ∆(|Ai|) . It is a (strict) Nash equilibrium if for both players i: EUi( pi, pj) > EUi( x, pj) for all x ∈ ∆(|Ai|) ; x = pi .

14 / 20

Classical GT Evolutionary GT

Definition (Symmetric vs. Asymmetric Games) A 2-player game is symmetric iff:

• A1 = A2,
• U1 = UT

2 , and

(utilities as matrices)

• players cannot distinguish their roles.

Otherwise it is called asymmetric. Nash Equilibrium in Symmetric Games A mixed strategy p is a symmetric Nash equilibrium iff for all other possible strategies q: EU( p, p) ≥ EU( p, q) . It is strict if the inequality is strict for all q = p.

15 / 20

Classical GT Evolutionary GT

Mean-Field Population

• (nearly) infinite populations for each distinguishable role
• each population is entirely homogeneous
• agents play pure strategies
• each agent interacts purely at random with agents from:
• the single population

(symmetric games)

• the other population

(asymmetric games)

• strategy update are rare

16 / 20

Classical GT Evolutionary GT

Expected Utility in Mean-Field Populations (Symmetric)

• let ni be the number of agents playing pure action ai
• let n be the size of the population
• population aggregate is a mixed strategy

p where: pi = ni n

• if population is large enough, an agent who plays ai has expected utility

as defined above: EU(ai, p) = ai · Ui p (interpret ai as the i-th unit vector) Expected Utility in Mean-Field Populations (Asymmetric)

• same, but with frequencies from respective other population

17 / 20

Classical GT Evolutionary GT

Evolutionary Stability (Intuition) A strategy p is evolutionarily stable if a population that consists entirely/mostly of p cannot be invaded by any minority of mutants playing strategy q. Definition (Evolutionarily Stable Strategy (Symmetric)) A strategy p is evolutionarily stable iff: (i) EU( p, p) ≥ EU(

• q,

p) for all q, and (ii) if EU( p, p) = EU(

• q,

p) for some q = p, then EU( p, q) > EU(

• q,

q) . Connection with ne (Symmetric Case)

• strict-nes

⊂ esss ⊂ nes Existence and Uniqueness, Properties

• a symmetric game can have zero, one or more esss (in mixed strategies)
• if a symmetric game has a (non-degenerate) mixed ess, then its the only

ess

18 / 20

Classical GT Evolutionary GT

esss of Asymmetric Games

• in a 2-population model a mutant would not play against itself
• we only need to compare incumbent and mutant
• in asymmetric games: esss= strict-nes

(Selten 1980) Existence, Uniqueness, Properties

• an asymmetric game can have zero, one or more esss
• all asymmetric esss contain only pure strategies

Reinhard Selten (1980). “A Note On Evolutionarily Stable Strategies in Asymmetric Animal Conflicts”. In: Journal of Theoretical Biology 84, pp. 93–101

19 / 20

Classical GT Evolutionary GT

Homework

• Look at the “hawks & doves” game, given on slide 8. Find all nes of

this game and determine if they are strict:

1 pure asymmetric, 2 pure symmetric, 3 mixed asymmetric, and 4 mixed symmetric.

Does it also have esss symmetric and asymmetric? If so, which? Reading

• Sections 1 & 2 of:

Gerhard J¨ ager (2004). “Evolutionary Game Theory for Linguists. A Primer”. Unpublished manuscript, Stanford/Potsdam

• Sections 2.1, 2.2, 2.3, 3.1, 3.2 of:

Gy¨

• rgy Szab´
• and G´

abor F´ ath (2007). “Evolutionary Games on Graphs”. In: Physics Reports 446, pp. 97–216

20 / 20