Models of Language Evolution Session 04 : Evolutionary Game Theory: - - PowerPoint PPT Presentation

▶
models of language evolution
SMART_READER_LITE
LIVE PREVIEW

Models of Language Evolution Session 04 : Evolutionary Game Theory: - - PowerPoint PPT Presentation

May 21, 2023 144 likes •434 views

Models of Language Evolution Session 04 : Evolutionary Game Theory: Evolutionary Dynamics Michael Franke Seminar f ur Sprachwissenschaft Eberhard Karls Universit at T ubingen Replicator Dynamics Other Evolutionary Dynamics Folk

slide-1
SLIDE 1

Models of Language Evolution

Session 04: Evolutionary Game Theory: Evolutionary Dynamics Michael Franke

Seminar f¨ ur Sprachwissenschaft Eberhard Karls Universit¨ at T¨ ubingen

slide-2
SLIDE 2

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Course Overview (tentative) date content 20-4 MoLE: Aims & Challenges 27-4 Evolutionary Game Theory 1: Statics 04-5 Evolutionary Game Theory 2: Macro-Dynamics 11-5 Guest Lecture by Gerhard J¨ ager 18-5 egt 3: Micro-Dynamics & Multi-Agent Systems 25-5 Communication, Cooperation & Relevance 01-6 Combinatoriality, Compositionality & Recursion 08-6 Evolution of Semantic Meaning & Pragmatic Strategies 15-6 Pentecost — no class 22-6 work on student projects 29-6 work on student projects 06-7 work on student projects 13-7 presentations 20-7 presentations

2 / 28

slide-3
SLIDE 3

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Last Session

1 (classical) game theory

  • static games
  • (strict) Nash equilibria
  • in pure strategies
  • in mixed strategies

2 games on populations

  • symmetric
  • asymmetric

3 evolutionarily stable states

  • in symmetric populations
  • in asymmetric populations
  • existence, uniqueness, some properties
  • relation to Nash equilibrium

3 / 28

slide-4
SLIDE 4

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Today’s Session

1 replicator dynamics

  • for matrix games
  • for bi-matrix games

2 other evolutionary dynamics

(just for comparison)

3 “Folk Theorem of Evolutionary Game Theory”

  • connection evolutionary dynamics with:

1

Nash equilibrium

2

evolutionary stability

fixed points of dynamics are static solutions? macro-dynamics are mean field of the micro-dynamics?

Static Solutions

Nash equilibrium evolutionary stability . . .

Marco-Dynamics (Population-Level)

replicator dynamics best response dynamics . . .

Micro-Dynamics (Agent-Based)

imitate the best conditional imitation reinforcement learning . . . 4 / 28

slide-5
SLIDE 5

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics: Intuition (Biologically, Agent-Based)

  • offspring inherit strategy from their single parent
  • reproductive success ∼ payoff for individual playing against population

NB: (later in this course: different agent-based motivation) Replicator Dynamics: From Intuition to Formalization In a population with pure strategy distribution p = (p1, . . . , pn), the per capita growth rate

˙ pi/pi is given by:

˙ pi pi = fitness of type i − average fitness in population .

5 / 28

slide-6
SLIDE 6

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Definition (Replicator Dynamics) (one population, continuous) Consider a mixed strategy p = (p1, . . . , pn) as a population distribution of pure strategies in a symmetric game. The continuous-time replicator dynamics defines the continuous change in the proportion of agents playing strategy i as: ˙ pi = pi [(U p)i − p · U p] . Definition (Replicator Dynamics) (one population, discrete) We look at a population distribution at concrete, discreet points in time.

  • p(t) = (p1, . . . , pn) is the population distribution of pure strategies in a

symmetric game at time t. Then the discrete-time replicator dynamics defines the proportion of agents playing strategy i at t + 1 as: pi(t + 1) = pi(t) ×

  • i · U

p(t)

  • p(t) · U

p(t) . (NB: i is the unit vector with a single 1 at position i)

6 / 28

slide-7
SLIDE 7

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics One-Population Time Series Coordination: U =

  • 1

1

  • 7 / 28
slide-8
SLIDE 8

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics One-Population Time Series Prisoner’s Dilemma: U =

  • 2

3 1

  • 8 / 28
slide-9
SLIDE 9

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics One-Population Time Series Anti-Coordination: U =

  • 1

1

  • 9 / 28
slide-10
SLIDE 10

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics One-Population Time Series Hawks & Doves: U =

  • 1

7 2 3

  • 10 / 28
slide-11
SLIDE 11

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Source Code for Plots

11 / 28

slide-12
SLIDE 12

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Definition (Replicator Dynamics) (two populations, continuous) Given mixed strategies p and q as the population distributions of pure strategies in an asymmetric game, the continuous-time replicator dynamics is: ˙ pi = pi [(U1 q)i − p · U1 q] , ˙ qi = qi [(U2 p)i − q · U2 p] . Definition (Replicator Dynamics) (two populations, discrete) With p(t) and q(t) population distributions at time t, the discrete-time replicator dynamics is given as: pi(t + 1) = pi ×

  • i · U1

q(t)

  • p(t) · U1

q(t) , qi(t + 1) = qi ×

  • i · U2

p(t)

  • q(t) · U2

p(t) .

12 / 28

slide-13
SLIDE 13

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics Two-Population Dynamic Field Coordination: U1,2 =

  • 1

1

  • 13 / 28
slide-14
SLIDE 14

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics Two-Population Dynamic Field Prisoner’s Dilemma: U1,2 =

  • 2

3 1

  • 14 / 28
slide-15
SLIDE 15

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics Two-Population Dynamic Field Anti-Coordination: U1,2 =

  • 1

1

  • 15 / 28
slide-16
SLIDE 16

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics Two-Population Dynamic Field Hawks & Doves: U1,2 =

  • 1

7 2 3

  • 16 / 28
slide-17
SLIDE 17

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Source Code for Plots

17 / 28

slide-18
SLIDE 18

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Best Response Dynamics: Intuition Every now and then a small fraction of a population changes its strategy and adopts a best response to the population average. Definition (Best Response) (recap) Player i’s best response to player j playing q is any (mixed) strategy p that maximizes player i’s expected utility given q: BRi(

  • q) = arg max
  • p∈∆(k) EUi(

p, q) . Definition (Best Response Dynamics) (one population, continuous) Fix a mixed strategy p = (p1, . . . , pn) as a population distribution of pure strategies in a symmetric game. Then the continuous-time best response dynamics is given as: ˙

  • p = BR(

p) − p . (NB: BR( p) not necessarily a unique mixed strategy)

18 / 28

slide-19
SLIDE 19

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Mutation Dynamics: Intuition Every now and then an individual dies and is replaced by another one, playing a random strategy. (NB: not necessarily sensible, but just for comparison) Definition (Mutation Dynamics) (one population, continuous) With ǫ ∈ R drawn at random from a suitable distribution: ˙ pi = pi + ǫ .

19 / 28

slide-20
SLIDE 20

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Properties of Evolutionary Dynamics

  • innovative vs. non-innovative: mutations confined to what is already

given in population or not

  • payoff monotone vs. non-monotone: reproductive success a monotone

function of (expected) utilities, or not

  • deterministic vs. stochastic: each population state uniquely defines

subsequent population state, or chance element replicator best response mutation innovative −

  • payoff monotone
  • −

deterministic

  • depends

−

20 / 28

slide-21
SLIDE 21

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Dynamic Stability & Attraction

  • fixed point: ˙
  • p =

p

  • a fixed point

p is (weakly / Lyapunov) stable iff:

  • all nearby points

stay nearby

  • for all open neighborhoods U of

p there is a neighborhood O ⊆ U of p such that any point in O never migrates out of U

  • a fixed point

p is attractive iff:

  • all nearby points

converge to it

  • there is an open neighborhood U of

p such that all points in U converge to p

  • basin of attraction of an attractive fixed point:
  • biggest U with the above property
  • a fixed point

p is asymptotically stable (aka. an attractor) iff:

  • all nearby points

converge to it on a path that stays close

  • it is stable and attractive

21 / 28

slide-22
SLIDE 22

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Replicator Dynamics in Matrix Games with 2 Actions

  • black dots: stable fixed point
  • white dots: unstable fixed point

Gy¨

  • rgy Szab´
  • and G´

abor F´ ath (2007). “Evolutionary Games on Graphs”. In: Physics Reports 446,

  • pp. 97–216

22 / 28

slide-23
SLIDE 23

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

“Folk Theorem” of Evolutionary Game Theory Any reasonable evolutionary dynamic should relate to Nash equilibrium in the following way: (Hofbauer and Sigmund, 1998)

1 stable rest points should be nes, 2 any convergent trajectory evolves to a ne, 3 strict nes are attractors.

E.g.: the “mutation dynamics” is clearly not reasonable in this sense.

Josef Hofbauer and Karl Sigmund (1998). Evolutionary Games and Population Dynamics. Cambridge, Massachusetts: Cambridge University Press

23 / 28

slide-24
SLIDE 24

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Relation: Replicator Dynamics - Nash Equilibrium (one population)

1 nes are fixed points 2 strict nes are attractors 3 if an interior orbit converges to

p, then p is a ne

4 if a fixed point is stable, then it is a ne

(NB: converses not generally true) Relation: Replicator Dynamics - Evolutionary Stability (one population)

1 esss are attractors 2 interior esss are global attractors, i.e., attract all interior points

(NB: converses not generally true)

24 / 28

slide-25
SLIDE 25

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Example: Attractor but not ess (p.120 Szab´

  • and F´

ath, 2007) U =    6 −4 −3 5 −1 3   

  • only one ess:

(1, 0, 0)

  • but attractor:

( 1 /3 , 1 /3 , 1 /3 )

25 / 28

slide-26
SLIDE 26

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Relation: Replicator Dynamics - ne - ess (two populations)

1 attractors = strict nes = esss

26 / 28

slide-27
SLIDE 27

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Homework

1 Play with the python scripts given in the lecture. 2 Find all asymmetric nes and esss of the following two games:

r1 r2 r3 r4 s1 1, 1 0, 0 .5, .5 .5, .5 s2 0, 0 1, 1 .5, .5 .5, .5 s3 .5, .5 .5, .5 .5, .5 .5, .5 s4 .5, .5 .5, .5 .5, .5 .5, .5 r1 r2 r3 r4 s1 .875, 1 −.125, 0 .625, 75 .125, .25 s2 −.175, 0 .825, 1 .575, .75 .075, .25 s3 .65, .75 .15, .25 .65, .75 .15, .25 s4 .05, .25 .55, .75 .55, .75 .05, .25

27 / 28

slide-28
SLIDE 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem

Reading

  • Sections 1 & 2 of:

Gerhard J¨ ager (2004). “Evolutionary Game Theory for Linguists. A Primer”. Unpublished manuscript, Stanford/Potsdam

  • Sections 2.1, 2.2, 2.3, 3.1, 3.2 of:

Gy¨

  • rgy Szab´
  • and G´

abor F´ ath (2007). “Evolutionary Games on Graphs”. In: Physics Reports 446, pp. 97–216

28 / 28