Models of Language Evolution Session 04 : Evolutionary Game Theory: - PowerPoint PPT Presentation

Models of Language Evolution Session 04 : Evolutionary Game Theory: Evolutionary Dynamics Michael Franke Seminar f¨ ur Sprachwissenschaft Eberhard Karls Universit¨ at T¨ ubingen

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Course Overview (tentative) date content 20 - 4 MoLE: Aims & Challenges 27 - 4 Evolutionary Game Theory 1 : Statics 04 - 5 Evolutionary Game Theory 2 : Macro-Dynamics 11 - 5 Guest Lecture by Gerhard J¨ ager 18 - 5 egt 3 : Micro-Dynamics & Multi-Agent Systems 25 - 5 Communication, Cooperation & Relevance 01 - 6 Combinatoriality, Compositionality & Recursion 08 - 6 Evolution of Semantic Meaning & Pragmatic Strategies 15 - 6 P entecost — no class 22 - 6 work on student projects 29 - 6 work on student projects 06 - 7 work on student projects 13 - 7 presentations 20 - 7 presentations 2 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Last Session 1 (classical) game theory • static games • (strict) Nash equilibria • in pure strategies • in mixed strategies 2 games on populations • symmetric • asymmetric 3 evolutionarily stable states • in symmetric populations • in asymmetric populations • existence, uniqueness, some properties • relation to Nash equilibrium 3 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Today’s Session 1 replicator dynamics • for matrix games • for bi-matrix games 2 other evolutionary dynamics (just for comparison) 3 “Folk Theorem of Evolutionary Game Theory” • connection evolutionary dynamics with: Nash equilibrium 1 evolutionary stability 2 Static Solutions Marco-Dynamics Micro-Dynamics (Population-Level) (Agent-Based) Nash equilibrium evolutionary stability fixed points of replicator dynamics macro-dynamics imitate the best . . dynamics are are mean field of the best response dynamics conditional imitation . static solutions? . micro-dynamics? . reinforcement learning . . . . 4 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics: Intuition (Biologically, Agent-Based) • offspring inherit strategy from their single parent • reproductive success ∼ payoff for individual playing against population NB: (later in this course: different agent-based motivation) Replicator Dynamics: From Intuition to Formalization p = ( p 1 , . . . , p n ) , the per In a population with pure strategy distribution � p i / p i is given by: ˙ capita growth rate p i ˙ = fitness of type i − average fitness in population . p i 5 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Definition (Replicator Dynamics) (one population, continuous) p = ( p 1 , . . . , p n ) as a population distribution of Consider a mixed strategy � pure strategies in a symmetric game. The continuous-time replicator dynamics defines the continuous change in the proportion of agents playing strategy i as: p i = p i [( U � p ) i p ] . ˙ − � p · U � Definition (Replicator Dynamics) (one population, discrete) We look at a population distribution at concrete, discreet points in time. p ( t ) = ( p 1 , . . . , p n ) is the population distribution of pure strategies in a � symmetric game at time t . Then the discrete-time replicator dynamics defines the proportion of agents playing strategy i at t + 1 as: � i · U � p ( t ) p i ( t + 1 ) = p i ( t ) × p ( t ) . p ( t ) · U � � (NB: � i is the unit vector with a single 1 at position i ) 6 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics One-Population Time Series � � 1 0 Coordination: U = 0 1 7 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics One-Population Time Series � � 2 0 Prisoner’s Dilemma: U = 3 1 8 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics One-Population Time Series � � 0 1 Anti-Coordination: U = 1 0 9 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics One-Population Time Series � � 1 7 Hawks & Doves: U = 2 3 10 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Source Code for Plots 11 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Definition (Replicator Dynamics) (two populations, continuous) Given mixed strategies � p and � q as the population distributions of pure strategies in an asymmetric game, the continuous-time replicator dynamics is: p i = p i [( U 1 � ˙ q ) i − � p · U 1 � q ] , q i = q i [( U 2 � p ) i p ] . ˙ − � q · U 2 � Definition (Replicator Dynamics) (two populations, discrete) With � p ( t ) and � q ( t ) population distributions at time t , the discrete-time replicator dynamics is given as: � i · U 1 � q ( t ) p i ( t + 1 ) = p i × q ( t ) , � p ( t ) · U 1 � � i · U 2 � p ( t ) q i ( t + 1 ) = q i × p ( t ) . q ( t ) · U 2 � � 12 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics Two-Population Dynamic Field � � 1 0 Coordination: U 1 , 2 = 0 1 13 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics Two-Population Dynamic Field � � 2 0 Prisoner’s Dilemma: U 1 , 2 = 3 1 14 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics Two-Population Dynamic Field � � 0 1 Anti-Coordination: U 1 , 2 = 1 0 15 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics Two-Population Dynamic Field � � 1 7 Hawks & Doves: U 1 , 2 = 2 3 16 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Source Code for Plots 17 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Best Response Dynamics: Intuition Every now and then a small fraction of a population changes its strategy and adopts a best response to the population average. Definition (Best Response) (recap) Player i ’s best response to player j playing � q is any (mixed) strategy � p that maximizes player i ’s expected utility given � q : BR i ( � q ) = arg max p ∈ ∆ ( k ) EU i ( � p , � q ) . � Definition (Best Response Dynamics) (one population, continuous) Fix a mixed strategy � p = ( p 1 , . . . , p n ) as a population distribution of pure strategies in a symmetric game. Then the continuous-time best response dynamics is given as: ˙ p = BR ( � p ) − � � p . (NB: BR ( � p ) not necessarily a unique mixed strategy) 18 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Mutation Dynamics: Intuition Every now and then an individual dies and is replaced by another one, playing a random strategy. (NB: not necessarily sensible, but just for comparison) Definition (Mutation Dynamics) (one population, continuous) With ǫ ∈ R drawn at random from a suitable distribution: p i = p i + ǫ . ˙ 19 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Properties of Evolutionary Dynamics • innovative vs. non-innovative : mutations confined to what is already given in population or not • payoff monotone vs. non-monotone : reproductive success a monotone function of (expected) utilities, or not • deterministic vs. stochastic : each population state uniquely defines subsequent population state, or chance element replicator best response mutation innovative � � − payoff monotone � � − deterministic depends � − 20 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Dynamic Stability & Attraction • fixed point : ˙ p = � � p • a fixed point � p is (weakly / Lyapunov) stable iff: • all nearby points • for all open neighborhoods U of � p stay nearby there is a neighborhood O ⊆ U of � p such that any point in O never migrates out of U • a fixed point � p is attractive iff: • all nearby points • there is an open neighborhood U of � p converge to it such that all points in U converge to � p • basin of attraction of an attractive fixed point: • biggest U with the above property • a fixed point � p is asymptotically stable (aka. an attractor ) iff: • all nearby points • it is stable and attractive converge to it on a path that stays close 21 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem Replicator Dynamics in Matrix Games with 2 Actions • black dots: stable fixed point • white dots: unstable fixed point Gy¨ orgy Szab´ o and G´ abor F´ ath ( 2007 ). “Evolutionary Games on Graphs”. In: Physics Reports 446 , pp. 97 – 216 22 / 28

Replicator Dynamics Other Evolutionary Dynamics Folk Theorem “Folk Theorem” of Evolutionary Game Theory Any reasonable evolutionary dynamic should relate to Nash equilibrium in the following way: (Hofbauer and Sigmund, 1998 ) 1 stable rest points should be ne s , 2 any convergent trajectory evolves to a ne , 3 strict ne s are attractors. E.g.: the “mutation dynamics” is clearly not reasonable in this sense. Josef Hofbauer and Karl Sigmund ( 1998 ). Evolutionary Games and Population Dynamics . Cambridge, Massachusetts: Cambridge University Press 23 / 28