Models of Language Evolution Replicator dynamic & signaling - PowerPoint PPT Presentation

Models of Language Evolution Replicator dynamic & signaling Michael Franke

Class survey results

Topics for today 1 replicator dynamic 2 evolutionary dynamics of signaling games

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling 7 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Levels of analysis in EGT Static Solutions Marco-Dynamics Micro-Dynamics (Population-Level) (Agent-Based) Nash equilibrium evolutionary stability are attractors of replicator dynamic is macro-dynamics imitate the best . . the dynamics the mean field of the best response dynamic conditional imitation . static solutions? . micro-dynamics? . reinforcement learning . . . . 8 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Replicator dynamic • arguably, the most general formalization of fitness-proportional growth • mathematically convenient: • discrete version � numeric simulation • continuous version � mathematical proof • uniform formalism, multiply interpretable • clear connection with stability & equilibrium notions 9 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Utility in Mean-Field Populations Recap • let n i be the number of agents playing action a i • let n be the size of the population • population aggregate is a probability vector � p where: p i = n i n • if population is huge, the average payoff of a i is: p ) = ∑ U ( a i , � p j × U ( a i , a j ) j • U ( a i , � p ) is the fitness of a i (given the population state) • U ( � p ) = ∑ i p i × U ( a i , � p ) is the average fitness in the population 10 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Discrete-time replicator dynamics: Derivation Assumptions Average offspring of an individual playing a i is a positive scaling function F of i ’s fitness U ( a i , � p ) : F ( x ) = kx with k > 0 . • n ′ i is the number of individuals playing a i at the next discrete time step • n ′ i = n i F ( U ( a i , � p )) n ′ n i F ( U ( a i , � p )) p ′ i i = = ∑ j n ′ ∑ j n j F ( U ( a j , � p )) j n i k U ( a i , � p ) n i U ( a i , � p ) = p ) = ∑ j n j k U ( a j , � ∑ j n j U ( a j , � p ) n p i U ( a i , � p ) p i U ( a i , � p ) p ) = p i U ( a i , � p ) = p ) = ∑ j n p j U ( a j , � ∑ j p j U ( a j , � U ( � p ) 11 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Discrete time replicator dynamic U ( a i , � p ) U ( a i , � p ) fitness of i p ′ i = p i p ) = p i = proportion of i × ∑ j p j U ( a j , � U ( � p ) average fitness If p i � = 0 , frequency p i of players of type a i . . . . . . increases when i ’s fitness is higher than average; . . . decreases when i ’s fitness is lower than average; . . . stays constant when i ’s fitness is exactly average. If p i = 0 , then p ′ i = 0 . 12 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Time series � � 1 0 Coordination: U = 0 1 13 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Time series � � 2 0 Prisoner’s Dilemma: U = 3 1 14 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Time series � � 1 7 Hawks & Doves: U = 2 3 15 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Time series � � 1 0 Coordination: U = 0 2 16 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Source Code for Plots 17 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Analyzing the replicator dynamics 1 0 . 8 � � 0 . 6 1 0 U = 1 p ′ 0 2 0 . 4 p 2 p ′ 1 0 . 2 1 = 2 ( 1 − p 1 ) 2 + p 2 1 0 0 0 . 2 0 . 4 1 0 . 6 0 . 8 p 1 1 0 . 5 p 2 0 1 p 1 0 0 0 . 5 1 p 1 18 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling 19 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Continuous time replicator dynamics Derivation p i = dp i ( t ) � p i ( t + δ t ) − p i ( t ) � ˙ = lim [def. of derivative] dt δ t δ t → 0 = p ′ i − p i [discrete time RD gives limit step] U ( a i , � p ) = p i − p i [def. of discrete time RD] U ( � p ) U ( a i , � p ) − U ( � p ) = p i [“payoff-adjusted RD”] U ( � p ) p i [ U ( a i , � p ) − U ( � p )] ”=” [drop constant denominator] 20 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Continuous time replicator dynamic p i = p i [ U ( a i , � p ) − U ( � p )] = proportion of i × [ fitness of i − average fitness ] ˙ If p i � = 0 , frequency p i of players of type a i . . . . . . increases when i ’s fitness is higher than average; . . . decreases when i ’s fitness is lower than average; . . . stays constant when i ’s fitness is exactly average. If p i = 0 , then ˙ p i = 0 . 21 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Conditional imitation Assumptions • “mean field population”: huge and homogeneous • every agent plays fixed strategy for long periods of time • occasionally i considers to adopt j ’s strategy • switching probability is proportional to how much better j ’s strategy is than i ’s Revision protocol A revision protocol gives the average propensity (non-normalized probability) of agent i switching to agent j ’s strategy: ρ � p � � ij = p j U ( a j , � p ) − U ( a i , � p ) + = proportion of j × fitness difference between j and i (if j is fitter) (e.g. Helbing, 1996 ; Schlag, 1998 ) 22 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Derivation of the replicator dynamic p i = flow into i − flow out of i ˙ = ∑ p j ρ � p ji − ∑ p i ρ � p ij j j = ∑ + − ∑ � � � � p j p i U ( a i , � p ) − U ( a j , � p ) p i p j U ( a j , � p ) − U ( a i , � p ) + j j �� � = p i ∑ U ( a i , � p ) − U ( a j , � p ) � � U ( a j , � p ) − U ( a i , � p ) � p j + − + j = p i ∑ � � p j U ( a i , � p ) − U ( a j , � p ) j � � p ) − ∑ ∑ = p i p j U ( a i , � p j U ( a j , � p ) j j = p i [ U ( a i , � p ) − U ( � p )] 23 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Rest points, dynamic stability & attraction • A rest point is a state � p with ˙ p i = 0 for all i . • a rest point � p is (weakly / Lyapunov) stable iff: • all nearby points stay • for all open neighborhoods U of � p there is a nearby neighborhood O ⊆ U of � p such that any point in O never migrates out of U • a rest point � p is attractive iff: • all nearby points • there is an open neighborhood U of � p such converge to it that all points in U converge to � p • basin of attraction of an attractive rest point: • biggest U with the above property • a rest point � p is asymptotically stable (aka. an attractor ) iff: • all nearby points • it is stable and attractive converge to it (on a path that stays close) 24 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Replicator dynamic, equilibrium & evolutionary stability Equilibrium 1 ne s ⊆ rest points 2 sne s ⊆ attractors 3 if an interior orbit converges to � p , then � p is a ne 4 if a rest point is stable, then it is a ne Evolutionary stability Special case: “potential games” ( U = U T ) 1 ess s ⊆ attractors 1 ess s = attractors 2 nss s ⊆ Lyapunov stable 2 every interior orbit converges (to a 3 all interior ess s are global attractors, ne ) i.e., attract all interior points (e.g. Hofbauer and Sigmund, 1998 ) 25 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling 26 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling 27 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Early simulation evidence In 2 - 2 - 2 Lewis games (with equiprobable states), all simulation runs of the (discrete, symmetric) RD converged to signaling systems. (Skyrms, 1996 ) 28 / 36

Replicator dynamic (discrete) Replicator dynamic (continuous) Dynamics of signaling Postitive result 2 - 2 - 2 Lewis game, equiprobable In a 2 - 2 - 2 Lewis game with equiprobable states, the set of initial population states that do not converge to a signaling system under the replicator dynamics has Lebesgue measure zero. Lebesgue measure zero: has an extension that does not stretch across all dimensions. (Huttegger, 2007 ) 29 / 36