Models of language Evolution Session 09: Evolution of Pragmatic - PowerPoint PPT Presentation

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA Models of language Evolution Session 09: Evolution of Pragmatic Strategies Roland M¨ uhlenbernd University of T¨ ubingen

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA S IGNALING G AME : D EFINITION A signaling game is a tuple SG = �{ S , R } , T , Pr , M , A , U S , U R � with ◮ { S , R } : set of players ◮ T : set of states ◮ Pr : prior beliefs: Pr ∈ ∆( T ) ◮ M : set of messages ◮ A : set of receiver actions ◮ U S , R : utility function: T × M × A → R

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA S IGNALING G AME : E XAMPLE A standard Lewis game is defined as: ◮ Set of players { S , R } ◮ Set of states T = { t 1 , t 2 } ◮ Equiprobable prior beliefs: Pr ( t 1 ) = . 5 , Pr ( t 2 ) = . 5 ◮ Set of messages M = { m 1 , m 2 } (no costs) ◮ Set of actions A = { a 1 , a 2 } � 1 if i = k ◮ Utility function U S , R ( t i , m j , a k ) = 0 else a 1 a 2 t 1 1,1 0,0 t 2 0,0 1,1

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA S TRATEGIES The players’ ”actions” can be represented as pure strategies. For the Lewis game there are 4 strategies for each player: m 1 m 1 m 1 m 1 t 1 t 1 t 1 t 1 S 1 : S 2 : S 3 : S 4 : m 2 t 2 m 2 t 2 m 2 t 2 m 2 t 2 m 1 a 1 m 1 a 1 m 1 a 1 m 1 a 1 R 1 : R 2 : R 3 : R 4 : a 2 m 2 a 2 m 2 a 2 m 2 a 2 m 2

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA E XPECTED U TILITIES The expected utility for a combination of strategies is given as: � EU ( S i , R j ) = Pr ( t ) × U ( t , S i ( t ) , R j ( S i ( t ))) (1) t ∈ T R 1 R 2 R 3 R 4 S 1 1 0 .5 .5 S 2 0 1 .5 .5 S 3 .5 .5 .5 .5 S 4 .5 .5 .5 .5 Table: Expected utilities for all strategy combinations of the Lewis game

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA S IGNALING G AMES AS S TATIC G AMES ◮ static games: agents choose simultaneously ◮ SG as static game: agents choose strategies ◮ a strategy represents a ”contingency plan”: what would an agent do in each state

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA S IGNALING G AMES AS S TATIC G AMES Extensions: 1. multi-agent system ◮ graph G = � N , V � as interaction structure ◮ nodes represent agents, edges represent connections for interaction ◮ different structures: grid, small world... 2. update rules ◮ imitate the majority ◮ imitate the best ◮ conditional imitation ◮ best response ◮ against mixed strategy over all neighbours ◮ against mixed strategy over preplayed rounds (fictitious play)

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA S IGNALING G AMES AS D YNAMIC G AMES ◮ dynamic games: agents choose in sequence ◮ SG as dynamic game: agents play behavioural strategies ◮ a behavioural strategy represents a behaviour: what would an agent do for a given situation

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA B EHAVIOURAL S TRATEGIES Behavioural strategies are functions that map choice points to probability distributions over actions available in that choice point. ◮ behavioural sender strategy σ ∈ S = (∆( M )) T ◮ behavioural receiver strategy ρ ∈ R = (∆( A )) M � m 1 �→ � a 1 �→ � �  . 9   . 33  t 1 �→ m 1 �→ m 2 �→ . 1 a 2 �→ . 67  � m 1 �→   � a 1 �→  σ = ρ =     � � . 5 1     t 2 �→ m 2 �→ m 2 �→ . 5 a 2 �→ 0

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA S IGNALING G AMES AS D YNAMIC G AMES Extensions: 1. multi-agent system ◮ graph G = � N , V � as interaction structure ◮ nodes represent agents, edges represent connections for interaction ◮ different structures: grid, small world... 2. update rules ◮ imitate the majority ◮ imitate the best ◮ conditional imitation ◮ best response ?? ◮ against mixed strategy over all neighbours ◮ against mixed strategy over preplayed rounds (fictitious play)

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA B EHAVIOURAL S TRATEGIES ◮ Behavioural strategies represent probabilistic behaviour Example: σ ( m 1 | t 2 ) = . 5 - for state t 2 the sender sends message m 1 with a probability of .5 ◮ Behavioural strategies represent beliefs Example: ρ ( a 1 | m 1 ) = . 33 - the sender believes that the receiver construes message m 1 with a 1 with a probability of .33 � m 1 �→ � a 1 �→ � �  . 9   . 33  t 1 �→ m 1 �→ m 2 �→ . 1 a 2 �→ . 67  � m 1 �→   � a 1 �→  σ = ρ =     � � . 5 1     t 2 �→ m 2 �→ m 2 �→ . 5 a 2 �→ 0

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA E XPECTED U TILITY & B EST R ESPONSE ◮ While for static SG the Expected utility EU ( S i , R j ) was defined for a strategy pair � S i , R j � , for a dynamic SG it is defined in the following way: � EU S ( m | t , ρ ) = ρ ( a | m ) × U ( t , m , a ) (2) a ∈ A � EU R ( a | m , σ ) = σ ( m | t ) × U ( t , m , a ) (3) t ∈ T ◮ The behavioural strategies σ and ρ represent beliefs about the participant ◮ Just as for static games to play Best Response means to maximize Expected utility

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA B ELIEF L EARNING ◮ Behavioural strategies represent beliefs about the interlocutor ◮ The belief is a result of observation ◮ Example: � a 1 �→ �   . 8 SO a 1 a 2 m 1 �→ a 2 �→ . 2  � a 1 �→  m 1 8 2 ρ =   � . 35   m 2 �→ m 2 7 13 a 2 �→ . 65 � m 1 �→  �  . 6 RO t 1 t 2 t 1 �→ m 2 �→ . 4  � m 1 �→  m 1 6 0 σ =   � 0   t 2 �→ m 2 4 4 m 2 �→ 1

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA B ELIEF L EARNING & B EST R ESPONSE ◮ After a played game both interlocutors can observe the resulting game path and update their beliefs accordingly ◮ Example: ◮ Given the following observations and appropriate beliefs: SO a 1 a 2 RO t 1 t 2 m 1 8+1 2 m 1 6+1 0 m 2 7 13 m 2 4 4 ◮ the sender is faced with state t 1 ◮ EU ( m 1 | t 1 , ρ ) = . 8, EU ( m 2 | t 1 , ρ ) = . 35 ◮ m 1 maximises EU , thus sending m 1 is best response ◮ the receiver has to construe message m 1 : ◮ EU ( a 1 | m 1 , σ ) = . 6, EU ( a 2 | m 1 , σ ) = 0 ◮ a 1 maximises EU , thus playing a 1 is best response ◮ players observe resulting game path � t 1 , m 1 , a 1 � and update observation counts and beliefs accordingly

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA E XAMPLE : R ESULT IN A SW NETWORK Figure: Resulting structure after 30 simulation steps of 100 BL agents playing the Lewis game on a SW network. The colours blue and green represent both signaling systems as target strategies.

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA R EINFORCEMENT L EARNING For a given signaling game SG ◮ the sender has a urn ℧ t for each t ∈ T filled with balls of a type m ∈ M ◮ the receiver has a urn ℧ m for each m ∈ M filled with balls of a type a ∈ A m 1 m 2 a 1 a 2 ◮ Example: ℧ t 1 8 2 ℧ m 1 2 3 7 13 7 1 ℧ t 2 ℧ m 2

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA R EINFORCEMENT L EARNING ◮ If the sender is faced with state t she draws a ball from urn ℧ t and sends the message appropriate to the ball type m . ◮ If the receiver receives message m he draws a ball from urn ℧ m and plays the action appropriate to the ball type a . ◮ Behavioural strategies represent probabilistic behaviour ◮ After a played round successful communication will be reinforced � m 1 �→ � a 1 �→ � �  . 8   . 4  t 1 �→ m 1 �→ m 2 �→ . 2 a 2 �→ . 6  � m 1 �→   � a 1 �→  σ = ρ =     � � . 35 . 875     t 2 �→ m 2 �→ m 2 �→ . 65 a 2 �→ . 125

S IGNALING G AME B EHAVIOURAL S TRATEGIES & U PDATE D YNAMICS M ODELING PRAGMATIC PHENOMENA R EINFORCEMENT L EARNING Example: Given the following urn settings: m 1 m 2 a 1 a 2 8 2+1 2 3 ℧ t 1 ℧ m 1 ℧ t 2 7 13 ℧ m 2 7+1 1 Play 1: Play 2: ◮ the sender is faced with t 1 ◮ the sender is faced with t 2 ◮ she draws ball type m 2 ◮ she draws ball type m 1 from urn ℧ t 1 from urn ℧ t 2 ◮ the receiver has to ◮ the receiver has to construe m 2 construe m 1 ◮ he draws ball type a 1 from ◮ he draws ball type a 1 from urn ℧ m 2 urn ℧ m 1 ◮ communication per ◮ communication per � t 1 , m 2 , a 1 � is successful: � t 2 , m 1 , a 1 � isn’t successful: reinforcement no reinforcement