15-382 C OLLECTIVE I NTELLIGENCE – S18 L ECTURES 29-31: G AME T HEORY 4-6 / E VOLUTIONARY G AME T HEORY 1-3 I NSTRUCTOR : G IANNI A. D I C ARO
T HE Q UEST FOR A LTRUISM AND C OOPERATION § Cooperation seems to be not the strategy to follow in rational, self-interested agents § Ritualized animal behavior in a conflict situation: “why are animals so gentlemanly or ladylike in contests for resources?” 15781 Fall 2016: Lecture 22 2
C AN WE DO ANY BETTER ? § Prisoner’s dilemma For player row (symmetric for player col) C D R is REWARD for mutual cooperation = 3 S SUCKER’s payoff = 0 C 𝑆 𝑇 T TEMPTATION to defect = 5 D 𝑈 𝑄 P PUNISHMENT for mutual defection = 1 with T>R>P>S § Classical game theory à both players D § Shame because they’d do better by both cooperating § Cooperation is a very general problem in biology and not only! § Import tariffs - Should countries remove them? § Price fixing- why not cheat? 15781 Fall 2016: Lecture 22 3
C AN WE LEARN TO BE ALTRUISTIC ? § Alternatives to one-shot normal-form games? § How can players update / improve their policies? § Iterated prisoner’s dilemma § Let’s repeat the game with the same opponent over and over, in this case new strategies are possible because of the iterated nature § What if we observe the outcomes and the strategy can be adapted? Axelrod’s tournament (1984), let’s play with multiple opponents § § Each strategy was paired with each other strategy for 200 iterations of a game, and scored on the total points accumulated through the tournament. A strategy could adapt based on the observed outcomes § The winner was … Tit-For-Tat! § Cooperates on the first move, and subsequently echoes (reciprocates) what the other player did on the previous move: Retaliation with forgivness 15781 Fall 2016: Lecture 22 § Cheap to implement (plausible model also for animals) 4
C AN WE LEARN TO BE ALTRUISTIC ? 15781 Fall 2016: Lecture 22 5
E VOLUTIONARY G AME T HEORY § Drop the assumption of rational players, able to perfectly predict each other's thought process and make rational choices (classical game theory) § Let’s consider a (large) population of decision makers (players) § A player is not faced always with the same opponents § There are many local interaction at the same § Individuals / Players do not make choices , but implement given strategies § The frequency with which a particular decision is made depends on the fraction of individuals in the population that have it in their strategy § The frequency is time-varying: the distribution of players' actions changes towards those that are (currently) better. § We could think of this as an evolutionary dynamics 6
E VOLUTIONARY G AME T HEORY Three justifications of evolutionary dynamics: § Biological interpretation : payoff == reproductive fitness. So agents following better strategies have more children, and so proportion playing such strategy grows over time. § Economics : unsuccessful firms are driven out of business, while successful ones expand. § Imitation: Players can look around, see whether a rival player is doing better than themselves and if so copy their strategy § Best Respond : Observe how the game is currently being played, and play the current best response to it. 7
E VOLUTIONARY G AME T HEORY § Agents do not make choices , as in classical game theory, but implement their currently adopted / given strategy § An Agent with a good strategy 𝜏 (i.e., higher fitness), will reproduce more à in the next time step the population will contain proportionally more individuals with the strategy 𝜏 8
E VOLUTIONARY G AME T HEORY § Natural selection processes replaces rational behavior 9
B ASIC NOTIONS Population of individuals that can use some set 𝑇 of pure strategies § Population profile: vector 𝒚 that gives a probability 𝑦 𝑡 with which each § strategy 𝑡 ∈ 𝑇 is played in the population § A population profile needs not correspond to a strategy adopted by any member of the population E.g., a population that can use two strategies, 𝑡 - , 𝑡 . § If every member of the population randomizes by playing each of § 𝒚 = ( ½ , ½ ), and the two pure strategies with probability ½ à the population profile is the same as the mixed strategy adopted by all members If half of the population adopts strategy 𝑡 - and the other half 𝑡 . à § Populations’ profile is still 𝒚 = ( ½ , ½ ), but no member adopts it Individual payoff: if an individual uses a mixed strategy 𝜏 in a population § with profile 𝒚 , its payoff is 𝜌(𝜏 , 𝒚 ) = ∑ 𝑞(𝑡)𝜌(𝑡 , 𝒚 ) = #descendants 4∈5 10
D ESCENDANTS 𝑂 agents / animals, programmed to use one of the pure strategies 𝑡 - or 𝑡 . § Let’s assume that 50% of the agents use each of the strategies: 𝒚 = ( ½ , ½ ) § Given payoffs: 𝜌(𝑡 - , 𝒚 ) = 6, 𝜌(𝑡 . , 𝒚 ) = 4 § Next generation: 6 𝑂/2 individuals using 𝑡 - and 4 𝑂/2 individuals using 𝑡 . § à New population profile: 𝒚 = (0.6, 0.4) § Core Question: In order to determine the next population, how do 𝜌(s , 𝒚 ) behave as a function of 𝒚 ? 11
T YPES OF GAMES § Game against the field : there’s no specific opponent for a given individual, their payoff depends on what everyone in the population is doing § Population-wide interactions Mean-field game theory ≠ Classical game theory § Payoff might be not linear in the probabilities 𝑦(𝑡) with which each § pure strategy is played by population member § Frequency-dependent selection § Analogous to dynamical systems mean-field assumption for population studies 12
T YPES OF GAMES Pairwise contest: a given individual plays against an opponent that has § been randomly selected (by Nature) from the population, and the payoffs depends on what the pair of individuals do Payoffs are linear in the probabilities 𝑦(𝑡) with which each pure § strategy is played by population member: 𝑞 𝑡 𝑦(𝑡 ; )𝜌(s,s’) 𝜌(𝜏 , 𝒚 ) = ∑ ∑ 4∈5 4;∈5 13
I MPORTANT QUESTIONS Does it exist / What is the stable population (equilibrium)? § How do we define an equilibrium? § Will the population evolve toward an equilibrium? § Will the population move away from the equilibrium (stability of equilibrium)? § What are the dynamics of population change (towards equilibria)? § ~1973 George Price John Maynard Smith 14
E QUILIBRIA Stability: under which strategy profiles is the population stable? § Let 𝒚 ∗ be the profile generated by a population where all individuals adopt § strategy 𝜏 ∗ (i.e., 𝒚 ∗ = 𝜏 ∗ ) § Necessary condition for evolutionary stability: 𝜏 ∗ ∈ arg max I∈J 𝜌(𝜏 , 𝒚 ∗ ) At equilibrium the strategy adopted by individuals must be a best response to the population profile that generates it If 𝜏 ∗ is a unique best response to 𝒚 ∗ , then the evolution of the population § stops § If there are multiple stable strategies, the population could drift to any other of these strategies 15
M UTANTS / I MMIGRANTS A population where initially all individuals adopt some strategy 𝜏 ∗ § A genetic mutation (or immigration) occurs and a small proportion 𝜁 of § individuals use some other strategy 𝜏 The new population is the post-entry population, with profile 𝒚 L § E.g., 𝑇 = 𝑡 - ,𝑡 . , 𝜏 ∗ = (½,½) , and the mutant strategy is 𝜏 = O - P , § P 1 2 ,½ + 𝜁 3 4, 1 1 2 + 𝜁 4, 1 4 − 𝜁 𝒚 L = 1 − 𝜁 𝜏 ∗ + 𝜁𝜏 = 1 − 𝜁 4 = 4 16
E VOLUTIONARY S TABLE S TRATEGY (ESS) ESS: A mixed strategy 𝜏 ∗ is an ESS if mutants that adopt any other § strategy 𝜏 leave fewer offspring in the post-entry population, provided that the population of the mutants is small. That is, if there exists an 𝜁̅ such that for every 0 < 𝜁 < 𝜁̅ and for every 𝜏 ≠ 𝜏 ∗ : 𝜌 𝜏 ∗ ,𝒚 L > 𝜌 𝜏,𝒚 L § ESS is a notion of equilibrium which is resistant to invasion of mutants: it’s an end-point of evolution § Not all Nash equilibria can guarantee resistance to mutant invasion Let’s see pairwise contest examples… 17
H AWK -D OVE P AIRWISE CONTEST GAME C C 15781 Fall 2016: Lecture 22 18
H AWK -D OVE P AIRWISE CONTEST GAME C C 15781 Fall 2016: Lecture 22 19
H AWK -D OVE P AIRWISE CONTEST GAME 15781 Fall 2016: Lecture 22 20
H AWK -D OVE P AIRWISE CONTEST GAME 15781 Fall 2016: Lecture 22 21
H AWK -D OVE P AIRWISE CONTEST GAME 15781 Fall 2016: Lecture 22 22
Recommend
More recommend
