Conventions and Coalitions in Repeated Games S. Nageeb Ali Ce Liu - - PowerPoint PPT Presentation

Conventions and Coalitions in Repeated Games

• S. Nageeb Ali

Ce Liu (Will)

Penn State UCSD

May 29, 2018

Cooperative game theory takes a “bird’s eye” view to group decisions. Studies stable “static conventions” immune to group deviations. Theory of repeated games studies dynamic ongoing relationships. Studies stable “dynamic conventions” immune to individual deviations.

Cooperative game theory takes a “bird’s eye” view to group decisions. Studies stable “static conventions” immune to group deviations. Theory of repeated games studies dynamic ongoing relationships. Studies stable “dynamic conventions” immune to individual deviations. Our objective: Develop a theory of repeated games for coalitional behavior.

the enterprise

Framework nests both noncooperative and cooperative environments. Results speak both to repeated play of a cooperative game as well as coalitional deviations in the repeated play of a non-cooperative game. Solution-concept combines farsightedness and limited commitment.

two results

two results

Perfect monitoring in both NTU + TU games: Folk Theorem. Every payoff in the set of feasible and strictly individually rational payoffs is supported as δ → 1. ⇒ coalitional deviations do not refine play if players are patient.

two results

Perfect monitoring in both NTU + TU games: Folk Theorem. Every payoff in the set of feasible and strictly individually rational payoffs is supported as δ → 1. ⇒ coalitional deviations do not refine play if players are patient. TU games + secret transfers: Anti-Folk Theorem. Only payoffs in the β-core are supported, and any such payoff is supported as δ → 1. ⇒ secret transfers + coalitional deviations undo dynamic incentives.

1 Introduction 2 Examples 3 Model 4 Folk Theorem for NTU 5 Transferable Utility 6 Related Lit + Summary

the roommates problem

2 rooms, 4 people

Ann Bob Carol

the roommates problem

2 rooms, 4 people

Ann Bob Carol David

the cycle

Arrows point to first preference.

the cycle

Arrows point to first preference. No one is a worse roommate than David.

the cycle

Arrows point to first preference. No one is a worse roommate than David. Ui(ij) = 4 − k if j is i’s kth favorite partner.

no stable match

no stable match

no stable match

no stable match

Suppose individuals play repeatedly. Coalitions cannot commit to future behavior, but can jointly deviate in the stage-game. Our solution-concept seeks robustness to one-shot coalitional deviations. The key idea:

• Coalitions have some short-term commitment (which one models

implicitly in the cooperative stage-game).

• But coalitions lack long-term commitment.

A stable convention if δ > 1

2.

A stable convention if δ > 1

2.

A stable convention if δ > 1

2.

A stable convention if δ > 1

2.

A divide-the-dollar example

Suppose that {1, 2, 3} are dividing a dollar. Any division requires approval of at least two players. Player 1 has veto power, and must be included in any winning coalition.

A divide-the-dollar example

Suppose that {1, 2, 3} are dividing a dollar. Any division requires approval of at least two players. Player 1 has veto power, and must be included in any winning coalition. Core of stage-game:

• Player 1 captures entire dollar.
• Any other outcome has a profitable deviation for {1, 2} or {1, 3}.

repeated divisions: perfect monitoring

Suppose the environment is repeated and coalitions cannot commit to long-term deviations. We claim that one can now support every division as δ → 1.

Claim: ( 0, 1

2, 1 2

) is supportable using a stable convention if δ ⩾ 1

2.

Claim: ( 0, 1

2, 1 2

) is supportable using a stable convention if δ ⩾ 1

2.

Pf: Use a core-reversion convention:

• Choose

( 0, 1

2, 1 2

) every period on path of play.

• Revert to (1, 0, 0) if any other allocation is chosen.

Claim: ( 0, 1

2, 1 2

) is supportable using a stable convention if δ ⩾ 1

2.

Pf: Use a core-reversion convention:

• Choose

( 0, 1

2, 1 2

) every period on path of play.

• Revert to (1, 0, 0) if any other allocation is chosen.

Even if 1 offers entire surplus to coalition-partner: (1 − δ)(1) + δ(0) ⩽ 1

2.

Key idea: Veto player can’t recruit anyone to join his coalition.

Claim: ( 0, 1

2, 1 2

) is supportable using a stable convention if δ ⩾ 1

2.

Pf: Use a core-reversion convention:

• Choose

( 0, 1

2, 1 2

) every period on path of play.

• Revert to (1, 0, 0) if any other allocation is chosen.

Even if 1 offers entire surplus to coalition-partner: (1 − δ)(1) + δ(0) ⩽ 1

2.

Key idea: Veto player can’t recruit anyone to join his coalition. Core-reversion supports any payoff vector u where min{u2, u3} ⩾ 1 − δ.

anti-folk theorem with secret transfers

Suppose players can make “secret transfers.” Convention conditions only on identity of the blocking coalition, and chosen allocation, but not eventual payoffs.

anti-folk theorem with secret transfers

Suppose players can make “secret transfers.” Convention conditions only on identity of the blocking coalition, and chosen allocation, but not eventual payoffs. Stable convention supports only stage-game core: (1, 0, 0). (regardless of δ).

1 Introduction 2 Examples 3 Model 4 Folk Theorem for NTU 5 Transferable Utility 6 Related Lit + Summary

primitives: NTU stage-game

Choice over alternatives: a ∈ A. Generated Payoffs: v : A → Rn. Set of Coalitions: C ≡ 2N\∅. If alternative a is recommended, EC(a) is the set of alternatives to which coalition C can deviate, assuming no one else deviates. EC(·) is coalition C’s effectivity function.

example: strategic-form game

a is action profile of the stage-game. Effectivity function extends usual notion of individual deviations to coalitional deviations. EC(a) ≡ { a′ ∈ A : a′

−C = a−C

}

example: cooperative game

a is a partition of N, and A is the set of all partitions. vi(a) is the payoff from partition a. EC(a) specifies partitions that coalition C can generate, where “untouched” coalitions continue to follow a. EC(a) ⊆ {a′ : C′ ∩ C = ∅ and C′ ∈ a ⇒ C′ ∈ a′}

examples

Voting: Convention specifies a default policy. But a coalition can introduce an alternative and block the default if it has more than a majority of players. Matching: Convention specifies who matches with whom. But pairs of players can defect and match differently.

assumptions on effectivity

Transitivity: For every C, a′ ∈ EC(a) & a′′ ∈ EC(a′) ⇒ a′′ ∈ EC(a). Reflexivity: For every C and a, a ∈ EC(a). Omnipotence of the Grand Coalition: EN(a) = A for every a. (For the β-core result)

repeated interaction

For a path p where at is alternative in period t, player i’s payoff is Ui(p) ≡ (1 − δ)

∞

∑

t=0

δtvi(at). Outcome at t ≡ ( chosen alternative, coalition that blocks (if any) ) . A history is a sequence of outcomes. A convention is a mapping, σ : Histories → A.

repeated interaction

For a path p where at is alternative in period t, player i’s payoff is Ui(p) ≡ (1 − δ)

∞

∑

t=0

δtvi(at). Outcome at t ≡ ( chosen alternative, coalition that blocks (if any) ) . A history is a sequence of outcomes. A convention is a mapping, σ : Histories → A. a(h | σ) ≡ σ(h) is alternative recommended by σ after history h. p(h | σ) is path recommended by σ after history h. (computed recursively from a(h | σ)).

• Definition. A convention is stable if for every history h, ∄ coalition C

and a′ ∈ EC(a(h | σ)) such that for all i in C, (1 − δ)vi(a′) + δUi(p(h, (a′, C)|σ)) > Ui(p(h | σ)). No coalition wishes to deviate at any history, anticipating no deviations at all other histories.

features of stable conventions

Recursivity. Relationship to core of stage-game. One-shot Deviation Principles?

recursivity

Can use APS to characterize implementable outcomes for a fixed δ.

relationship to core of stage-game

• Definition. An alternative a is a core-alternative of stage-game if ∄

coalition C and a′ ∈ EC(a) such that for all i in C. vi(a′) > vi(a).

relationship to core of stage-game

• Definition. An alternative a is a core-alternative of stage-game if ∄

coalition C and a′ ∈ EC(a) such that for all i in C. vi(a′) > vi(a). [Stable convention of G∞: Core of G] ≃ [SPE of G∞ : NE of G].

Formally, warding off one-shot coalitional deviations. Because this includes singleton coalitions, this is a refinement of SPE. But the sufficiency of examining only one-shot deviations (the so-called “one-shot deviation principle”) is typically a result, not an assumption. What comes from assuming it?

• ne-shot deviation principle

for an individual

If Alice profits from stringing together a (possibly infinite) sequence of

• ne-shot coalitional deviations, each of which is profitable for every

deviating coalition partner but not necessarily for herself, then there is a profitable one-shot coalitional deviation. We prove this statement using an argument analogous to classical result.

• ne-shot deviation principle

for coalitions

If a coalition of two or more players have a profitable multi-shot deviation, then it has a profitable one-shot deviation. This version is false, except in the case of secret transfers.

1 Introduction 2 Examples 3 Model 4 Folk Theorem for NTU 5 Transferable Utility 6 Related Lit + Summary

Minmax and Folk Theorem

vi ≡ min

a∈A

max

a′∈E{i}(a) vi(a′).

(Player i’s minmax)

Minmax and Folk Theorem

vi ≡ min

a∈A

max

a′∈E{i}(a) vi(a′).

(Player i’s minmax)

Theorem

Suppose that set of feasible and strictly individually rational payoffs has a non-empty interior. For every payoff in in that set, there is a stable convention that sustains that payoff if players are sufficiently patient.

who guards the guardians?

Classical question highlights difficulties of simultaneously punishing too many people. Our solution: punish a player in each coalition, grant amnesty to others.

step 1: individual minmax is lower-bound

vi ≡ min

a∈A

max

a′∈E{i}(a) vi(a′).

No stable convention can generate payoffs below vi for player i. Usual argument: at every history, player i can take a myopic best-response to the play of others.

step 2: player-specific punishments

step 2: player-specific punishments

Suppose you would like to support payoff vector v∗ in the interior of the feasible and strictly individually rational payoffs.

• Design player-specific punishments (vi)n

i=1 where each vi is a vector.

• Player i’s punishment: vi

i < v∗ i.

• Better off if someone else is punished: vi

i < vj i.

Assume (for now) that for each payoff, ∃ an alternative that generates it.

step 3: convention-design

Use Fudenberg & Maskin (1986) to ward off individual deviations:

• Play v∗ on the path of play.
• Player i deviates ⇒ Minmax for T(δ) periods, and then vi.
• vi

j > vj j ⇒ player j doesn’t wish to deviate when i is being punished.

Use a scapegoat policy to ward off coalitional deviations: at each history, for each coalition C, pick a scapegoat i ∈ C, and punish her when coalition C deviates as if she were the only deviator. Scapegoat can be picked arbitrarily, except someone being minmaxed.

step 4: how to convexity

Public correlation devices. Sequences of play. (Sorin 1986; Fudenberg & Maskin 1991).

We have looked at only one-shot coalitional deviations. Can allow for coalitional deviations that are up to T < ∞ periods long. Holding T fixed, and taking δ → 1 results in the same theorem.

1 Introduction 2 Examples 3 Model 4 Folk Theorem for NTU 5 Transferable Utility 6 Related Lit + Summary

how we model transfers

Describe transfers using a matrix m ≡ [mij] where:

• mii = 0.
• mij ⩾ 0 are the transfers paid to player j by player i.

M is the set of all such matrices. Player i’s payoff = vi(a) + Incoming Transfers - Outgoing Transfers. Set of feasible payoffs is U = { u ∈ Rn : ∃a ∈ A such that ∑

i∈N

ui = ∑

i∈N

vi(a) } .

Outcomes are now (a, C, m). Conventions are mappings of the form σ : Histories→ A × M. If coalition C blocks (a, m), it chooses:

• Feasible alternative deviation: a′ ∈ EC(a).
• Transfers: (m′

C, m−C).

Those outside the blocking coalition still make transfers as before.

A restriction

We consider conventions such that continuation values are (uniformly) bounded across all histories. Formally, {u ∈ Rn : ∃h ∈ H such that U(h | σ = u} is a bounded subset of Rn.

stable conventions with public transfers

definition

A convention is stable if for every history h, ∄ coalition C, alternative a′ ∈ EC(a(h | σ)), and transfers m′

C such that for all i in C,

Ui(p(h | σ)) < (1 − δ) + δ .

stable conventions with public transfers

definition

A convention is stable if for every history h, ∄ coalition C, alternative a′ ∈ EC(a(h | σ)), and transfers m′

C such that for all i in C,

Ui(p(h | σ)) < (1 − δ) ( vi(a′) + ∑

j∈C m′ ji + ∑ j/ ∈C mji − ∑ j∈N m′ ij

) + δ .

stable conventions with public transfers

definition

A convention is stable if for every history h, ∄ coalition C, alternative a′ ∈ EC(a(h | σ)), and transfers m′

C such that for all i in C,

Ui(p(h | σ)) < (1 − δ) ( vi(a′) + ∑

j∈C m′ ji + ∑ j/ ∈C mji − ∑ j∈N m′ ij

) + δ Ui(p(h, (a′, C, m′

C, m−C(h | σ)))).

folk theorem with public transfers

When transfers are observable, punish players not only for participating in coalitions but also for their transfers. Transfers may allow coalitions to consider their joint surplus today, but continuation values may be adjusted on the basis of these transfers.

secret transfers

Secret transfers: if a coalition blocks, then transfers within that coalition are unobserved by those outside the coalition. We model this as a measurability condition. σ can condition on:

• chosen alternative a
• the blocking coalition (if any)
• payoffs outside a blocking coalition

But not on payoffs within a blocking coalition.

Say that two histories are identical up to transfers if the only difference between the two are transfers within blocking coalitions. A convention under secret transfers must respect σ(h) = σ(h′) for any two histories h and h′ that are identical up to transfers.

a one-shot deviation principle

coalitions

• Lemma. If a coalition has a profitable multi-shot deviation, then it has a

profitable one-shot deviation.

a one-shot deviation principle

coalitions

• Lemma. If a coalition has a profitable multi-shot deviation, then it has a

profitable one-shot deviation. Idea of proof:

• We show that the total gain for a coalition C from the multi-shot

deviation is bounded.

• Therefore, there is a profitable multi-shot deviation that deviates at

finitely many steps.

• Use standard backward-induction logic to find a profitable one-shot

deviation for the coalition.

statement

The β-characteristic function is defined by ψ(C) ≡ min

a∈A

max

a′∈EC(a)

∑

i∈C

vi(a′)

statement

The β-characteristic function is defined by ψ(C) ≡ min

a∈A

max

a′∈EC(a)

∑

i∈C

vi(a′)

Definition

The β-core is the set K ≡ {u ∈ U : ∑

i∈C

ui ⩾ ψ(C) for all C ∈ C} The strong β-core is the set Ks ≡ {u ∈ U : ∑

i∈C

ui > ψ(C) for all C ∈ C\{N}; ∑

i∈N

ui = ψ(N)}

Anti-Folk Theorem

Theorem

Fix δ < 1. No payoff profile outside of the β-core can be supported by a stable convention.

Anti-Folk Theorem

Theorem

Fix δ < 1. No payoff profile outside of the β-core can be supported by a stable convention. If the strong β-core is non-empty, then for every element of it, ∃ δ such that for all δ > δ, that payoff profile can be supported by a stable convention.

step 1: coalitional minmax is lower-bound

ψ(C) = min

a∈A

max

a′∈EC(a)

∑

i∈C

vi(a′) If joint surplus for coalition C is less than ψ(C), that coalition has a profitable multi-shot deviation (myopic BR for coalitional value). ⇒ coalition C has a profitable one-shot deviation.

step 2: achieving any point in β-core

Payoffs are on efficiency frontier: chosen alternative a maximizes ∑

i∈N vi(a).

Our goal: generate a folk theorem on that frontier. Set of coalitions is 2n − 1. Payoffs satisfies Abreu-Dutta-Smith’s NEU condition: no two coalitions have payoffs that are positive affine transformations of each other. ⇒ we can generate player-specific punishments for the 2n − 1 players.

1 Introduction 2 Examples 3 Model 4 Folk Theorem for NTU 5 Transferable Utility 6 Related Lit + Summary

related literature

A Natural Approach: Collapse infinitely repeated game into its normal-form, and study its core.

related literature

A Natural Approach: Collapse infinitely repeated game into its normal-form, and study its core. Corresponds to Aumann (1959):

• Strong Nash Equilibria of Repeated Game.
• Limit-of-means discounting (⇒ no gain from short-term deviations).

related literature

A Natural Approach: Collapse infinitely repeated game into its normal-form, and study its core. Corresponds to Aumann (1959):

• Strong Nash Equilibria of Repeated Game.
• Limit-of-means discounting (⇒ no gain from short-term deviations).

Solution-concept is peculiar:

• Coalitions deviate to strategies in the continuation game.

⇒ Coalitions have unlimited commitment in the infinite-horizon. ⇒ Why don’t they commit from the very beginning?

Farsighted Stability

Konishi and Ray (2003) Hyndman and Ray (2007) Ray and Vohra (1997,1998,2015) Chwe (1994) Gomes and Jehiel (2005) Jordan (2006) Acemoglu, Egorov, and Sonin (2012) Mert Kimya (2015) Dutta and Vohra (2016)

“It would be of interest to investigate dynamic noncooperative games with (nonbinding) coalition formation. One might begin with the partition function, so that the formation of a coalition structure at any date has a definite impact on payoffs, perhaps through the writing of binding agreements within coalitions in any period. But the important difference...is that such agreements would—by assumption—be up for grabs at the end

• f every period. There are no binding agreements that last for

longer than a single date.” - Ray’07

legal vs. community enforcement

Laws reflect a shared understanding of how future unfolds as a consequence of choices today. In the end, all are caught in a web of self-reinforcing sanctions. Law’s empire, tangible and all-encompassing as it may seem, is founded on little else than beliefs. – Kaushik Basu (2000) Words written on a piece of paper don’t alter the laws of

• physics. Yet laws do influence behavior. - Mailath, Morris, &

Postlewaite (2001,2015). But people being punished attempt to side-deal with enforcers. ⇒ Credible laws must ward off individuals and coalitional deviations.

Framework for studying coalitional behavior in repeated games.

• Coalitional deviations in repeated games.
• Self-enforcing behavior in repeated cooperative games.

With full observability, short-term coalitional deviations come at little cost as δ → 1. Secret transfers severely limit what may be implemented for every δ. Offers foundations for β-core in exponential discounting.

Thanks!