Recursive Lexicographical Search: Finding all Markov Perfect - - PowerPoint PPT Presentation

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions

Recursive Lexicographical Search: Finding all Markov Perfect Equilibria in Directional Dynamic Games

Fedor Iskhakov, University of New South Wales John Rust, Georgetown University Bertel Schjerning, University of Copenhagen 5th Microeconometric Network Meeting Copenhagen June 19th, 2014

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Problem: How to Compute All Markov Perfect Equilibria?

The Markov Perfect Equilibrium (MPE) concept of Maskin and Tirole (1988) is now a widely used in empirical IO. However computing MPE remains a daunting computation problem Quote (H¨

• rner et. al. Econometrica 2011)

“Dynamic games are difficult to solve. In repeated games, finding some equilibrium is easy, as any repetition of a stage-game Nash equilibrium will do. This is not the case in stochastic games. The characterization of even the most elementary equilibria for such games, namely (stationary) Markov equilibria, in which continuation strategies depend on the current state only, turns out to be often challenging.”

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Finding even a single MPE is challenging!

How do people “find” MPEs? Theorists: Guess and Verify Applied people: Iterate on the player Bellman equations Pakes and McGuire (1994): some of the earliest work on computing MPE. Proposed a deterministic, iterative algorithm to compute MPE. Found a curse of dimensionality in trying to solve MPE model of firm dynamics with even moderate numbers of firms Pakes and McGuire (2001): Proposed a stochastic algorithm to approximate an MPE, in an attempt to break this curse of dimensionality

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

The Problem with “Iterating on the Value Function”

Solving infinite horizon dynamic games is difficult in part because there is no “last period” to start a standard backward induction calculation. The applied version of “guess and verify” is to make a guess about the players’ value functions and then use this guess as a starting point for sucessive approximations of the players’ Bellman equations (value function iteration) The problem is that in dynamic games, the requisite continuity conditions for the Bellman equations to constitute contraction mappings generally fails to hold. This implies that successive approximations is not guaranteed to converge to a fixed point/MPE, unlike in the case of single agent “game against nature” problems

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Successive Approximations and Equilibrium Selection

Sometimes successive approximations does converge, and if it does, one can prove that it converges to an set of value functions that constitutes an MPE of the game G However if there are multiple MPE, successive approximations can converge to different MPE depending on how the algorithm is initialized Despite these problems, applied work in economics typically relies on successive approximations as the method of choice. People seem unaware that this algorithm can be inadvertently

• perating as an equilibrium selection mechanism

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Existing methods for finding ALL MPE

Much better to compute all MPE and select one of them on economic grounds rather than haphazardly selecting an MPE depending on how the solution algorithm is initialized Homotopy/path following methods (Borkovsky et. al. 2010 and Besanko et. al. 2010) Algebraic methods (Datta 2010, Judd et. al. 2012) Problem with the homotopy methods: may not be able to follow all the bifurcations in the path from a unique equilibrium of a “perturbed problem” to find all MPE of the problem you want to solve Problem with the algebraic methods: limited to problems where the equations defining the state-specific Nash equilibria can be expressed as systems of polynomial equations that have specific forms.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Our approach: Recursive Lexicographical Search

This is a systematic approach to “building” the set of all MPE from the MPE to simpler dynamic games we refer to as stage games of the original dynamic game G. It is based on a generalization of backward induction that is applicable to a class of dynamic games that we call directional dynamic games (DDGs) A DDG is a game where a subset of the state variables evolve in a manner that satisfies certain conditions including an intuitive notion of directionality. When the state space is finite we can exploit this directionality and partition it into a finite number of elements we call stages. We can find the MPE of the game using a generalized version

• f backward induction over the stages of the game, a process

we call state recursion

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Examples of DDGs

Any finite horizon game: time is always an obviously “directional variable” Chess: the number of chess pieces still on the board only decreases and never increases Bargaining over a stochastically shrinking pie: the game is directional if the pie can only shrink Patent races (Judd, Schmedders, Yeltekin, 2012) Bertrand price competition with leapfrogging investments: directionality comes from the fact that firms can “leapfrog” each other by investing in a state of the art production technology that steadily improves

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

State Recursion versus Backward Induction

State recursion can be viewed as a generalization of the method of backward induction which is done using a time variable t where t ! t + 1 with probability 1 State recursion is backward induction over the stages of the game ⌧, and transitions between stages can be stochastic. Thus ⌧ can be thought of as a stochastic time variable which evolves unidirectionally, but is not restricted to satisfy the restriction that ⌧ ! ⌧ + 1 with probability 1

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Relation to Subgame Perfection

Kuhn (1953) and Selten (1965) showed that standard backward induction on the game tree (the extensive form representation of the game) can be used to compute all subgame perfect equilibria of finite games In a DDG, we can represent the directionality of the game via a directed acyclic graph (DAG). While every game tree is a DAG, the DAGs that represent directionality in a dynamic game are not game trees. Instead, the DAG summarizes the directionality of the game in terms of the state space instead

• f the temporal ordering implied by the game tree.

State recursion can be viewed as a generalization of standard backward induction, but a type of backward induction that is performed on the DAG rather than on the game tree.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Recursive Lexicographical Search (RLS)

State recursion can be used to find a single MPE of a dynamic game G It depends on a specification of an equilibrium selection rule for selecting a particular MPE at each state of the game that constitute the behavior strategies used by the players Recursive lexicographical search (RLS) is an algorithm that repeatedly invokes state recursion in an efficient way to compute all MPE of the DDG by systematically cycling through all feasible equilibrium selection rules (ESRs) for each

• f the component stage games of the DDG.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Relation to Literature on Finitely Repeated Games

The idea of how multiple equilibria of a stage game can be used to construct a much larger set of equilibria in the overall game was used by Benoit and Krishna (1985) to show that a version of the Folk Theorem can hold in finitely repeated games. The prevailing view prior to their work was that the extreme multiplicity of equilibria implied by the Folk Theorem for infinitely repeated games cannot happen because backward induction from the last period results in a unique equilibrium in a finitely repeated game. Benoit and Krishna showed that multiplicity of equilibria in the stage games can be used to create a much larger set of subgame perfect equilibria in the finitely repeated game, so the Folk Theorem can emerge if the time horizon is sufficiently large.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Benoit and Krishna vs RLS

However Benoit and Krishna did not propose an algorithm to enumerate all possible subgame perfect equilibria of a finitely repeated game, whereas the RLS algorithm we propose can find and enumerate all such equilibria. Though we do not claim that all dynamic games will have exploitable directional structure, we show there is a sense in which the RLS algorithm can approximate the set of all MPE for a wide class of finite and infinite-horizon dynamic games, even if there is no exploitable directionality in the game other than the passage of time. We show how backward induction performed in the right way can approximate all MPE of a fairly broad class of infinite horizon games. We view this as an analog of Benoit and Krishnas Folk Theorem approximation result for finitely repeated games.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions How to find all equilibria? Problems with Value Function Iteration Existing methods for finding ALL MPE Our Approach and Road Map for the Rest of the Talk

Road Map for the Talk

Part I: Example of a DDG (The Leapfroging Model)

1 Illustrate RLS by using it to find all MPE of a dynamic model

• f Bertrand price competition with leapfrogging investments.

2 Some theoretical results and numerical simulations 3 Danger of imposing symmetry 4 Limitations of homotopy parameter methods 5 Implications for empirical work

Part II: The general case Rigorously define the concept of a general class of directional dynamic games Introduce the state recursion algorithm and show that it can compute a single MPE of the game G Introduce the recursive lexicographical search algorithm and show that it can find all MPE of G

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

PART I A DDG example: The Dynamics of Bertrand Price Competition with Cost-Reducing Investments (Iskhakov, Rust, Schjerning (2013))

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Example: The leapfrogging model

Basic Setup Discrete time, infinite horizon (t = 1, 2, 3, . . .) Two firms, homogenous/differentiated goods, no entry or exit Each firm maximizes expected discounted profits Bertrand competition: set product prices (simultaneously) Investment decision: Whether to invest in state of the art production technology Pay investment cost of K(c) to obtain marginal cost c 0 Time to build: state of the art technology is operational after a one period lag State of the art costs follows exogenous Markov process and

• nly improves

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Example: The leapfrogging model - cont.

Timing of cost reducing investment decisions

1 Simultaneous moves:

Fully directional, d = (c1, c2, c)

2 Alternating moves:

The right to move, m, follows a Markov process (deterministic alternation as a special case). m is clearly a non-directional, but game still have a directional component, d = (c1, c2, c)

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

State space of the game: a “quarter pyramid”

S = {(c1, c2, c)|c1 ≥ c, c2 ≥ c, c ∈ [0, c]}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

End Games States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

No Investment End Games States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Multiple Equilibria End Games States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Second order best response function, η = 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 End Game Equilibria (c1,c2)=(0.714286,2.14286) k=7 beta=0.95 Firm 1’s probability of investing 2nd order best response function

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Calculated Equilibria for End Games States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Recursion Level 1 States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Recursion Level 1 States, isolated

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

No Investment Recursion Level 1 States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Multiple Equilibria Recursion Level 1 States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Calculated Equilibria Recursion Level 1 States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Recursion Level 2 States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Recursion Level 2 States, isolated

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

No Investment Recursion Level 2 States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Multiple Equilibria Recursion Level 2 States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Calculated Equilibria Recursion Level 2 States

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Continue recursion to calculate equilibria in all states

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Partial ordering of states

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Recursive Lexicographic Search Algorithm

Building blocks of RLS algorithm:

1 State recursion algorithm solves the game conditional on

equilibrium selection rule (ESR)

2 RLS algorithm efficiently cycles through all feasible ESRs

Properties of RLS algorithm: Complete: Computes all MPE equilibria of the game Fast: time spent of search of feasible ESRs is negligible in comparison to time spent on solving the game

Efficiently skip infeasible ESRs Re-use results of previously computed subgames

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Represent ESR as equilibrium string of digits

Use numbers in base-K number system with digits 0, 1, .., K − 1

Dependence preserving property: Any point of the state space may depend on points to the left and not the points to the right

corner edges interior c e e e e i i i i c e e i c

ESR string

14 13 12 11 10 9 8 7 6 5 4 3 2 1

c 1 1 1 1 2 c1 2 1 2 2 1 1 1 1 2 2 2 c2 2 1 2 1 2 1 1 2 1 2 2 End game

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

State space of the game: a “quarter pyramid”

S = {(c1, c2, c)|c1 ≥ c, c2 ≥ c, c ∈ [0, c]}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Particular ESRs examples

c e e e e i i i i c e e i c

ESR string

14 13 12 11 10 9 8 7 6 5 4 3 2 1

c 1 1 1 1 2 c1 2 1 2 2 1 1 1 1 2 2 2 c2 2 1 2 1 2 1 1 2 1 2 2 End game Examples:

0 0 0 0 0 0 0 0 0 0 0 0 0 0 First equilibrium always 0 0 2 2 2 0 0 2

High cost to invest

2 2 0 0 0 2 2 0

Low cost to invest

1 1 1 1 1 1 Mixed when equal

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

All possible ESRs

Lexicographic order

c e e e e i i i i c e e i c

ESR string

14 13 12 11 10 9 8 7 6 5 4 3 2 1

Lexicograph 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 2 1 4,782,969 0 0 0 0 0 0 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 … 2 2 2 2 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Recalculation of fesibility condition for new ESR

Avoid recalculation of subgames

c e e e e i i i i c e e i c

ESR string

14 13 12 11 10 9 8 7 6 5 4 3 2 1

0 0 0 0 0 0 0 0 0 0 0 0 0 0 always admissible Nr of eqb

1 1 1 1 1 3 3 3 3 1 1 1 3 1

0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 admissible, solve

1 1 1 1 1 3 3 3 3 1 1 1 3 *

Might have changed No changes in the solution of the game including the number of stage equilibria

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Jumping over blocks of infeasibles ESRs

Using block structure of lexicographic ordering

c e e e e i i i i c e e i c

ESR string

14 13 12 11 10 9 8 7 6 5 4 3 2 1 Iteration:

0 0 0 0 0 0 0 0 0 0 0 0 0 0 1

1 1 1 1 1 3 3 3 3 1 1 1 3 1 1a

0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 2

1 1 1 1 1 3 3 3 3 1 1 1 3 1 2a

0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 0 0 2 0 3

1 1 1 1 1 3 3 3 3 1 1 1 3 1

0 0 0 0 0 0 0 0 0 0 0 0 2 1 3a 0 0 0 0 0 0 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 3b

• 0 0 0 0 0 0 0 0 0 0 1 0 0 0 3c
• 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3d
• 0 0 0 0 0 0 0 0 1 0 0 0 0 0 4
• Iskhakov, Rust, and Schjerning (2013)

Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

RLS algorithm: running times

K = 3

Simultaneous moves n = 3 n = 4 Total number ESRs 4,782,969 3,948,865,611 Number of feasible ESRs 127 46,707 Time used 0.008 sec. 0.334 sec. Simultaneous moves n = 5 Total number ESRs 174,449,211,009,120,166,087,753,728 Number of feasible ESRs 192,736,405 Time used 45 min. Alternating moves n = 5 Total number ESRs 174,449,211,009,120,166,087,753,728 Number of feasible ESRs 1 Time used 0.006 sec.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Another Road Map for the rest of Talk

Part I: Results and Simulations from the Leapfroging Model

1 Resolution to the Bertrand investment paradox 2 Sufficient conditions for uniqueness of equilibria 3 Characterization of the set of equilibrium payoffs 4 Efficiency of equilibria 5 Leap-frogging or preemption and rent-dissipation 6 Danger of imposing symmetry 7 Limitations of homotopy parameter methods 8 Implications for empirical work

Part II: Dynamic Directional Games and RLS How to solve a general class of games using RLS

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Resolutions to Bertrand Investment Paradox

Earlier work: Fudenberg et al. (1983 RIE), Reinganum (1985 QJE), Fudenberg and Tirole (1985 ReStud), .... Riordan and Salant (1994 JIE): Preemption and rent dissipation (unique equilibrium) We show:

1 Many types of endog. coordination is possible in equilibrium

Leapfrogging (alternating investments) Preemption (investment by cost leader) Duplicative (simultaneous investments)

2 The equilibria are generally inefficient due to over-investment

Duplicative or excessively frequent investments

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Resolution to the Bertrand investment paradox

Theorem (Solution to Bertrand investment paradox) If investment is socially optimal at a state point (c1, c2, c) 2 S, then no investment by both firms cannot be an MPE outcome in the subgame starting from (c1, c2, c) in either the simultaneous or alternating move versions of the dynamic game.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Multiplicity of equilibria

Theorem (Sufficient conditions for uniqueness) In the dynamic Bertrand investment and pricing game a sufficient condition for the MPE to be unique is that

1 firms move in alternating fashion (i.e. m 6= 0), and, 2 for each c > 0 in the support of ⇡ we have ⇡(c|c) = 0. 1 Corollary: If firms move simultaneously, equilibrium is

generally not unique.

2 Corollary: If technological change is stochastic, equilibrium is

generally not unique.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Multiplicity of equilibria

Theorem (Number of equilibria in simultaneous move game) If investment is socially optimal, and the support of the Markov process {ct} for the state of the art marginal costs is the full interval [0, c0] (i.e. continuous state version), the simultaneous move Bertrand investment and pricing game has a continuum of MPE.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Pay-offs in the simultaneous move game

Theorem (Triangular payoffs in the simultaneous move game) Suppose that the {ct} process has finite support, that there are no idiosyncratic shocks to investment (i.e. ⌘ = 0) and that firms move simultaneously The (convex hull of the) set of the expected discounted equilibrium payoffs at the apex state (c0, c0, c0) 2 S is a triangle The vertices of this triangle are at the points (0, 0), (0, VM) and (VM, 0) where VM = vN,i(c0, c0, c0) is the expected discounted payoff to firm i in the monopoly equilibrium where firm i is the monopolist investor.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Pay-offs (deterministic tech progress, simultaneous moves)

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Pay-offs (stochastic tech progress, simultaneous moves)

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Pay-offs in the alternating move game

Theorem (Equilibrium payoffs in the alternating move game) The (convex hull of the) set of expected discounted equilibrium payoffs at the apex state (c0, c0, c0) 2 S of the alternating game is a strict subset of the triangle with the vertices (0, 0), (0, VM) and (VM, 0)

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Pay-offs: alternating vs simultaneous move games

10 20 30 40 50 60 71.776 10 20 30 40 50 60 71.776 Panel (a): Non−monotonic tech. progress 17826 equilibria, 792 distinct pay−off points Size: number of repetitions Color: efficiency

0.93 0.94 0.95 0.96 0.97 0.98 0.99 1

10 20 30 40 50 60 75.750 10 20 30 40 50 60 75.750 Panel (b): Simultaneous move 28528484 equilibria, 16510 distinct pay−off points Size: number of repetitions Color: efficiency

0.78 0.8 0.82 0.84 0.86 0.88 0.9 0.92 0.94 0.96 0.98

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Efficiency: alternating vs simultaneous move games

0.92 0.94 0.96 0.98 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Panel (c): Non−monotonic tech. progress 8913 equilibria, 7817 leapfrog, 2752 mixed strategy all equilibria leapfrog mixed strategy 0.75 0.8 0.85 0.9 0.95 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Panel (d): Simultaneous move 14264242 equilibria, 2040238 leapfrog, 2730910 mixed strategy all equilibria leapfrog mixed strategy Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Efficiency of equilibria

Simultaneous move game

Theorem (Inefficiency of mixed strategy equilibria) A necessary condition for efficiency in the dynamic Bertrand investment and pricing game is that along MPE path only pure strategy stage equilibria are played.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Riordan and Salant: Full Preemption

Theorem (Riordan and Salant, 1994) The continuous time investment game where

1 right to move alternates deterministically. 2 K(c) = K and is not prohibitively high. 3 technological progress is deterministic: c(t) is a continuous,

decreasing function has a unique MPE with preemptive investments: by only one firm and no investment in equilibrium by its opponent. rent dissipation: discounted payoffs of both firms in equilibrium is 0, so the entire surplus is wasted on excessively frequent investments by the preempting firm.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Riordan and Salant: Full preemption and rent dissipation

Confirm the result with high K and small dt

10 20 30 40 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time Marginal Costs, Prices Panel (a): Preemption and rent−dissipation c1 c2 cmonopoly c

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Underinvestment

Rent-dissapation is not a general outcome - disappears when K is low relative dt

5 10 15 20 25 30 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time Marginal Costs, Prices Panel (b): Underinvestment c1 c2 cmonopoly c

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Leap-frogging

Preemption is not the general outcome - disappears when K is even lower

5 10 15 20 25 30 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time Marginal Costs, Prices Panel (c): Leap−frogging c1 c2 cmonopoly c

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Random alternation ! Leapfrogging

Riordan and Salant’s result is not robust

5 10 15 20 25 30 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time Marginal Costs, Prices Panel (a): Random alternating moves c1 c2 cmonopoly c

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Random onestep technology ! Leapfrogging

Riordan and Salant’s result is not robust

10 20 30 40 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time Marginal Costs, Prices Panel (b): Non−monotonic tech. progress c1 c2 cmonopoly c

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Random multistep technology ! Leapfrogging

Riordan and Salant’s result is not robust

5 10 15 20 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time Marginal Costs, Prices Panel (c): Non−monotonic multistep tech. progress c1 c2 cmonopoly c

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Simultaneous moves: Leapfrogging

Riordan and Salant’s conjecture is wrong

5 10 15 20 25 30 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Time Marginal Costs, Prices Panel (d): Simultaneous move c1 c2 cmonopoly c

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Symmetric equilibria: V1(c1, c2, c) = V2(c2, c1, c)

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Limitations of homotopy approach

Homotopy parameter: ⌘ In each period each firm incurs additive random costs/benefit from not investing and investing ⌘ is a scaling parameter that index variance of idiosyncratic shocks to investment High ⌘ ! unique equilibrium ⌘ ! 0 ! multiple equilibria Problems: Multiplicity of equilibria ! too many bifurcations along the path Equilibrium correspondence is not lower hemi-continuous

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Limits of the homotopy approach

Equilibrium correspondance, alternating move game: VN,1(c0, c0, c0) vs. η

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Limits of the homotopy approach

Video: Set of equilibrium outcomes as variance of shocks decreases to zero

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Sample objective criterion

Identification and estimation problems in games with multiple equilibria θ L(θ) θ0 Equilibria−specific objective functions Upper envelope Discontinuities Local maxima Global maxima

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

Identification and estimation problems

Problems in games with multiple equilibria Discontinuities in the objective function ! non-standard asymptotic inference. Kinks in objective function ! non-standard asymptotic inference. Multiple local optima ! leaves researchers clueless whether true ✓ is found. No unique maximum ! set identification ! standard asymptotic theory breaks down. As the number of equilibria grows, these problems increase.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

How to estimate θ?

Estimation strategies Brute force approach We can use RLS to compute all equilibria and construct upper envelope and do NFXP. MPEC Egesdal, Lai and Su (2012): MPEC, handles the non-smoothness. Two step methods ! small sample problems K-step stable NPL Aguirregabiria and Mira (2014): Nonparametric estimates of equilibrium strategies act as an equilibrium selection rule

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Recursive Lexicographic Search (RLS) Algorithm Results and Simulations Limitations of homotopy approach

What now?

Compare existing methods: We can generate data from leap-frogging model.

Assuming a unique/multiple equilibrium at the true parameters. Assuming only one equilibrium played in the data, or one at each market (panel data).

Investigate the identifying power of the Markov assumption, by comparing

1

upper envelope likelihood for all MPE generated by RLS

2

likelihood derived from equilibrium sets generated by equilibrium contraction methods.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

PART II The General Case: Recursive Lexicographical Search in Directional Dynamic Games

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Components of Markovian (stochastic) game G

1 n players who take actions at times t 2 {1, 2, . . . , T}

where T may be 1,

2 A finite state space S and and action space A 3 state-specific constraint sets Ai(s) representing the set of

feasible actions of player i in state s

4 Von Neumann-Morgenstern utility functions ui(st, at) $

the payoff to player i in state st under actions at

5 Markovian state transition probability p(s0|s, a), where

a = (a1, . . . , an) is the vector of actions chosen by players

6 Player-specific discount factors (1, . . . , n), i 2 [0, 1) 7 Common knowledge of state s and all the objects above 8 Private information ✏i Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Behavior strategies

– feasible set of Markovian behavior strategies of the players in game G, i.e. n-tuple of mappings = (1, . . . , n) where i : S ! P(A) P(A) – set of all probability distributions on the set A. Feasibility requires that support(i(s)) ✓ Ai(s) for each s 2 S A pure strategy is a special case where i(s) places a unit mass on a single action a 2 Ai(s) Σ(G) – set of all feasible Markovian strategies of the game G

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Definition of equilibium

Definition (MPE) A Markov perfect equilibrium of the stochastic game G is a pair of feasible strategy n-tuple ⇤, and n-tuple of value functions V (s) = (V1(s), . . . , Vn(s)) where Vi : S ! R, such that

1 the system of Bellman equations of the problem is satisfied

(with the expectations taken probability distributions induced by opponents’ strategies in ⇤), and

2 the strategies constitute mutual best responses of the players,

and assign positive probabilities only to the actions in the set

• f maximizers of the Bellman equation.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Bellman equations

Vi(s) = max

a2Ai(s)

⇥ E

• ui(s, (a, ⇤

i(s))

+ iE (X

s02S

Vi(s0)p(s0|s, (a, ⇤

i(s)))

)# , where the expectation is taken over the probability distributions given by the opponents’ strategies ⇤

j , j 6= i.

If the maximizer over a 2 Ai(s) is unique, then ⇤

i (s) is a unit

mass on this optimal action. If there are multiple a 2 Ai(s) that attain the maximum, then ⇤

i (s) is a probability distribution whose support is a subset of

the set of a 2 Ai(s) that attain the maximum.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Bellman equations encode subgame perfection

In definition of MPE the notion of “subgame perfectness” is reflected by the restriction implicit in the “Principle of

• ptimality” of dynamic programming.

For each player’s strategy ⇤

i , the following holds

Definition (Principle of Optimality/One Shot Deviation Principle) “whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the fir st decision” (Bellman, 1957). Thus the equilibrium is subgame perfect

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Directional components in the state space

Suppose S can be written as S = D ⇥ X so s = (d, x) where we refer to d as the directional component of s and x as the non-directional component of s In the leapfrogging example, s = (c1, c2, c, m), so we have d = (c1, c2, c) and x = m. If G is a finite horizon game, then s = (t, x), where t denotes

• time. Then clearly d = t is the directional component of the

state space. How to formalize the notion of directionality?

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Markovian Games as Directed Graphs (DGs)

Games (a) to (c) are directional. What about game (d)?

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Strategy-specific partial order over D

Let be a feasible Markovian strategy in the game, and let ⇢(d0|(d, x), (d, x)) be the conditional hitting probability of the state d0 2 D starting from the state s = (d, x). Definition d0 σ d iff 9x 2 X ⇢(d0|(d, x), (d, x)) > 0 and 8x0 2 X ⇢(d|(d0, x0), ⇢(d0, x0)) = 0. Lemma σ is a strict partial order of D, i.e. it is irreflexive, asymmetric, and transitive.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

The No-Loop (Anti-Cycling) Condition

The partial order σ defines a clear notion of directionality in the strategy-induced law of motion of the game G. Because σ is transitive and asymmetric, there can be no loops (cycles) in any subset of comparable elements of D There are two ways (d0, d) can be non-comparable w.r.t σ: a) there may be no communication between d0 and d, b) there may be a loop (cycle) between d0 and d. Definition: No Loop Condition d0 6σ d and d0 6= d iff 8x 2 X ⇢(d0|(d, x), (d, x)) = 0 and 8x0 2 X ⇢(d|(d0, x0), (d0, x0)) = 0.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Bargaining over a stochastically shrinking pie

d1

d2

d3

d4

0.5 0.4 0.1 0.6 0.4 0.8 0.2 1.0

Notice that d2 and d3 are not comparable under the induced partial order σ. However the no-loop condition is satisified.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Bargaining over a shrinking/growing pie

d1

d2

d3

d4

0.6 0.4 0.6 0.4 0.5 0.3 0.2 1.0

This game induces the same partial order on D, σ, but the no-loop condition fails due to the loop (cycle) between d2 and d3.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Consistency in Induced Partial Orders and DDGs

Definition If and 0 are two feasible, Markovian strategies

• f G, we say the induced partial orders of D are consistent if

d0 σ d = ) d 6σ0 d0 Definition (DDG) A Dynamic Directional Game (DDG) is a finite state Markovian game G that satisfies the following two conditions:

1 Every feasible, Markovian strategy satisifies the No-Loop

Condition

2 Every pair of feasible, Markovian strategies and 0 induce

consistent partial orderings, σ and σ0, respectively.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Defining a Strategy-Independent Partial Order for a DDG

Definition Let σ and σ0 be two strategy-induced partial

• rders of D. We say that σ0 is a refinement of σ iff

8d, d0 2 D we have d0 σ d = ) d0 σ0 d Definition Let {σ | 2 Σ(G)} be a set of partial orders of D induced by the set of all feasible Markovian strategies in the game G, Σ(G). Then let G be the join (or coarsest common refinement) of the the set of partial orders {σ | 2 Σ(G)}. Theorem The join partial order G exists, is strategy-independent, and equals the transitive closure of the union of the partial orders in the set {σ | 2 Σ(G)}.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DDGs and DAGs

Definition A directed acyclic graph (DAG) is a directed graph with no cycles connecting any two vertices of the graph. Lemma If G is a finite state DDG, then the directed graph induced by G, D(G), is a DAG. We use D(G) to partition D into T elements that are totally

• rdered and form the indices we need to do state recursion to

find a MPE of G, D = {D1, D2, . . . , DT }. The ⌧ index is a generalization of the notion of a time index used in standard backward induction arguments. We say that ⌧ indexes the stage of the DDG G.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-recursion: identifying the stages of a finite state DDG

Definition The terminal nodes of a DAG D(G) are the set of all vertices d 2 D that have no descendants. We let N(D(G)) denote the set of terminal nodes of D(G). Definition The non-terminal sub-DAG of a DAG D(G) is the DAG D1(G) given by D1(G) = D(G) N(D(G)). Thus, D1(G)) is the sub-DAG of D(G) the results when you remove its terminal nodes N(D(G)). Definition (DAG-Recursion) Define a sequence of sub-DAGs

• f D(G), {Dτ(G)} recursively by

Dτ+1(G) = Dτ(G) N(Dτ(G)).

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-recursion terminates with the empty set

Definition The stages of the finite state DDG G are given by {S1, S2, . . . , Sτ, Sτ+1, . . . , ST } where Sτ = (Dτ ⇥ X) where {D1, . . . , DT } is the partition of D defined by the DAG-recursion.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Finding the Stages of a Dynamic Directional Game

A B C D E F G H I J K L M

A DDG has a graph that is a DAG. Is this graph a DAG?

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Recursion on Directed Acyclic Graphs (DAG-recursion)

A B C D E F G H I J K L M

Identify the terminal nodes of the DAG, S1 = {G, M}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-recursion: eliminate terminal nodes

A B C D E F H I J K L

Now eliminate the terminal nodes to obtain the first “sub-DAG”

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-recursion: identify the terminal nodes of the sub-DAG

A B C D E F H I J K L

Now identify the terminal nodes of the 1st sub-DAG, S2 = {K}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-recursion: eliminate terminal nodes of sub-DAG 1

A B C D E F H I J L

We now have identified stages S1 = {G, M}, S2 = {K}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-Recursion: identify terminal nodes of sub-DAG 2

A B C D E F H I J L

Now identify the terminal nodes of the 2nd sub-DAG, S3 = {L}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-recursion: eliminate I,J nodes to get sub-DAG 2

A B C D E F H I J

We now have identified stages S1 = {G, M}, S2 = {K}, S3 = {L}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-Recursion: eliminate E,H nodes to get sub-DAG 3

A B C D E F H

S1 = {G, M}, S2 = {K}, S3 = {L}, S4 = {I, J}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-Recursion: eliminate D-node to get sub-DAG 4

A B C D F

S1 = {G, M}, S2 = {K}, S3 = {L}, S4 = {I, J}, S5 = {E, H}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-Recursion: eliminate C-node to get sub-DAG 5

A B C F

S1 = {G, M}, S2 = {K}, S3 = {L}, S4 = {I, J}, S5 = {E, H}, S6 = {D}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-Recursion: eliminate B-node to get sub-DAG 6

A B F

S1 = {G, M}, S2 = {K}, S3 = {L}, S4 = {I, J}, S5 = {E, H}, S6 = {D}, S7 = {C}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-Recursion: eliminate A-node to get sub-DAG 7

A F

S1 = {G, M}, S2 = {K}, S3 = {L}, S4 = {I, J}, S5 = {E, H}, S6 = {D}, S7 = {C}, S8 = {B}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-Recursion: eliminate F-node to get sub-DAG 8

F

S1 = {G, M}, S2 = {K}, S3 = {L}, S4 = {I, J}, S5 = {E, H}, S6 = {D}, S7 = {C}, S8 = {B}, S9 = {A}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

DAG-Recursion: we’re done! No nodes left to eliminate

S1 = {G, M}, S2 = {K}, S3 = {L}, S4 = {I, J}, S5 = {E, H}, S6 = {D}, S7 = {C}, S8 = {B}, S9 = {A}, S10 = {F}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Subgames and stage games of DDG

Given the partition {D1, . . . , DT } of the directional component D Let Ωτ = [T

t=τSt where St = Dt ⇥ X

Gτ – Stage ⌧ subgame of G DDG with state space Ωτ and other elements of the game (number of players, time horizon, utility functions, discount factors, action sets and laws of motion) be properly restricted for this state space versions of the element of the original game G Gτ(d) – d-subgame of G Similarly defined subgame on state space Ωτ(d) = {d ⇥ X} [

• [T

t=τ+1St

• Iskhakov, Rust, and Schjerning (2013)

Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State recursion over subgames of DDGs

State recursion involves finding a MPE of the overall game G inductively, starting by finding MPEs at all points in the endgame, i.e. the stage T subgame GT , and proceeding by backward induction over the stages of the game, from stage T 1 to stage T 2 until the initial stage 1 is reached and solved. When stage 1 is solved in this backward induction procedure, effectively the whole G is also solved, as follows from the following lemma. Lemma If G is a finite state DDG, and G1 is its stage 1 subgame, then G = G1.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Multiplicity of Equilibrium and Equilibrium Selection Rules

E(G) – the set of all MPE of G. Multiple MPEs in some of the d-subgames Gτ(d) ! the equilibria in the d0-subgames at the earlier stages ⌧ 0 < ⌧ will be dependent on which of the MPEs of the d-subgames is played at stage ⌧ ) Solution computed with backward induction is dependent

• n the equilibrium selection rule (ESR) that selects one of the

equilibria at every d-subgame of G, and thus induces (or selects) a particular MPE in the whole game. e(G) 2 E(G) – a particular selected MPE from the set of all MPE of G.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Definition of Equilibrium Selection Rule

Definition (Equilibrium selection rule) Let Γ denote a deterministic rule for selecting one of the MPE from every d-subgame Gτ(d), i.e. e(Gτ(d)) = Γ (E(Gτ(d))) 8d 2 D. (1) By selecting an equilibrium in every d-subgame, ESR Γ also induces (or selects) an equilibrium in every subgame Gτ, e(Gτ) = Γ (E(Gτ)). Define the projections eσ(G) = ⇤ and eV (G) = V that pick each of these objects from a given equilibrium.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Stage τ Continuation strategies

Definition (Stage ⌧ continuation strategy) A stage ⌧ continuation strategy τ(s|Sτ, eσ(Gτ+1)) is any feasible Markovian strategy for points s 2 Sτ that reverts to a MPE strategy eσ(Gτ+1) in the stage ⌧ + 1 subgame Gτ+1. That is, τ(s|Sτ, eσ(Gτ+1)) = ⇢ (s) if s 2 Sτ, eσ(Gτ+1)

• therwise.

(2)

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Generalized Stage Games

Definition (Stage game) Let G be a finite state DDG, and consider a particular stage of the game ⌧ 2 {1, . . . , T } and d 2 Dτ. A d-stage game, SGτ(d), is a d-subgame Gτ(d) where the set of feasible strategies is restricted to continuation strategies, i.e. if Σ(SGτ(d)) is the set of feasible, Markovian strategies of the stage game and Σ(Gτ(d)) is the set of feasible Markovian strategies of the d-subgame Gτ(d), then we have 2 Σ(SGτ(d)) iff (s) = τ(s|(d ⇥ X), eσ(Gτ+1)), s 2 (d ⇥ X) [ Ωτ+1. (3)

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State recursion decomposes a big game into smaller ones

It follows that Σ(SGτ(d)) ⇢ Σ(Gτ(d)), i.e. the set of feasible Markovian strategies in a d-stage game SGτ(d) is a subset of the set of feasible Markovian strategies in the d-subgame Gτ(d). Similarly the set of feasible Markovian strategies in the stage game Gτ is a subset of the feasible Markovian strategies in the stage ⌧ subgame Gτ. By restricting strategies in this way, we reduce the problem of finding MPE strategies of a stage game SGτ(d) to the much smaller, more tractable problem of computing a MPE on the reduced state space (d ⇥ X) instead of the much larger state space Ωτ(d).

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State recursion produces a subgame perfect equilibrium

Theorem (Subgame perfection) Let E(SGτ(d)) be the set of all MPE of the stage game SGτ(d) and let E(Gτ(d)) be the set of all MPE of the d-subgame Gτ(d). Then we have E(SGτ(d)) = E(Gτ(d)) (4) i.e. there is no loss in generality from computing all MPE of every d-subgame Gτ(d) by restricting the search for equilibria to finding all MPE of the corresponding stage game SGτ(d) using only continuation strategies.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State recursion algorithm

For ⌧ = T , T 1, . . . , 1 do

For i = 1, . . . , nτ do

1

compute E(SGτ(di,τ))

2

using an equilibrium selection rule Γ, select a particular MPE from E(SGτ(di,τ)), e(SGτ(di,τ)) = Γ(E(SGτ(di,τ)))

3

e(SGτ(di,τ)) is a MPE of the d-subgame Gτ(di,τ)

union of the MPEs for each d-stage game {e(SGτ(di,τ)|i = 1, . . . , nτ} is a MPE for the stage game at stage ⌧ e(Gτ)

E(G1) = E(G)

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Convergence of the state recursion algorithm

Theorem (Convergence of State Recursion) Let G be a finite state DDG. Given an equilibrium selection rule Γ the state recursion algorithm computes an MPE of G.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion: DAG recursion in reverse

To get ready to do state recursion, which is a backward induction in the DAG, reverse index the stages of the game S10 = {G, M}, S9 = {K}, S8 = {L}, S7 = {I, J}, S6 = {E, H}, S5 = {D}, S4 = {C}, S3 = {B}, S2 = {A}, S1 = {F}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step T = 10: the “end game”

G M

Find all equilibria in the end game states S10 = {G, M}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion T 1 = 9: Now find all MPE in S9 = {K}

G K M

Find all MPE in the generalized stage game S9 = {K}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step T 2 = 8: Find all MPE in S8 = {L}

G K L M

Find all MPE in the generalized stage game S8 = {L}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step τ = 7: Find all MPE in S7 = {I, J}

G I K L M

Find all MPE in the generalized stage game S7 = {I, J}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step τ = 6: Find all MPE in S6 = {E, H}

E G H I J K L M

Find all MPE in the generalized stage game S6 = {E, H}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step τ = 5: Find all MPE in S5 = {D}

D E G H I J K L M

Find all MPE in the generalized stage game S5 = {D}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step τ = 4: Find all MPE in S4 = {C}

C D E G H I J K L M

Find all MPE in the generalized stage game S4 = {C}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step τ = 3: Find all MPE in S3 = {B}

B C D E G H I J K L M

Find all MPE in the generalized stage game S3 = {B}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step τ = 2: Find all MPE in S2 = {A}

A B C D E G H I J K L M

Find all MPE in the generalized stage game S2 = {A}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

State Recursion step τ = 1: Find all MPE in S1 = {F}

A B C D E F G H I J K L M

Find all MPE in the generalized stage game S1 = {F}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

We’re done: state recursion has found a MPE of G

A B C D E F G H I J K L M

It found one MPE of the DDG G given ESR Γ

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Finding all MPE

Approach:

1 Formalize the notion of ESR Γ on a computer 2 Make a loop over all ESR and solve the model for each Γ

Problems: Choice of a particular MPE for any stage game at any stage may alter the set and even the number of stage equilibria at earlier stages Need to find feasible ESRs

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Recursive Lexicographic Search Algorithm

Building blocks of RLS algorithm:

1 State recursion algorithm solves the game conditional on

equilibrium selection rule (ESR)

2 RLS algorithm efficiently cycles through all feasible ESRs

Properties of RLS algorithm: Complete: Computes all MPE equilibria of the game Fast: time spent of search of feasible ESRs is negligible in comparison to time spent on solving the game

Efficiently skip infeasible ESRs Re-use results of previously computed subgames

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Assumptions of RLS

1 There is a method to find all MPE in every d-stage game (i.e.

equilibria within the class of continuation strategies)

2 The number of equilibria in every d-stage game is finite 3 ) RLS finds all MPE of the DDG G.

RLS also works if this algorithm can only find some of the equilibria of d-stage games. In the latter case RLS is not guaranteed to find all MPE of G, but it can still find, potentially, very many MPE of G.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Efficiency of RLS

1 [Decompose] State recursion decomposes a large, complex

game into a sequence of much simpler component stage games

2 [Reuse] RLS minimizes the computational cost when if

traverses the set of all MPE

3 [Jump] RLS is caplable to jump from one feasible ESR to

next in one step

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Equilibrium Selection Strings (ESS)

ESS formalizes the ESR Γ K – the least upper bound on the number of possible equilibria in any stage game of G.

Implementation does not require the prior knowledge of K

N – the total number of d-substages of the DDG G. The state recursion algorithm must loop over all N of these substages to find a MPE in the stage games that correspond to each of these N d-stages to construct a MPE of G. Equilibria in every d-stage games are indexed {0, 1, . . . , K 1}

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Equilbrium Selection Strings (ESS), continued

Definition (Equilibrium Selection Strings) An equilibrium selection string (ESS), denoted by , is a vector in Z N

+ whose components are integers expressed in base K arithmetic,

i.e. each coordinate (or “digit”) of takes values in the set {0, 1, . . . , K 1}. = (T , T 1, . . . , 1), τ = (1,τ, . . . , nτ,τ) 0 = (0, . . . , 0) – the initial ESS that consists of N zeros, always feasible assuming at least one equilibrium exists in every d-stage game

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Ordering of stages/digits in ESSs

Elements of are ordered in a particular way τ are ordered from right to left corresponding to stages of G from ⌧ to T Higher digits of ESS correspond to later stages of G ) when digit j is changed in the loop over ESS, only stages to the right (⌧ < j) have to be resolved

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Enumerating all possible Equilibrium Selection Strings

K N possible equilibrium strings for the DDG G

this is an upper bound on the number of possible MPE of G

The loop starts from 0 = (0, . . . , 0) One step in the loop is mod(K) addition +1 Second ESR is 1 = (0, . . . , 0, 1) = 1 in base-K representation

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Does γ1 correspond to an MPE of G?

1 = (0, 0, . . . , 0, 1) may or may not correspond to a MPE of G because there may be only a single MPE at the d1,n1-stage game of G. If there is only a single MPE in this substage ! the equilibrium string 1 is infeasible because the only feasible equilirbrium index for the first d-stage is 0 Definition (Feasible Equilibrium Selection String) An equilibrium string is feasible if all of its digits index a MPE that exists at each of the corresponding d-stage games of G.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Tracking the number of MPE at each substage

Definition (ne() string) Let the N ⇥ 1 vector ne() be the maximum number of MPE at each stage game of G under the ESR implied by the equilibrium string . Define ne() using the same format as the equilibrium string, so that the digits of the equilibrium string are in one to

• ne correspondence with the elements of the vector ne() as

follows: ne() = ⇣ neT , neT 1

• >T 1
• , . . . , ne1
• >1

⌘ , where >τ =

• τ+1, . . . , T
• is a T ⌧ ⇥ 1 vector listing the

equilibrium selection sub-string for stages of G higher than ⌧.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Characterization of Feasible ESSs

We use the notation neτ(>τ) to emphasize that the number

• f MPE at stage ⌧ depends only on the equilibria selected at

higher stages of G. Notice that in the endgame T there are no further stages of the game, so the maximum number of MPE in this stage, nT does not depend on any substring of the equilibrium string . Lemma The ESS is feasible if and only if i,τ < nei,τ(>τ), ⌧ = 1, . . . , T , i = 1, . . . , nτ

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

The Jump Function

J () is the “smallest” ESS after that is also a feasible ESS. Definition (Jump function) Let J : Z N

+ ! Z N + [ [STOP] be defined by

J () = ⇢ argminγ0{0 : 0 > and 0 is feasible} [STOP] if 6 90 : 0 > and 0 is feasible

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Variable base arithmetics

Replace the base-K (mod(K)) arithmetics with variable base arithmetics Let (3 1 2) be bases ! Allowed digits in the numbers are {0, 1, 2}, {0} and {0, 1} The 3-digit numbers in this system are: 0 0 0 + 1 ! 0 0 1 + 1 ! 1 0 0 + 1 ! 1 0 1 + 1 ! 2 0 0 + 1 ! 2 0 1

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

ESS γ in variable base arithmetics

Bases: ne() = ⇣ neT , neT 1

• >T 1
• , . . . , ne1
• >1

⌘ Successor function S : Z N

+ ! Z N : S() = + 1

Defined correctly in variable bases: +1 = ⇢ ( . . . , (1) + 1) if (1) + 1 < ne(1), (. . . , (jk)+1, 0, . . . , 0)

• therwise,

where k : (jk)+1 < ne(jk) In the latter case dependent bases change to

• ne(1), . . . , ne(jk), ˜

ne(jk+1), . . . , ˜ ne(N) But + 1 is still a well-defined number in this new bases because all digits with new bases are zeros

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Relation between the Successor and Jump Function

S0 – augmented with [STOP] signal when more than N digits are needed for the successor result Theorem Assume that ESS are expressed in variable bases as above. If is a feasible ESS, then J () = S0(). In other words, when the bases for the equilibrium selection string are chosen in the right way, the function that returns next feasible ESS is just a successor function The number of steps RLS takes is the same as number of MPE in the model!

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

RLS Algorithm

1 Set 0 = (0, . . . , 0), k = 0 2 Solve for an MPE given the ESS k with State Recursion 3 Fix the bases for the ESS at computed ne() 4 Apply a successor function to find next feasible ESS

k+1 = S0(k)

5 Stopping rule: k+1 ?

= [STOP]

6 Return to step 2 Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Definition of Directional Dynamic Games (DDGs) DAGs and State Recursion algorithm Recursive Lexicographical Search algorithm (RLS)

Main result of the RLS Algorithm

Theorem (Decomposition theorem) Assume there exists an algorithm that can find all MPE of every stage game of the DDG G, and that the number of these equilibria is finite in every stage game. Then the RLS algorithm finds all MPE of DDG G in at most |E(G)| steps, which is the total number of MPE of the DDG G.

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Theoretical conclusions Methodological conclusions

Theoretical conclusions

Endogenous coordination (e.g. leapfrogging) is possible in equilibrium of dynamic Bertrand investment game Leapfrogging gives new interpretation of the price wars Numerous MPE equilibria and ”Folk theorem”-like result Most equilibria are inefficient due to over-investment

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria PART I : Solving the Leapfrogging Model with RLS PART II: Recursive Lexicographical Search in General DDGs Conclusions Theoretical conclusions Methodological conclusions

Methodological conclusions

When equilibrium is not unique the computation algorithm inadvertently acts as an equilibrium selection mechanism State recursion algorithm is preferred to time iterations Imposing symmetry restriction on equilibria knocks out most equilibria in the model Plethora of Markov perfect equilibria leads to new paradox: How can firms, without any explicit communication, coordinate on a single equilibrium in these games when there is generally such a vast multiplicity of possible equilibria. As the number of equilibria grows, problems associated with estimation increase (identification, estimation, inference and computation).

Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games