Polymatrix Games: Algorithms and Applications Rahul Savani - - PowerPoint PPT Presentation

Polymatrix Games: Algorithms and Applications

Rahul Savani

Department of Computer Science University of Liverpool Tutorial at the Conference on Web and Internet Economics WINE 2015

Some of talk relates to joint work with Argyrios Deligkas, John Fearnley, Paul Goldberg, Paul Spirakis, and Bernhard von Stengel

What is a polymatrix game?

Polymatrix games are many-player games

What is a polymatrix game?

Polymatrix games are many-player games For us, they are graphical games: player interactions are captured by an interaction graph (though sometimes this graph is assumed to be complete)

What is a polymatrix game?

Polymatrix games are many-player games For us, they are graphical games: player interactions are captured by an interaction graph (though sometimes this graph is assumed to be complete) They model pairwise interactions

What is a polymatrix game?

Polymatrix games are many-player games For us, they are graphical games: player interactions are captured by an interaction graph (though sometimes this graph is assumed to be complete) They model pairwise interactions Nodes correspond to players

What is a polymatrix game?

Polymatrix games are many-player games For us, they are graphical games: player interactions are captured by an interaction graph (though sometimes this graph is assumed to be complete) They model pairwise interactions Nodes correspond to players Edges correspond to bimatrix games

What is a polymatrix game?

Polymatrix games are many-player games For us, they are graphical games: player interactions are captured by an interaction graph (though sometimes this graph is assumed to be complete) They model pairwise interactions Nodes correspond to players Edges correspond to bimatrix games Each player chooses a single strategy for all his bimatrix games and receives the sum of the payoffs from his bimatrix games

History of polymatrix games

Introduced in: Janovskaya (1968) Equilibrium points in polymatrix games (in Russian) Latvian Mathematical Collection We will touch on the following papers here:

Both classical: Eaves 1973 [9] Howson 1972 [15] Howson & Rosenthal 1974 [16] Miller & Zucker 1991 [19] And more recent: Cai et al 2015 [4] Fearnley et al 2015 [8] Mehta 2012 [18] Govindan & Wilson 2004 [14] Rubinstein 2015 [21]

Polymatrix game

n players i = 1, . . . , n finite pure strategy sets Si payoff matrices for every player i and j i Aij ∈ R|Si|×|Sj|

Polymatrix game

n players i = 1, . . . , n finite pure strategy sets Si payoff matrices for every player i and j i Aij ∈ R|Si|×|Sj| For mixed profile (x1, . . . , xn), the payoff to player i is ui(x1, . . . , xn) =

• ij

(xi)⊤Aijxj

Example polymatrix game

1 2 3 a b a 0,0 2,2 b 2,2 0,0 a b a 0,0 2,2 b 2,2 0,0 a b a 0,0 1,1 b 1,1 0,0

Example polymatrix game

1 2 3 a b a 0,0 2,2 b 2,2 0,0 a b a 0,0 2,2 b 2,2 0,0 a b a 0,0 1,1 b 1,1 0,0

Equilibria: 1 2 3 a b b b b a (0.5, 0.5) (0.5, 0.5) (0.5, 0.5)

Advantage: succinctness

In terms of the number of players, the size of a strategic-form game is exponential polymatrix game is polynomial (quadratic)

# players # actions (per player) # payoff entries strategic-form n k n × k n polymatrix 2k 2 × (n

2)

Applications

Polymatrix games are general modelling tool for multi-player games via pairwise interactions We will also discuss some other applications from the literature:

1

Relaxation Labelling Problems for Artificial Neural Networks [19]

2

Graph Transduction in Machine Learning [10]

3

To model 2-player Bayesian Games [16]

4

As a sub-routine for solving general multi-player games [14]

Take-home message

Many things carry over from bimatrix to polymatrix games: Rational equilibria Formulation as a Linear Complementarity Problem Applicability of complementary pivoting algorithms (e.g. Lemke-Howson, Lemke) Descent methods using Linear Programming for finding Approximate Equilibria

Take-home message

Many things carry over from bimatrix to polymatrix games: Rational equilibria Formulation as a Linear Complementarity Problem Applicability of complementary pivoting algorithms (e.g. Lemke-Howson, Lemke) Descent methods using Linear Programming for finding Approximate Equilibria There are also important differences. For polymatrix games: PPAD-hard to find ǫ-Nash equilibrium for constant ǫ Finding a pure equilibrium is PLS-hard

Outline

1 Nash equilibria of bimatrix games 2 Linear Complementarity Problems (LCPs) 3 The Lemke–Howson Algorithm and the class PPAD 4 Lemke’s algorithm 5 PLS-hardness of pure equilibria, Graph Transduction 6 Reduction from Polymatrix Game to LCP 7 Descent method for ǫ-Nash equilibria of polymatrix games 8 Other recent work on polymatrix games

Nash equilibria of bimatrix games

❅ ❅

I II T M B l r 3 3 1 2 5 2 6 4 3

Nash equilibria of bimatrix games

❅ ❅

I II T M B l r 3 3 1 2 5 2 6 4 3 Nash equilibrium = pair of strategies x, y with x best response to y and y best response to x

Mixed equilibria

❅ ❅

I II T M B l r 3 3 1 2 5 2 6 4 3

Ay =         3 3 2 5 6        

• 1/3

2/3 T =         3 4 4         xTB =         1/3 2/3        

T 

       1 2 4 3         =

• 8/3

8/3

• nly only pure best responses can

have probability > 0

Outline

1 Nash equilibria of bimatrix games 2 Linear Complementarity Problems (LCPs) 3 The Lemke–Howson Algorithm and the class PPAD 4 Lemke’s algorithm 5 PLS-hardness of pure equilibria, Graph Transduction 6 Reduction from Polymatrix Game to LCP 7 Descent method for ǫ-Nash equilibria of polymatrix games 8 Other recent work on polymatrix games

Linear Complementarity Problem

Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that z ≥ 0 ⊥ w = q + Mz ≥ 0

⊥ means orthogonal:

zTw = 0 ⇔ ziwi = 0 all i = 1, . . . , n

Linear Complementarity Problem

Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that z ≥ 0 ⊥ w = q + Mz ≥ 0

⊥ means orthogonal:

zTw = 0 ⇔ ziwi = 0 all i = 1, . . . , n If q ≥ 0, the LCP has trivial solution w = q , z = 0.

LP in inequality form

primal : max cTx subject to Ax ≤ b x ≥ 0 dual : min yTb subject to yTA ≥ cT y ≥ 0

LP in inequality form

primal : max cTx subject to Ax ≤ b x ≥ 0 dual : min yTb subject to yTA ≥ cT y ≥ 0 Weak duality: x, y feasible (fulfilling constraints) ⇒ cTx ≤ yTAx ≤ yTb

LP in inequality form

primal : max cTx subject to Ax ≤ b x ≥ 0 dual : min yTb subject to yTA ≥ cT y ≥ 0 Strong duality: primal and dual feasible ⇒ ∃ feasible x, y : cTx = yTb (x, y optimal)

LCP generalizes LP

LCP encodes complementary slackness of strong duality: cTx = yTAx = yTb ⇔ (yTA − cT)x = 0, yT(b − Ax) = 0. ≥ 0 ≥ 0 ≥ 0 ≥ 0

LCP generalizes LP

LCP encodes complementary slackness of strong duality: cTx = yTAx = yTb ⇔ (yTA − cT)x = 0, yT(b − Ax) = 0. ≥ 0 ≥ 0 ≥ 0 ≥ 0 LP ⇔ LCP

• x

y

• z

≥ 0 ⊥

• −c

b

• q

+

• AT

−A

• M
• x

y

• z

≥ 0

Outline

1 Nash equilibria of bimatrix games 2 Linear Complementarity Problems (LCPs) 3 The Lemke–Howson Algorithm and the class PPAD 4 Lemke’s algorithm 5 PLS-hardness of pure equilibria, Graph Transduction 6 Reduction from Polymatrix Game to LCP 7 Descent method for ǫ-Nash equilibria of polymatrix games 8 Other recent work on polymatrix games

Symmetric equilibria of symmetric games

Given: n × n payoff matrix A for row player AT for column player mixed strategy x = probability distribution on {1,...,n} ⇔ x ≥ 0 , 1Tx = 1 equilibrium (x, x) ⇔ x best response to x Remark: As general as m × n games (A, B).

Best responses

Given: n × n payoff matrix A, mixed strategy y of column player Ay = vector of expected payoffs against y, components (Ay)i x best response to y ⇔ x maximizes expected payoff xTAy best response condition: ⇔ ∀i : xi > 0 ⇒ (Ay)i = u = maxk (Ay)k

Symmetric equilibria as LCP solutions

equilibrium (x, x) of game with payoff matrix A ⇔ x best response to x ⇔ 1Tx = 1, x ≥ 0

⊥ Ax ≤ 1u

w.l.o.g. A > 0 ⇒ u > 0, equilibrium (x, x) ⇔ z = (1/u) x ( 1/u = 1Tz ), z ≥ 0

⊥ Az ≤ 1 "equilibrium z"

Best response polyhedron 2 1 1 1 2 2 0 A =

1

x

2

x u

< >

x 0, { ( , ) | x u } 1Tx= 1, x u A 1 1

Best response polyhedron 1 1 2 2 2 1 1 2 2 1 1 1 2 2 0 A =

1

x

2

x u

< >

x 0, { ( , ) | x u } 1Tx= 1, x u A 1 1

Best response polyhedron 1 1 2 2 2 1 1 2 2 1 1 1 2 2 0 A =

1

x

2

x u

< >

x 0, { ( , ) | x u } 1Tx= 1, x u A 1 (2/3, 1/3) (completely labeled) equilibrium 1

Projective transformation 1 2 2 0 A =

1

x

2

x u

< >

x 0, { ( , ) | x u } 1Tx= 1, x u A 1

>

x 0,

<

x A 1 { ( , ) | 1 x } 1

>

z 0,

<

z A 1 Best response polytope { | z } 2 1 2 1 1 2 2 0 A =

2

z

1

z

Symmetric Lemke−Howson algorithm

1

z

2

z

z3

(bottom) (back)

2 1

1

2 3 3

Symmetric Lemke−Howson algorithm 1 missing label

1

z

2

z

z3

(bottom) (back)

2 1

1

2 3 3

Symmetric Lemke−Howson algorithm 1 missing label

1

z

2

z

z3

(bottom) (back)

2 1

1

2 3 3

Symmetric Lemke−Howson algorithm 1 missing label

1

z

2

z

z3

(bottom) (back)

2 1

1

2 3 3

1 missing label Symmetric Lemke−Howson algorithm

1

z

2

z

z3

(bottom) (back)

2 1

1

2 3 3

1 missing label Symmetric Lemke−Howson algorithm

1

z

2

z

z3

(bottom) (back)

2 1

1

2 3 3

found label 1 Symmetric Lemke−Howson algorithm

1

z

2

z

z3

(bottom) (back)

2 1

1

2 3 3

Why Lemke-Howson works

LH finds at least one Nash equilibrium because

• finitely many "vertices"

for nondegenerate (generic) games:

• unique starting edge given missing label
• unique continuation

⇒ precludes "coming back" like here:

END OF LINE (Papadimitriou 1991)

start end Given a graph G of indegree/outdegree at most 1, and a start vertex of indegree 0 and outdegree 1, find another vertex of degree 1

END OF LINE (Papadimitriou 1991)

start 0000 0101 end Catch: graph is exponentially large defined by two boolean circuits S, P that take a vertex in {0, 1}n and output its successor and predecessor S(0000) = 0101 P(0101) = 0000

END OF LINE (Papadimitriou 1991)

start end A problem belongs to PPAD if it is reducible in poly-time to END OF LINE; and PPAD-complete if END OF LINE is reducible to it.

END OF LINE (Papadimitriou 1991)

start end A problem belongs to PPAD if it is reducible in poly-time to END OF LINE; and PPAD-complete if END OF LINE is reducible to it. Not to be confused with OTHER END OF THIS LINE

• utput unique vertex end

found by “following the line” from the start – this is PSPACE-hard

PPAD-hardness for bimatrix games

Theorem (DGP06, CDT06 [5, 6]) It is PPAD-complete to compute an exact Nash equilibrium of a bimatrix game. Later we will see PPAD-hardness for approximate equilibria

• f bimatrix and polymatrix games

Outline

1 Nash equilibria of bimatrix games 2 Linear Complementarity Problems (LCPs) 3 The Lemke–Howson Algorithm and the class PPAD 4 Lemke’s algorithm 5 PLS-hardness of pure equilibria, Graph Transduction 6 Reduction from Polymatrix Game to LCP 7 Descent method for ǫ-Nash equilibria of polymatrix games 8 Other recent work on polymatrix games

Costs instead of payoffs

1 2 2 1 2 0 → 1 3 aik 3 − aik payoff cost with new cost matrix A > 0 : equilibrium z ⇔ z ≥ 0 ⊥ Az ≥ 1

Polyhedral view 1 + 3 2 + 1

1

z ≥ 0

2

z

1

z ≥ 1

2

z

1

z ≥ 1

2

z ≥ 0

1

z

2

z 1 2 1 2

Lemke's algorithm

given LCP z ≥ 0 ⊥ w = q + Mz ≥ 0

Lemke's algorithm

augmented LCP z ≥ 0 ⊥ w = q + Mz + dz0 ≥ 0 z0 ≥ 0

Lemke's algorithm

augmented LCP z ≥ 0 ⊥ w = q + Mz + dz0 ≥ 0 z0 ≥ 0 where d > 0 covering vector z0 extra variable z0 = 0 ⇔ z ⊥ w solves original LCP

Lemke's algorithm

augmented LCP z ≥ 0 ⊥ w = q + Mz + dz0 ≥ 0 z0 ≥ 0 Initialization: z = 0 ⊥ w = q + dz0 ≥ 0 z0 ≥ 0 minimal ⇒ wi = 0 for some i pivot z0 in, wi out, ⇒ can increase zi while maintaining z ⊥ w .

Lemke's algorithm for M = 2 1 , d = 2 1 3 1

w1 −1 2 1 2 = + z1 +

z2 + z0

w2 −1 1 3 1 w1 1 −5 −2 = + z1 +

z2 + w2

z0 1 −1 −3 −1

w1 −1 2 1 2 = + z1 +

z2 + z0

w2 −1 1 3 1 w1 1 −5 −2 = + z1 +

z2 + w2

z0 1 −1 −3 −1 z2 0.2 −0.2 −0.4 = + z1 +

w1 + w2

z0 0.4 −1 0.6 0.2

w1 1 −5 −2 = + z1 +

z2 + w2

z0 1 −1 −3 −1 z2 0.2 −0.2 −0.4 = + z1 +

w1 + w2

z0 0.4 −1 0.6 0.2 z2 0.2 −0.2 −0.4 = + z0 +

w1 + w2

z1 0.4 −1 0.6 0.2

Polyhedral view of Lemke

Polyhedral view of Lemke

1

z

2

z 1 2 1 2

Polyhedral view of Lemke z

1

z

2

z 1 2 1 2

Polyhedral view of Lemke

1

z

2

z z 1 2 1 2

Polyhedral view of Lemke

1

z

2

z z 1 2 1 2

Polyhedral view of Lemke

1

z

2

z z 1 2 1 2

Polyhedral view of Lemke

1

z

2

z z z = 0 1 2 1 2

Outline

1 Nash equilibria of bimatrix games 2 Linear Complementarity Problems (LCPs) 3 The Lemke–Howson Algorithm and the class PPAD 4 Lemke’s algorithm 5 PLS-hardness of pure equilibria, Graph Transduction 6 Reduction from Polymatrix Game to LCP 7 Descent method for ǫ-Nash equilibria of polymatrix games 8 Other recent work on polymatrix games

The class PLS (Polynomial Local Search)

s Given a starting solution s ∈ S = Σn a P-time algorithm that computes the cost c(s) a P-time function that computes a neighbouring solution s′ ∈ N(s) with lower cost, i.e. s.t. c(s′) < c(s), or reports that no such neighbour exists: find a local optimum of the cost function c “every DAG has a sink”

Local Max Cut

Find local optimum of Max Cut with the FLIP-neighbourhood (exactly one node can change sides) Sch¨ affer and Yannakakis [22] showed that Local Max Cut is PLS-complete (via an extremely involved reduction) Local Max Cut is to PLS what 3-SAT is to NP

Local Max Cut

Find local optimum of Max Cut with the FLIP-neighbourhood (exactly one node can change sides) Sch¨ affer and Yannakakis [22] showed that Local Max Cut is PLS-complete (via an extremely involved reduction) Local Max Cut is to PLS what 3-SAT is to NP 1 2 3 4 1 1 −4 3 1 −2

Local Max Cut

Find local optimum of Max Cut with the FLIP-neighbourhood (exactly one node can change sides) Sch¨ affer and Yannakakis [22] showed that Local Max Cut is PLS-complete (via an extremely involved reduction) Local Max Cut is to PLS what 3-SAT is to NP 1 2 3 4 1 1 −4 3 1 −2 Solutions: {{1, 3, 4}, {2}} (actual Max Cut)

Local Max Cut

Find local optimum of Max Cut with the FLIP-neighbourhood (exactly one node can change sides) Sch¨ affer and Yannakakis [22] showed that Local Max Cut is PLS-complete (via an extremely involved reduction) Local Max Cut is to PLS what 3-SAT is to NP 1 2 3 4 1 1 −4 3 1 −2 Solutions: {{1, 3, 4}, {2}} (actual Max Cut) {{3}, {1, 2, 4}}

Pure Equilibrium in Polymatrix Game

1 2 3 2 −1 2

Pure Equilibrium in Polymatrix Game

1 2 3 a b a 0,0 2,2 b 2,2 0,0 a b a 0,0 2,2 b 2,2 0,0 a b a 0,0 -1,-1 b -1,-1 0,0

Pure Equilibrium in Polymatrix Game

1 2 3 a b a 0,0 2,2 b 2,2 0,0 a b a 0,0 2,2 b 2,2 0,0 a b a 0,0 -1,-1 b -1,-1 0,0 The bimatrix games (A, B) we used are examples of team games because A = B; also called coordination games

Proof that the reduction is correct

Define potential function for “team” polymatrix games Φ(S) = 1 2

• i

ui(S) This is an exact potential function: when i changes strategy then the potential function changes by exactly i’s change in utility Fact: in exact potential games, pure equilibria ↔ local optima of exact potential function Our exact potential function value equals value of the cut for all strategy profiles

Summary on PLS and polymatrix games

In contrast to bimatrix games, computing a pure equilibrium in polymatrix games is PLS-hard Next, an application of team polymatrix games

Application: Graph Transduction

semi-supervised learning: estimate a classification function defined over graph of labeled and unlabeled nodes

• ie. propagate labels to unlabelled nodes in consistent way

Application: Graph Transduction

semi-supervised learning: estimate a classification function defined over graph of labeled and unlabeled nodes

• ie. propagate labels to unlabelled nodes in consistent way

INPUT: Weighted graph, where some nodes are labelled; edge weights represent similarities

• ne approach is to use global optimization

an alternative approach is to use a polymatrix game

Application: Graph Transduction

semi-supervised learning: estimate a classification function defined over graph of labeled and unlabeled nodes

• ie. propagate labels to unlabelled nodes in consistent way

INPUT: Weighted graph, where some nodes are labelled; edge weights represent similarities

• ne approach is to use global optimization

an alternative approach is to use a polymatrix game Note: without the labelled examples, this is a clustering problem; also see e.g., “Hedonic Clustering Games” [12, 2]

Application: Graph Transduction

1 2 3 a b a 2,2 0,0 b 0,0 2,2 a b a 2,2 0,0 b 0,0 2,2 a b a -1,-1 0,0 b 0, 0 -1,-1

Application: Graph Transduction

1 2 3 a b a 2,2 0,0 b 0,0 2,2 a b a 2,2 0,0 b 0,0 2,2 a b a -1,-1 0,0 b 0, 0 -1,-1 Note: asymmetric similarity measures have also been

• considered. Then we may no longer have pure equilibria, but

mixed equilibria are still considered meaningful

Open question for team polymatrix games

Can we compute a mixed Nash equilibrium of a team polymatrix game in polynomial-time? [7] Note that this problem lies in PPAD ∩ PLS so is unlikely to be hard for either of them

Open question for team polymatrix games

Can we compute a mixed Nash equilibrium of a team polymatrix game in polynomial-time? [7] Note that this problem lies in PPAD ∩ PLS so is unlikely to be hard for either of them Question: Can anyone think of an easy mixed equilibrium for the local max cut game?

Open question for team polymatrix games

Can we compute a mixed Nash equilibrium of a team polymatrix game in polynomial-time? [7] Note that this problem lies in PPAD ∩ PLS so is unlikely to be hard for either of them Question: Can anyone think of an easy mixed equilibrium for the local max cut game? Suggested reading: Daskalakis & Papadimitriou Continuous local search SODA 2011

Outline

1 Nash equilibria of bimatrix games 2 Linear Complementarity Problems (LCPs) 3 The Lemke–Howson Algorithm and the class PPAD 4 Lemke’s algorithm 5 PLS-hardness of pure equilibria, Graph Transduction 6 Reduction from Polymatrix Game to LCP 7 Descent method for ǫ-Nash equilibria of polymatrix games 8 Other recent work on polymatrix games

Polymatrix games → LCPs

At least three different reductions to LCP; each gives an almost-complementarity algorithm

1 Howson 1972 [15] 2 Eaves 1973 [9] (more general) 3 Miller and Zucker 1991 [19]

Instead we are going to present bilinear games which appeared in Ruta Mehta’s thesis [18, 13], and which are a specialization of Eave’s games

Bilinear Games

Inspired by sequence form of Koller, Megiddo, von Stengel (1996) [17] They turn out to be are a special case of Eaves’ polymatrix games with joint constraints [9], where we restrict to: two players polytopal strategy constraint sets

Bilinear Games

A bilinear game is given by: two m × n dimensional payoff matrices A and B polytopal strategy constraint sets: X = {x ∈ Rm | Ex = e, x ≥ 0} Y = {y ∈ Rn | Fy = f, y ≥ 0} With payoffs xTAy and xTBy for the strategy profile (x, y) ∈ X × Y

Bilinear Games

A bilinear game is given by: two m × n dimensional payoff matrices A and B polytopal strategy constraint sets: X = {x ∈ Rm | Ex = e, x ≥ 0} Y = {y ∈ Rn | Fy = f, y ≥ 0} (x, y) ∈ X × Y is a Nash equilibrium iff xTAy ≥ ¯ xTA for all ¯ x ∈ X and xTBy ≥ xTB¯ y for all ¯ y ∈ Y

An LCP for Bilinear Games

Encode best response condition via an LP: max

x

x⊤(Ay) s.t. x⊤E⊤ = e⊤, x ≥ 0

An LCP for Bilinear Games

Encode best response condition via an LP: max

x

x⊤(Ay) s.t. x⊤E⊤ = e⊤, x ≥ 0 The dual LP has an unconstrained vector p: min

y

e⊤p s.t. E⊤p ≥ Ay We will again use complementary slackness:

An LCP for Bilinear Games

Encode best response condition via an LP: max

x

x⊤(Ay) s.t. x⊤E⊤ = e⊤, x ≥ 0 The dual LP has an unconstrained vector p: min

y

e⊤p s.t. E⊤p ≥ Ay We will again use complementary slackness: Feasible x, p are optimal iff x⊤(Ay) = e⊤p = x⊤E⊤p, i.e.,

An LCP for Bilinear Games

Encode best response condition via an LP: max

x

x⊤(Ay) s.t. x⊤E⊤ = e⊤, x ≥ 0 The dual LP has an unconstrained vector p: min

y

e⊤p s.t. E⊤p ≥ Ay We will again use complementary slackness: x⊤(−Ay + E⊤p) = 0

An LCP for Bilinear Games

Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that M =                     −A E⊤ −E⊤ −B⊤ F⊤ −F⊤ −E E −F F                     q =                     e −e f −f                     z = (x, y, p′, p′′, q′, q′′)⊤ where p = p′ − p′′, q = q′ − q′′

Lemke’s algorithm for Bilinear Games

Theorem 4.1 in [17] says: If we have

1 z⊤Mz ≥ 0 for all z ≥ 0, and 2 z ≥ 0, Mz ≥ 0 and z⊤Mz = 0 imply that z⊤q ≥ 0

then Lemke’s algorithm computes an solution to the LCP M, q

Polymatrix games as Bilinear Games

Polymatrix game (with complete interaction graph): players i = 1, . . . , n, with pure strategy sets Si and payoff matrices for player i, Aij ∈ R|Si|×|Sj| for pairs of players (i, j) let (x1, . . . , xn) in ∆(Si) × · · · × ∆(Sn) be a mixed strategy profile, then the payoff to player i is ui(x1, . . . , xn) =

• ij

(xi)⊤Aijxj

Polymatrix games as Bilinear Games

(Symmetric) bilinear game: (A, A⊤, E, E, e, e) payoff matrices (A, A⊤) strategy constraints Ex = e where e = ✶n, and A =               A12 · · · A1n A21 A2n . . . ... An1 An2 · · ·               E =                  ✶⊤

|S1|

· · · ✶⊤

|S2|

· · · . . . ... · · · ✶⊤

|Sn|

                

Reductions for sparse polymatrix games

Existing reductions apply to polymatrix games on complete interaction graphs For other interactions graphs, missing edges are replaced with games with all 0 payoffs Can we come up with more space efficient reductions for non-complete interaction graphs?

Outline

1 Nash equilibria of bimatrix games 2 Linear Complementarity Problems (LCPs) 3 The Lemke–Howson Algorithm and the class PPAD 4 Lemke’s algorithm 5 PLS-hardness of pure equilibria, Graph Transduction 6 Reduction from Polymatrix Game to LCP 7 Descent method for ǫ-Nash equilibria of polymatrix games 8 Other recent work on polymatrix games

Approximation - Background

Definition (ǫ-Nash equilibrium) A strategy profile is an ǫ-Nash equilibrium if: no player can gain more than ǫ by a unilateral deviation (additive notion of approximation)

Approximation - Background

Definition (ǫ-Nash equilibrium) A strategy profile is an ǫ-Nash equilibrium if: no player can gain more than ǫ by a unilateral deviation (additive notion of approximation) Theorem (Rubinstein 2014) There exists a constant ǫ such that it is PPAD-hard to find an ǫ-Nash equilibrium of a n-player polymatrix game.

Approximation - Background

Definition (ǫ-Nash equilibrium) A strategy profile is an ǫ-Nash equilibrium if: no player can gain more than ǫ by a unilateral deviation (additive notion of approximation) Theorem (CDT 2006) If there is an FPTAS for computing an ǫ-Nash of a bimatrix game, then PPAD = P.

Background: bimatrix games

What is the smallest ǫ such that an ǫ-Nash equilibrium can be computed in polynomial time (payoffs in [0, 1])?

Background: bimatrix games

What is the smallest ǫ such that an ǫ-Nash equilibrium can be computed in polynomial time (payoffs in [0, 1])? HISTORY: 0.5 Daskalakis Mehta Papadimitriou (WINE 06) 0.382 DMP (EC 2007) 0.364 Bosse Byrka Markakis (WINE 07) 0.339 Tsaknakis Spirakis (WINE 07) Tsaknakis & Spirakis use gradient descent

Background: many-player games

Two players: 0.3393 [Tsaknakis and Spirakis] n players: 1 − 1/n [obvious extension of DMP]

Background: many-player games

Two players: 0.3393 [Tsaknakis and Spirakis] n players: 1 − 1/n [obvious extension of DMP] DMP idea extends solution for n − 1 players to n players:

Three players: 0.6022 Four players: 0.7153 Guarantee goes to 1 as n goes to infinity

Background: many-player games

Two players: 0.3393 [Tsaknakis and Spirakis] n players: 1 − 1/n [obvious extension of DMP] DMP idea extends solution for n − 1 players to n players:

Three players: 0.6022 Four players: 0.7153 Guarantee goes to 1 as n goes to infinity

Next we show for the class of n-player polymatrix games: (0.5 + δ) in time polynomial in the input size and 1/δ

Gradient descent on max regret

Extend method of Tsaknakis and Spirakis Definition For a strategy profile x we define f(x) as the regret: f(x) := max

i∈players u∗ i (x) − ui(x)

Gradient descent on max regret

Extend method of Tsaknakis and Spirakis Definition For a strategy profile x we define f(x) as the regret: f(x) := max

i∈players u∗ i (x) − ui(x)

define δ-stationary point of f via combinatorial “gradient” LP to find a corresponding steepest descent direction

The algorithm

1 Choose an arbitrary strategy profile x ∈ ∆ 2 Solve steepest descent LP with input x to obtain x′ 3 Set x := x + α(x′ − x), where α = δ δ+2 4 If f(x) ≤ 0.5 + δ then stop, otherwise go to step 2

The result

Theorem A (0.5 + δ)-Nash equilibrium of a polymatrix game can be found in time polynomial in the size of the game and in 1/δ. Proof sketch: We do not get stuck at a bad point: Every δ-stationary point x∗ of f is a (0.5 + δ)-NE, i.e., f(x∗) ≤ 0.5 + δ Each descent step makes enough progress in reducing f, so that after polynomially many iterations f(x) ≤ 0.5 + δ

Open questions on approximate equilibria

Better upper bounds: Constant number of players or strategies Extend methods for bimatrix games that solve a single LP ǫ-well-supported approximate equilibria Lower bounds: It is PPAD-hard to find an ǫ-Nash equilibrium of a polymatrix game for a constant but very small ǫ [Rubinstein] Improve the value of ǫ in such a lower bound

Application: 2-player Bayesian games

Howson and Rosenthal (1974) observed that these games can be written as a complete bipartite polymatrix games Types of P1 Types of P2 1 2 3 1 2

Application: 2-player Bayesian games

Howson and Rosenthal (1974) observed that these games can be written as a complete bipartite polymatrix games Types of P1 Types of P2 1 2 3 1 2 The descent algorithm gives a 1/2-Nash but this is easily achievable by the DMP method Open question: do other methods for bimatrix games also extend to Bayesian two-player games?

Enumerating equilibria

All methods we discussed are to find one sample equilibrium Often a proper analysis of a game requires an enumeration of all equilibria Well-developed enumeration methods for bimatrix games [1] It is an interesting direction to develop similar methods for polymatrix games

Outline

1 Nash equilibria of bimatrix games 2 Linear Complementarity Problems (LCPs) 3 The Lemke–Howson Algorithm and the class PPAD 4 Lemke’s algorithm 5 PLS-hardness of pure equilibria, Graph Transduction 6 Reduction from Polymatrix Game to LCP 7 Descent method for ǫ-Nash equilibria of polymatrix games 8 Other recent work on polymatrix games

Other recent work on polymatrix games

Solving general multi-player games [14] (also see [11]) Zero-sum polymatrix games [4] Efficiency of equilibria in polymatrix coordination games [20] QPTAS for tree polymatrix games [3]

References I

[1] David Avis, Gabriel D. Rosenberg, Rahul Savani, and Bernhard von Stengel. Enumeration of Nash equilibria for two-player games. Economic Theory, 42(1):9–37, 2009. [2] Haris Aziz and Rahul Savani. Hedonic Games, chapter 15. Cambridge University Press, 2015. In press. [3] Siddharth Barman, Katrina Ligett, and Georgios Piliouras. Approximating nash equilibria in tree polymatrix games. In Algorithmic Game Theory - 8th International Symposium, (SAGT), pages 285–296, 2015.

References II

[4] Yang Cai, Ozan Candogan, Constantinos Daskalakis, and Christos Papadimitriou. Zero-sum polymatrix games: A generalization of minmax. Mathematics of Operations Research, To appear. [5] Xi Chen, Xiaotie Deng, and Shang-Hua Teng. Settling the complexity of computing two-player Nash equilibria. Journal of the ACM, 56(3):14:1–14:57, 2009. [6] Constantinos Daskalakis, Paul W. Goldberg, and Christos H. Papadimitriou. The complexity of computing a Nash equilibrium. SIAM Journal on Computing, 39(1):195–259, 2009.

References III

[7] Constantinos Daskalakis and Christos Papadimitriou. Continuous local search. In Proceedings of the twenty-second annual ACM-SIAM symposium on Discrete Algorithms, pages 790–804. SIAM, 2011. [8] Argyrios Deligkas, John Fearnley, Rahul Savani, and Paul Spirakis. Computing approximate Nash equilibria in polymatrix games. Algorithmica, 2015. Online first; Preliminary conference version appeared at WINE 2014. [9] B Curtis Eaves. Polymatrix games with joint constraints. SIAM Journal on Applied Mathematics, 24(3):418–423, 1973.

References IV

[10] Aykut Erdem and Marcello Pelillo. Graph transduction as a noncooperative game. Neural Computation, 24(3):700–723, 2012. [11] Uriel Feige and Inbal Talgam-Cohen. A direct reduction from k-player to 2-player approximate Nash equilibrium. In Algorithmic Game Theory - Third International Symposium (SAGT), pages 138–149, 2010. [12] Moran Feldman, Liane Lewin-Eytan, and Joseph Seffi Naor. Hedonic clustering games. In Proceedings of the 24th Annual ACM symposium on Parallelism in Algorithms and Architectures SPAA, pages 267–276. ACM, 2012.

References V

[13] Jugal Garg, Albert Xin Jiang, and Ruta Mehta. Bilinear games: Polynomial time algorithms for rank based subclasses. In Internet and Network Economics - 7th International Workshop, WINE, pages 399–407, 2011. [14] Srihari Govindan and Robert Wilson. Computing Nash equilibria by iterated polymatrix approximation. Journal of Economic Dynamics and Control, 28(7):1229–1241, April 2004. [15] Joseph T. Howson. Equilibria of polymatrix games. Management Science, 18(5):pp. 312–318, 1972.

References VI

[16] Jr. Howson, Joseph T. and Robert W. Rosenthal. Bayesian equilibria of finite two-person games with incomplete information. Management Science, 21(3):pp. 313–315, 1974. [17] Daphne Koller, Nimrod Megiddo, and Bernhard von Stengel. Efficient computation of equilibria for extensive two-person games. Games and Economic Behavior, 14(2):247–259, 1996. [18] Ruta Mehta. Nash Equilibrium Computation in Various Games. PhD thesis, Dept. of CSE, IIT-Bombay, 8 2012.

References VII

[19] Douglas A. Miller and Steven W. Zucker. Copositive-plus lemke algorithm solves polymatrix games. Operations Research Letters, 10(5):285 – 290, 1991. [20] Mona Rahn and Guido Sch¨ afer. Efficient equilibria in polymatrix coordination games. CoRR, abs/1504.07518, 2015. [21] Aviad Rubinstein. Inapproximability of Nash equilibrium. In Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, STOC, pages 409–418, 2015. [22] Alejandro A Sch¨ affer and Mihalis Yannakakis. Simple local search problems that are hard to solve. SIAM journal on Computing, 20(1):56–87, 1991.