Approximate Nash Equilibrium Computation Paul W. Goldberg 1 1 - - PowerPoint PPT Presentation

Approximate Nash Equilibrium Computation

Paul W. Goldberg1

1Department of Computer Science

University of Oxford, U. K.

Invited talk, WINE conference 12th Dec. 2015

Goldberg Approximate Nash Equilibrium Computation

The computational challenge Input: payoff matrices R, C of bimatrix game G Output: a Nash equilibrium of G

• Centralised. No “strategic data”. I don’t care about social welfare.

Goldberg Approximate Nash Equilibrium Computation

The computational challenge Input: payoff matrices R, C of bimatrix game G Output: a Nash equilibrium of G

• Centralised. No “strategic data”. I don’t care about social welfare.

Two key (annoying?) features PPAD-complete pseudo-polytime (but not polytime) algorithm known for approximate version

Goldberg Approximate Nash Equilibrium Computation

The computational challenge Input: payoff matrices R, C of bimatrix game G Output: a Nash equilibrium of G

• Centralised. No “strategic data”. I don’t care about social welfare.

Two key (annoying?) features PPAD-complete pseudo-polytime (but not polytime) algorithm known for approximate version Unusual answers for both exact and approx versions... Coincidence??

Goldberg Approximate Nash Equilibrium Computation

The computational challenge Input: payoff matrices R, C of bimatrix game G Output: a Nash equilibrium of G

• Centralised. No “strategic data”. I don’t care about social welfare.

Two key (annoying?) features PPAD-complete pseudo-polytime (but not polytime) algorithm known for approximate version Unusual answers for both exact and approx versions... Coincidence??

similar results for other classes of games.

Goldberg Approximate Nash Equilibrium Computation

PPAD-completeness

Goldberg Approximate Nash Equilibrium Computation

PPAD-completeness

PPA... what? — Papadimitriou

Goldberg Approximate Nash Equilibrium Computation

PPAD-completeness

PPA... what? — Papadimitriou Pathways to Equilibria: Pretty Pictures And Diagrams (PPAD) — von Stengel

Goldberg Approximate Nash Equilibrium Computation

PPAD-completeness

PPA... what? — Papadimitriou Pathways to Equilibria: Pretty Pictures And Diagrams (PPAD) — von Stengel

My attempt at a talk title

Goldberg Approximate Nash Equilibrium Computation

PPAD-completeness

PPA... what? — Papadimitriou Pathways to Equilibria: Pretty Pictures And Diagrams (PPAD) — von Stengel

My attempt at a talk title

Prospects for Progress in Approximate

Goldberg Approximate Nash Equilibrium Computation

PPAD-completeness

PPA... what? — Papadimitriou Pathways to Equilibria: Pretty Pictures And Diagrams (PPAD) — von Stengel

My attempt at a talk title

Prospects for Progress in Approximate Duo.../Double... (can’t think of a good word)

Goldberg Approximate Nash Equilibrium Computation

PPAD-completeness

PPA... what? — Papadimitriou Pathways to Equilibria: Pretty Pictures And Diagrams (PPAD) — von Stengel

My attempt at a talk title

Prospects for Progress in Approximate Duo.../Double... (can’t think of a good word) In fact, something like “Polynomial Parity Argument on a graph, Directed version”

Papadimitriou, C.H.: On the Complexity of the Parity Argument and Other Inefficient Proofs of Existence, J.

• Comput. Syst. Sci. (1994)

Goldberg Approximate Nash Equilibrium Computation

What’s wrong with NP-completeness?

NP-complete problems have yes-instances and no-instances...

Goldberg Approximate Nash Equilibrium Computation

What’s wrong with NP-completeness?

NP-complete problems have yes-instances and no-instances... In searching for a Nash equilibrium, every instance (game) is a yes-instance! Every game has one (Nash 1951), and a suggested NE is easy to check.

Goldberg Approximate Nash Equilibrium Computation

What’s wrong with NP-completeness?

NP-complete problems have yes-instances and no-instances... In searching for a Nash equilibrium, every instance (game) is a yes-instance! Every game has one (Nash 1951), and a suggested NE is easy to check. reduce from (say) SAT to NASH: what should happen to the no-instances?

It’s conceivable some other “NASH is as hard as NP” proof could exist...

Goldberg Approximate Nash Equilibrium Computation

TFNP: total function computation in NP

NASH∈TFNP: “TF”: every game has an outcome “NP”: a “transparent”, easily-checkable outcome Best-known “hard” TFNP problem: FACTORING — given a number, output its prime factorisation; hardness needed for much crypto

Goldberg Approximate Nash Equilibrium Computation

TFNP: total function computation in NP

NASH∈TFNP: “TF”: every game has an outcome “NP”: a “transparent”, easily-checkable outcome Best-known “hard” TFNP problem: FACTORING — given a number, output its prime factorisation; hardness needed for much crypto

Hard TFNP problems: an unhappy family

Happy families are all alike; every unhappy family is unhappy in its own way. — Leo Tolstoy For our purposes: NP-complete problems are all alike; every hard TFNP problem is hard in its own way. — don’t quote me Work in progress on this...

Goldberg Approximate Nash Equilibrium Computation

PPAD: a happy subfamily of TFNP

END OF THE LINE Circuits Succ and Pred; n inputs, n outputs; graph on 2n vertices with arc from u to v iff Succ(u)=v, Pred(v)=u Given 0 has successor but no predecessor, find another vertex of degree 1.

Goldberg Approximate Nash Equilibrium Computation

PPAD: a happy subfamily of TFNP

PPA: same sort of thing, but undirected graph As it happens, FACTORING belongs to PPA, related to PPAD...

Emil Jeˇ r´ abek: Integer factoring and modular square roots

• J. Comput. Syst. Sci., to appear

suggestive —but only suggestive— that PPAD is hard Digression: oracle model of PPAD assumes query access to functions Succ and Pred:2n → 2n. Query complexity of search for a solution is poly in the circuit model but not in the oracle model. There are oracle separation results for PPAD and other subclasses

• f TFNP

Goldberg Approximate Nash Equilibrium Computation

From NASH to ǫ-NASH: Bounded rationality fixes irrationality

With 3 players, NE may have irrational values (Nash ’51); ....even for 3-player, 2-strategy anonymous games

G and Turchetta: Query Complexity of Approximate Equilibria in Anonymous Games, these proceedings

and in general, for any k > 2 players, n strategies, algebraic degree

• f values may be exponential in n... also for graphical games

ǫ-Nash equilibrium No incentive —————– ≤ ǫ incentive to deviate — solution can have values that are multiples of ǫ/kn ∈ Q.

To be meaningful, assume payoffs in some bounded range, usually [0, 1].

Negative (hardness) results carry over to exact NE (useful for first PPAD-hardness results)

Goldberg Approximate Nash Equilibrium Computation

ǫ-NASH versus ǫ-Well-Supported NASH

ǫ-NASH: average payoff is worse than best-response by at most ǫ — but player may do much worse, with low probability ǫ-WSNE (stronger!): anything a player does with positive probability, pays at most ǫ less than best-response. The support of a probability distribution is the set of events that get non-zero probability — for a mixed strategy, all the pure strategies that may get chosen. i.e. anything in the support of a player’s mixed strategy, is within ǫ

• f best

Goldberg Approximate Nash Equilibrium Computation

A good start: ǫ = 1

2 in poly time

Daskalakis, Mehta, and Papadimitriou: A note on approximate Nash equilibria. WINE’07; TCS 2009

1 2

0.2 0.1 0.9 0.2 0.2 0.3 0.2 0.4 0.1 0.5 0.2 0.6 0.2 0.7 0.2 0.8 0.8

1 Player 1 chooses arbitrary strategy i; gives it probability 1

2.

Goldberg Approximate Nash Equilibrium Computation

A good start: ǫ = 1

2 in poly time

Daskalakis, Mehta, and Papadimitriou: A note on approximate Nash equilibria. WINE’07; TCS 2009

1 2

1

0.2 0.1 0.9 0.2 0.2 0.3 0.2 0.4 0.1 0.5 0.2 0.6 0.2 0.7 0.2 0.8 0.8

1 Player 1 chooses arbitrary strategy i; gives it probability 1

2.

2 Player 2 chooses best response j; gives it probability 1. Goldberg Approximate Nash Equilibrium Computation

A good start: ǫ = 1

2 in poly time

Daskalakis, Mehta, and Papadimitriou: A note on approximate Nash equilibria. WINE’07; TCS 2009

1 2 1 2

1

0.2 0.1 0.9 0.2 0.2 0.3 0.2 0.4 0.1 0.5 0.2 0.6 0.2 0.7 0.2 0.8 0.8

1 Player 1 chooses arbitrary strategy i; gives it probability 1

2.

2 Player 2 chooses best response j; gives it probability 1. 3 Player 1 finds best response k to j; gives it probability 1

2.

They also find 5

6-WSNE in poly-time... Goldberg Approximate Nash Equilibrium Computation

Computing ǫ-NE: the key facts

fact card

1 For ǫ > 0, support size of ǫ-NE is O(log n) (Alth¨

• fer; Lipton

et al); for ǫ < 1

2 support is Ω(nlog n) (Feder et al)

2 For any fixed ǫ > 0, complexity is O(nlog n); gives us hope for

poly-time approx’n scheme PTAS (LMM ’03)

3 PTAS for ǫ-NE can be turned into a PTAS for

ǫ-well-supported-NE (DGP’09; CDT’09) (kick out strategies from ǫ-NE that pay less than best-response− ǫ

2). But, ǫ-WSNE

requires ǫ2/8-NE. (E.g., 3

4-WSNE needs approx 0.07-NE)

4 but, best ǫ for NE is just over 1

3, for WSNE, just under 2 3

Alth¨

• fer: On sparse approximations to randomized

strategies and convex combinations. Linear Algebra and its Applications 1994 Lipton, Markakis, and Mehta: Playing Large Games using Simple Strategies. EC, ’03 Feder, Nazerzadeh and Saberi: Approximating Nash Equilibria using Small-Support Strategies. EC (2007)

Goldberg Approximate Nash Equilibrium Computation

ǫ-Nash equilibrium in quasi-poly time

Given: n × n game... Let N be a Nash equilibrium. (mixed: in general a probability distribution) Draw N samples from N; let ˆ N be uniform distribution over these samples Empirical payoffs converge to payoffs arising from N... How big does N need to be for uniform convergence to within additive ǫ? O(log n/ǫ2)! So, ˆ N is an ǫ-NE with support size O(log n) ˆ N can be found by support enumeration in time O(nlog n); also works for k (constant) players

Goldberg Approximate Nash Equilibrium Computation

ǫ-Nash equilibrium in quasi-poly time

Given: n × n game... Let N be a Nash equilibrium. (mixed: in general a probability distribution) Draw N samples from N; let ˆ N be uniform distribution over these samples Empirical payoffs converge to payoffs arising from N... How big does N need to be for uniform convergence to within additive ǫ? O(log n/ǫ2)! So, ˆ N is an ǫ-NE with support size O(log n) ˆ N can be found by support enumeration in time O(nlog n); also works for k (constant) players

Is there a PTAS? ← the big question

Poly-time for constant ǫ is unsatisfying; PTAS would be redemptive

Goldberg Approximate Nash Equilibrium Computation

progress on additive ǫ-NE

1/2 + δ 3/4 0.5 0.38197+ζ 0.36392 0.3393 2/3 0.6619 0.6608 5/6 ǫ = 1 ǫ = 0 2006 07 08 09 10 11 12 13 14 Now

• KPP
• DMP
• DMP •

BBM

• TS
• DMP
• KS
• FGSS

0.6619

• FGSS

0.6608

• CFJ

1/2 + δ (symmetric only)

• CDFFJS

0.6528

• GP

0.732

• GP

0.732 0.437

Well-supported in blue comm-bounded in red

Goldberg Approximate Nash Equilibrium Computation

constant support size not enough for ǫ < 1

2:

consider random zero-sum win-lose games of size n × n:

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Feder, Nazerzadeh and Saberi: Approximating Nash Equilibria using Small-Support Strategies. EC (2007)

Goldberg Approximate Nash Equilibrium Computation

constant support size not enough for ǫ < 1

2:

consider random zero-sum win-lose games of size n × n: 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 With high probability, for

any pure strategy by player 1, player 2 can “win”

Feder, Nazerzadeh and Saberi: Approximating Nash Equilibria using Small-Support Strategies. EC (2007)

Goldberg Approximate Nash Equilibrium Computation

constant support size not enough for ǫ < 1

2:

consider random zero-sum win-lose games of size n × n: 0.4 0.6 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 With high probability, for

any pure strategy by player 1, player 2 can “win”

2 Indeed, as n increases,

this is true if player 1 may mix 2 of his strategies

Feder, Nazerzadeh and Saberi: Approximating Nash Equilibria using Small-Support Strategies. EC (2007)

Goldberg Approximate Nash Equilibrium Computation

constant support size not enough for ǫ < 1

2:

consider random zero-sum win-lose games of size n × n:

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1 With high probability, for

any pure strategy by player 1, player 2 can “win”

2 Indeed, as n increases,

this is true if player 1 may mix 2 of his strategies

3 or indeed, any constant

number of strategies

Feder, Nazerzadeh and Saberi: Approximating Nash Equilibria using Small-Support Strategies. EC (2007)

Goldberg Approximate Nash Equilibrium Computation

constant support size not enough for ǫ < 1

2:

1/n 1/n 1/n 1/n 1/n 1/n

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

But, for large n, player 1 can guarantee a payoff of about 1/2 by randomizing

• ver his strategies (w.h.p.,

as n increases)

Goldberg Approximate Nash Equilibrium Computation

How big a support do you need?

If less than log(n) strategies are used by player 1, there is a high probability that player 2 can win... Hence Ω(log(n)) is a lower bound on support size needed. Matches O(log(n)) upper bound.

Goldberg Approximate Nash Equilibrium Computation

ǫ-Well-Supported NE

The KS algorithm (for 2

3-WSNE of game (R, C)):

1 look for pure profiles that pay each player ≥ 1

3

2 If we find one, use it 3 else solve (R − C, C − R); use resulting profile

Kontogiannis and Spirakis: Well supported approximate equilibria in bimatrix games. Algorithmica (2010)

Goldberg Approximate Nash Equilibrium Computation

Relieving a bottleneck in KS

Fearnley, G, Savani, and Sørensen. Approximate Well-supported Nash Equilibria Below Two-thirds. Algorithmica, to appear

Worst case for KS: (R,C) (R-C,C-R)

1 3

1 1

1 3

− 2

3 2 3 2 3

− 2

3

1

1 2 1 2

Goldberg Approximate Nash Equilibrium Computation

Relieving a bottleneck in KS

Fix: (R,C) (R-C,C-R)

1 3

1 1

1 3

− 2

3 2 3 2 3

− 2

3

1

1 2 + δ 1 2 − δ

In n × n game, search for this configuration and fix as above

Goldberg Approximate Nash Equilibrium Computation

Need to shift probability in equal and opposite directions?

(R,C) (R-C,C-R)

1 3

1 1

1 3 1 3

1 1

1 3

− 2

3 2 3 2 3

− 2

3 2 3

− 2

3

− 2

3 2 3

Goldberg Approximate Nash Equilibrium Computation

Fix for this case

(R,C) (R-C,C-R)

1 3

1 1

1 3 1 3

1 1

1 3

− 2

3 2 3 2 3

− 2

3 2 3

− 2

3

− 2

3 2 3

1 2 1 2 1 2 1 2

Goldberg Approximate Nash Equilibrium Computation

ǫ-Well-Supported NE

So, we can approximate ǫ-WSNE for ǫ slightly less than 2

3....

Feels like we should be able to do better.... Next: should we give up on search for PTAS, and prove that none exists?

Goldberg Approximate Nash Equilibrium Computation

Problems that require nlog n time?

Example: compute VC-dimension of n × n incidence matrix

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Papadimitriou and Yannakakis: On Limited Nondeterminism and the Complexity of the V-C

• Dimension. J. Comput. Syst. Sci. (1996)

Goldberg Approximate Nash Equilibrium Computation

Problems that require nlog n time?

Example: compute VC-dimension of n × n incidence matrix

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

VC-dim ≥d: ∃d rows that “induce” all 2d distinct columns.

Goldberg Approximate Nash Equilibrium Computation

Problems that require nlog n time?

Example: compute VC-dimension of n × n incidence matrix

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

VC-dim(above)=3. Easy brute-force O(nlog n) algorithm.

Goldberg Approximate Nash Equilibrium Computation

Reduce to ǫ-NASH?

Syntactic guarantee: VC-dim ≤ ⌊log n⌋. Obstacle:

Goldberg Approximate Nash Equilibrium Computation

Reduce to ǫ-NASH?

Syntactic guarantee: VC-dim ≤ ⌊log n⌋. Obstacle: If VC-dim of a matrix is d < ⌊log n⌋ − 1, no obvious certificate it’s NOT d + 1, say.

Goldberg Approximate Nash Equilibrium Computation

Reduce to ǫ-NASH?

Syntactic guarantee: VC-dim ≤ ⌊log n⌋. Obstacle: If VC-dim of a matrix is d < ⌊log n⌋ − 1, no obvious certificate it’s NOT d + 1, say. Need a “hard-looking” TFNP problem in O(nlog n)

Goldberg Approximate Nash Equilibrium Computation

Reduce to ǫ-NASH?

Syntactic guarantee: VC-dim ≤ ⌊log n⌋. Obstacle: If VC-dim of a matrix is d < ⌊log n⌋ − 1, no obvious certificate it’s NOT d + 1, say. Need a “hard-looking” TFNP problem in O(nlog n) Reasonable question: Can we reduce, for sufficiently small ǫ, from ǫ/2-NASH to (say) ǫ-NASH? (trying to “compare like with like”)

Goldberg Approximate Nash Equilibrium Computation

new starting-point for “LOGNP-hardness” of ǫ-NASH

Babichenko, Papadimitriou and Rubinstein: Can Almost Everybody be Almost Happy? PCP for PPAD and the Inapproximability of Nash, ArXiv (2015)

“exponential time hypothesis” for PPAD: END OF THE LINE requires time 2˜

Ω(n)

Goldberg Approximate Nash Equilibrium Computation

new starting-point for “LOGNP-hardness” of ǫ-NASH

Babichenko, Papadimitriou and Rubinstein: Can Almost Everybody be Almost Happy? PCP for PPAD and the Inapproximability of Nash, ArXiv (2015)

“exponential time hypothesis” for PPAD: END OF THE LINE requires time 2˜

Ω(n)

PPAD-completeness of NASH goes via intermediate problem GCIRCUIT: compute fixpoint of arithmetic circuit. Approximate version ǫ-GCIRCUIT also recently shown to be PPAD-complete

Rubinstein: Inapproximability of Nash equilibrium, STOC (2015)

New conjecture (BPR’15): For some δ, ǫ > 0, there’s a quasilinear reduction from END OF THE LINE to (ǫ, δ)-GCIRCUIT. With ETH for PPAD, (ǫ, δ)-GCIRCUIT requires time 2˜

Ω(n)

Goldberg Approximate Nash Equilibrium Computation

It follows from the conjecture, there exist ǫ′, δ′ such that it takes exponential time to ǫ′-satisfy a fraction 1 − δ′ of GCIRCUIT elements. (With the close relationship between GCIRCUIT and graphical games, you can’t keep almost everybody almost happy...) From that, there’s an ǫ > 0 such that ǫ-NASH really requires time n˜

Ω(log n).

Goldberg Approximate Nash Equilibrium Computation

bipartite 2-strategy graphical games

NE for this can “capture” fixpoint of GCIRCUIT...

Goldberg Approximate Nash Equilibrium Computation

Reducing graphical games to 2-player games

1/poly(n)-NE of this can “capture” ǫ-NE of graphical game

Goldberg Approximate Nash Equilibrium Computation

A slightly more robust but less efficient version

1/poly(ǫ)-NE of this can capture ǫ-NE of graphical game that “fails” for δ of players

Goldberg Approximate Nash Equilibrium Computation

The self-critical bit: are we asking the right question?

Why ǫ-NE? :-( lack of scale-invariance is a downside. :-( Any result just for some constant ǫ is unsatisfying, even ǫ = 0.01 “rival” notions Trembling-hand perfect: if a strategy is suboptimal, give it prob at most ǫ proper ǫ-NE: if s is worse than s′, Pr[s] ≤ ǫ. Pr[s′]

Goldberg Approximate Nash Equilibrium Computation

Conclusions

There’s a rich theory developing aiming to explain limits to what we seem to manage to achieve in ǫ-NASH results scope for progress in reducing ǫ...

very little known about games of > 2 players. fun stuff being done in query complexity of approx NE

Goldberg Approximate Nash Equilibrium Computation

Conclusions

There’s a rich theory developing aiming to explain limits to what we seem to manage to achieve in ǫ-NASH results scope for progress in reducing ǫ...

very little known about games of > 2 players. fun stuff being done in query complexity of approx NE

Thanks!

Goldberg Approximate Nash Equilibrium Computation