Algorithmic Game Theory CoReLab (NTUA) Lecture 3: Tractability of - - PowerPoint PPT Presentation

Algorithmic Game Theory CoReLab (NTUA)

Lecture 3: Tractability of Nash Equilibria PPAD completeness Lemke-Howson algorithm LMM

So far

Nash’s Theorem (1950) Every (finite) game has a Nash Equilibrium. Brouwer’sTheorem (1911) Every continuous function from a closed compact convex set to itself has a fixed point. Sperner’s Lemma (1950) Every proper coloring of a triangulation has a panchromatic triangle. Parity Argument (1990) If a directed graph has an unbalanced node, then it must have another. NE in 2-player zero sum ↔ LP Duality

What we know

Sperner Brouwer Nash

FNP

General 2-player games

A slightly more ambitious attempt would be to face general 2- player games and provide efficient algorithms or prove hardness results. An other direction would be to face 3-player zero sum games…. but in fact these games can only be harder.(!)

Nash vs NP

The problem resisted polynomial algorithms for a long time which altered the research direction towards hardness results. The first idea would be to prove Nash an FNP-complete problem. But accepting an FSAT Nash reduction directly implies NP=coNP. (!)

Nash vs TFNP

What prevented our previous attempt was the fact that Nash problem always has solution. So the next idea would be to prove it complete for this class. But no complete problem is known for TFNP.

Complexity Theory of Total Search Problems

In order to overcome the obstacles we face we need to work as follows:

• 1. Identify the combinatorial structure that makes our problems total.
• 2. Define a new complexity class inspired from our observation.
• 3. Check the ‘tightness’ of our class – in other words that our problems are

complete for the class.

Sperner’s Lemma revisited

No matter how the internal nodes are colored there exists a tri-chromatic triangle. In fact there will be an

• dd number of them.

no blue no yellow no red

Why Sperner is hard?

We have to work with a graph of exponential size!

Input: 2 n-bit x numbers y yes/no

Circuit

2𝑜 2𝑜

Proof of Sperner’s Lemma

• 1. We introduce an

artificial vertex on the bottom left

• 2. We define a

directed walk crossing red-yellow doors having red on

• ur left

▪ Claim: The walk can’t get out nor can it loop into itself ▪ It follows that there is an odd number of tri-chromatic triangles

!

Parity Argument

Graph Representation

▪ Every vertex has in and out degree at most 1 ▪ Each vertex with degree 1 is an acceptable solution (except the artificial one) ▪ By the parity argument there is always an even number of solutions ▪ Notice that if we insist in finding the pair of our green node the problem is beyond FNP! ...

The PPAD Class [Papadimitriou ’94]

END OF THE LINE

F C

node id node id node id node id Given F and C : If 0n is an unbalanced node, find another unbalanced node. Otherwise say “yes”.

father child

PPAD = { Search problems in FNP reducible to END OF THE LINE}

𝐺 𝑤2 = 𝑤1 ˄ 𝐷 𝑤1 = 𝑤2

What we know

FNP NP=coNP

TFNP Semantic

PPAD Sperner Brouwer Nash

2-Nash PPAD-complete

[Daskalakis,Goldberg,Papadimitriou 2006]

...

0n

Generic PPAD

[Pap ’94] [DGP ’05]

Embed PPAD graph in [0,1]3

[DGP ’05]

3D-SPERNER 

:=



• +

xa >

Arithmetic Circuit Sat Polymatrix game [DGP ’05] [DGP ’05] [DGP ’05]

Arithmetic Circuit Sat

▪ Input A circuit with :

• Variable nodes 𝑦1, 𝑦2, … , 𝑦𝑜
• Gate nodes 𝑕1, 𝑕2, … , 𝑕𝑛 ∈ {

, , , , , }

• Directed edges connecting variable with gates and vice versa (loops are allowed)
• Output An assignment of values 𝑦1, 𝑦2, … , 𝑦𝑜 ∈ [0,1] satisfying the gate

constraints: Assignment : 𝑧 == 𝑦1 Set to const : 𝑧 == max{0, min 𝑏, 1 } Addition : 𝑧 == min 1, 𝑦1 + 𝑦2 Multiply const: 𝑧 == max{0, min 𝑏𝑦, 1 } Subtraction : 𝑧 == max 0, 𝑦1 − 𝑦2

:= +

• a

×a >

Arithmetic Circuit Sat

Comparison gate: 𝑧 == 1, 𝑗𝑔 𝑦1 > 𝑦2 0, 𝑗𝑔 𝑦1 < 𝑦2 ∗, 𝑗𝑔 𝑦1 = 𝑦2 ▪ Example :

1/3

𝑦1 > := 𝑦2 𝑦3

Unique solution: 𝑦1 = 𝑦2 = 𝑦3 = 1 3

Arithmetic Circuit Sat

• We can get an approximate version of Arithmetic Circuit Sat, by relaxing the gate

constraints by 𝜗 ≥ 0: Assignment : 𝑧 == 𝑦1 ± 𝜗 Set to const : 𝑧 == max{0, min 𝑏, 1 } ± 𝜗 Addition : 𝑧 == min 1, 𝑦1 + 𝑦2 ± 𝜗 Multiply const: 𝑧 == max{0, min 𝑏𝑦, 1 } ± 𝜗 Subtraction : 𝑧 == max 0, 𝑦1 − 𝑦2 ± 𝜗 Comparison gate: 𝑧 == 1, 𝑗𝑔 𝑦1 > 𝑦2 − 𝜗 0, 𝑗𝑔 𝑦1 < 𝑦2 + 𝜗 ∗, 𝑗𝑔 𝑦1 = 𝑦2 ± 𝜗 Both versions of the problem are PPAD-complete!

Graphical Games

• Players are nodes in a directed graph.
• The player’s payoff 𝑣𝑗 depends on her strategy as well as

the strategies of the players pointing to her.

Polymatrix Games

• Special case of Graphical Games.
• Payoff is edge-wise separable:

𝑣𝑤 𝑦1, 𝑦2, … , 𝑦𝑜 =

(𝑥,𝑤)∈𝐹

𝑣𝑥,𝑤(𝑦𝑥, 𝑦𝑤)

Arithmetic Circuit Sat Polymatrix Game

• In order to reduce Arithmetic Circuit Sat to Polymatrix

Games, we will present polymatrix gadgets which simulate the arithmetic functions of the circuit.

• Every player chooses her strategy from 0,1 .
• For every player 𝑞, representing a variable node 𝑦𝑞,

Pr[𝑞: 1] represents the value of 𝑦𝑞.

• Finally, every Nash Equilibrium can be translated to a

feasible solution of Arithmetic Circuit Sat.

Arithmetic Circuit Sat Polymatrix Game

Addition Gadget

𝑦 𝑨 𝑥 𝑧 Variable nodes

Gate node

𝑣 𝑥: 0 = Pr 𝑦: 1 + Pr[𝑧: 1] 𝑣 𝑥: 1 = Pr[z: 1] 𝑣 𝑨: 0 = 0.5 𝑣 𝑨: 1 = 1 − Pr[𝑥: 1] In any Nash equilibrium of a game containing this Gadget Pr z: 1 = min{1, Pr x: 1 + Pr y: 1 }

Arithmetic Circuit Sat Polymatrix Game

Addition Gadget

𝑣 𝑥: 0 = Pr 𝑦: 1 + Pr[𝑧: 1] 𝑣 𝑥: 1 = Pr[z: 1] 𝑣 𝑨: 0 = 0.5 𝑣 𝑨: 1 = 1 − Pr[𝑥: 1]

• Pr 𝑨: 1 < min 1, Pr 𝑦: 1 + Pr 𝑧: 1

Pr 𝑥: 0 = 1 Pr 𝑨: 1 = 1

• Pr 𝑨: 1 > Pr 𝑦: 1 + Pr[𝑧: 1] Pr 𝑥: 1 = 1 Pr 𝑨: 0 = 1

Pr[𝑨: 1] = min{1, Pr[𝑦: 1] + Pr[𝑧: 1]}

Arithmetic Circuit Sat Polymatrix Game

Comparison Gadget

𝑦 z 𝑧 Variable nodes 𝑣 𝑨: 0 = Pr 𝑧: 1 𝑣 𝑨: 1 = Pr[x: 1]

Pr 𝑦: 1 > Pr[𝑧: 1] Pr 𝑨: 1 = 1 Pr 𝑦: 1 < Pr[𝑧: 1] Pr[𝑨: 1] = 0 Pr[𝑦:1] = Pr 𝑧: 1 anything is possible

From Polymatrix Game to 2-player Game

𝑦 𝑨 𝑥 𝑧 Variable nodes

Gate node

Every gadget can be turned into a bipartite graph with variable node-players sharing the same side and gate node-players on the

• ther.

We define a 2-player game where the yellow lawyer represents all the yellow players and similarly the red lawyer represents all the red players.

The Lawyer Game

In order to analyze the Lawyer Game we will first define and analyze two games, that combined will give us the appropriate game. Our goal : If (𝑦, 𝑧) is a Nash Equilibrium for the Lawyer Game, then the marginal distributions that 𝑦 assigns to the strategies of the yellow nodes and the marginal distributions that 𝑧 assigns to the red nodes comprise a Nash Equilibrium in the Polymatrix Game.

Breaking down the Lawyer Game

▪ The Representation Game: The set of strategies for the yellow lawyer is the union of the strategies of every yellow node. The same goes for the red lawyer. The payoff for the lawyers is the payoff that their clients would had gotten had they played the same strategies themselves. ▪ The High Stakes Chase The sets of strategies remain the same. Image an arbitrary labelling {1, . . , 𝑜} for the yellow clients and a respective labelling 1, … , 𝑜 for the red clients. Whenever both lawyers get to pick the same label, the red lawyer pays M to the yellow. Otherwise they both get 0.

The Lawyer Game

The High Stakes Chase

M,-M 𝟏, 𝟏 𝟏, 𝟏 𝟏, 𝟏 𝟏, 𝟏 M,-M 𝟏, 𝟏 𝟏, 𝟏 𝟏, 𝟏 𝟏, 𝟏 M,-M 𝟏, 𝟏 𝟏, 𝟏 𝟏, 𝟏 𝟏, 𝟏 M,-M

Given this observation we could claim Proposition 1: Taking M arbitrarily big would essentially lead the lawyers to play with probability (approximately) 1/𝑜 each of their clients in the Combined Game! It is easy to see that the High Stake Chase is a zero-sum game where in every NE the lawyers play uniformly

• ver their clients.

(We no longer have to worry about our marginal distributions being ill-defined)

Strategies of red node i Strategies

• f yellow

node j

The Lawyer Game

The Representation Game On the other hand if both lawyers play uniformly over their clients, the way that the probability is split among each client’s strategies will not affect the High Stakes Game. The split will be solely determined by the Representation Game and this directly implies that our marginal distributions are indeed a NE for the Polymatrix Game. Notice that we are being a little bit inaccurate as Proposition 1 holds up to an error 𝜗, but the sketch remains the same and the error can be accommodated by the Approximate Arithmetic Circuit Sat!

PPAD completeness of 2-Nash

TFNP

Arguments of existence and respective complexity classes

PPA[Papadimitriou ’94] ‘If a node has odd degree then there must be an other.’

N

node id { node id1 , node id2}

n-bit input 2n-bit

• utput

𝑤1 ∈ 𝑂 𝑣2 & 𝑣2 ∈ 𝑂(𝑣1)

Given N: If 0n has odd degree, find another node with odd degree. Otherwise say “yes”.

ODD DEGREE NODE

PPA = { Search problems in FNP reducible to ODD DEGREE NODE}

PPA

Graph Representation

{0,1}n ... 0n

Exponentially large graph Every node has degree at most 2

PLS [JPY ‘89]

‘Every DAG must have a sink.’

• Local Max Cut is a well known PLS-complete problem.
• Spoiler! PNE in Congestion Games is also PLS-complete.

PLS [JPY ‘89]

N

node id {node id1, …, node idk}

V

node id

‘Every DAG must have a sink.’

n-bit input kn-bit

• utput

𝑤2 = 𝑂 𝑤1 & 𝑊 𝑤2 > 𝑊(𝑤1)

FIND SINK Given N, V: Find x s.t. 𝑊(𝑦) ≥ 𝑊(𝑧), for all y  N(x). PLS = { Search problems in FNP reducible to FIND SINK}

PLS

Graph Representation

{0,1}n

Exponentially large directed acyclic graph

PPP

“If a function maps n elements to n-1 elements, then there is a collision.”

F

node id node id COLLISION Given F: Find x s.t. F( x )= 0n; or find x ≠ y s.t. F(x)=F(y). PPP = { Search problems in FNP reducible to COLLISION }

Inclusions

2 −player Symmetric Games

A bimatix game represented by two matrices 𝐵, 𝐶 is called Symmetric if 𝐶 = 𝐵𝑈 (i.e., the two players have the same set of strategies and their utilities remain the same if their roles are reversed). A strategy profile 𝑦 is a Symmetric Nash Equilibrium if both players playing 𝑦 results in a Nash Equilibrium. Looking at Symmetric Games is no loss of generality!

Reduction from Nash to Symmetric Nash

Fix any bimatrix game represented by the matrices 𝐵, 𝐶 (w.l.o.g. with positive entries). Now consider the Symmetric Game defined by the matrices below:

𝟏 𝑩 𝑪𝑼 𝟏

𝑑1 =

𝟏 𝑪 𝑩𝐔 𝟏

𝑑2 =

𝑦 𝑦 𝑧 𝑧 Let 𝑦, 𝑧 be a Symmetric NE. In order 𝑦, 𝑧 to be a best response to itself, 𝑦 must be a best response to 𝑧 and vice versa.

Lemke-Howson

▪ Fix any Symmetric Game with an, 𝑜 × 𝑜, utility matrix 𝐵. ▪ W.l.o.g. assume non negative entries and no zero column or raw. ▪ Consider the (non degenerate) polytope 𝑄 defined by the following inequalities: 𝑨 ≥ 𝟏 𝐵𝑨 ≤ 𝟐 (!)

Lemke-Howson

1. A strategy 𝑗 is represented at vertex 𝑨 if 𝑨𝑗 = 0 or 𝐵𝑗𝑨 = 1 or both. 2. Define set 𝑊 with all the vertices of 𝑄 that represent every strategy except possibly strategy 𝑜 3. Any vertex (other than 𝟏) at which all strategies are represented is a NE. 4. In order to find such a vertex we shall develop a (simplex-like) pivoting method beginning at vertex 𝟏 and ending at a SNE.

This vertex represents every strategy It follows that here we get a SNE

Lemke-Howson

𝑨𝑗 = 0 𝐵𝑗𝑨 = 1 𝑨1 = 0 𝑨2 = 0 𝑨3 = 0 …. 𝑨𝑜 = 0

𝑤0

𝑨4 = 0 Choose next inequality to relax

𝑤1

𝑨1 = 0 𝑨2 = 0 𝑨3 = 0 𝐵3𝑨 = 1 𝑨4 = 0 …. Choose next inequality to relax

𝑤𝑙

𝑨2 = 0 𝐵3𝑨 = 1 𝑨4 = 0 …. 𝐵1𝑨 = 1 𝐵𝑜𝑨 = 1 Symmetric Nash Equilibrium!

Lemke-Howson

▪ Claim the walk can not loop neither can it reach the 𝟏 vertex.(!) ▪ There are exponentially many but finite vertices in 𝑄. It follows that the algorithm halts returning a SNE. Final remark: Although it may seem like there are no direction in the edges we define, in fact this algorithm relates to PPAD.

Recap

▪ NE in 2-player zero sum ↔ LP Duality ▪ NE in general 2-player games PPAD complete

( Lemke-Howson exponential running time algorithm )

In order to sidestep the probable intractability of NE we are going to relax

• ur equilibrium concept!

Approximate Nash Equilibrium

For any 𝜁 > 0 a pair of mixed strategies 𝑦, 𝑧 is called an 𝜁 -Nash equilibrium if: i. For every mixed strategy 𝑦′ of the row player, 𝑦′, 𝐵𝑧 ≤ 𝑦, 𝐵𝑧 + 𝜁 ii. For every mixed strategy 𝑧′ of the column player, 𝑦 , 𝐶𝑧′ ≤ 𝑦, 𝐶𝑧 + 𝜁

Lipton Markakis Mehta ’03

Main result For any NE 𝑦∗, 𝑧∗ and for any 𝜁 > 0, there exists, for every 𝑙 ≥

12 ln 𝑜 𝜁2 a

pair of 𝑙 −uniform strategies 𝑦′, 𝑧,′ such that:

• 1. 𝑦′, 𝑧′ is an 𝜁-NE
• 2. | 𝑦′, 𝐵𝑧′ − 𝑦∗, 𝐵𝑧∗ | < 𝜁 (row player gets almost the same payoff as

in the NE)

• 3. | 𝑦′, 𝐶𝑧′ − 𝑦∗, 𝐶𝑧∗ | < 𝜁 (column player gets almost the same payoff

as in the NE) (Assuming all entries of 𝐵, 𝐶 between 0,1)

Lipton Markakis Mehta ’03

Proof Sketch via Probabilistic Method ▪ Given 𝑦∗, 𝑧∗, 𝜁 > 0 fix 𝑙 ≥

12 ln 𝑜 𝜁2

▪ Form multiset 𝑌 sampling 𝑙 times independently, from the pure strategies of the row player according to the distribution 𝑦∗. Respectively, form 𝑍 from the pure strategies of the column player. ▪ Let 𝑦′ be the 𝑙 −uniform strategy related with multiset 𝑌 and 𝑧′ the 𝑙 −uniform strategy related with multiset 𝑍.

Lipton Markakis Mehta ’03

Proof Sketch via Probabilistic Method ▪ Finally consider the following events:

• 𝜒1 = { 𝑦′, 𝐵𝑧′ − 𝑦∗, 𝐵𝑧∗

<

𝜁 2}

• 𝜒2 = { 𝑦′, 𝐶𝑧′ − 𝑦∗, 𝐶𝑧∗

<

𝜁 2}

• 𝜌1,𝑗 = { 𝑦𝑗, 𝐵𝑧′ − 𝑦′, 𝐵𝑧′ < 𝜁} (𝑗 = 1,2 … , 𝑜)
• 𝜌2,𝑘 = { 𝑦′, 𝐶𝑧𝑘 − 𝑦′, 𝐶𝑧′ < 𝜁} (𝑘 = 1,2 … , 𝑜)

𝐻𝑃𝑃𝐸 = 𝜒1 ∩ 𝜒2 ∩𝑗=1

𝑜

𝜌1,𝑗 ∩𝑘=1

𝑜

𝜌2,𝑘 Goal: Pr 𝐻𝑃𝑃𝐸𝑑 < 1

Lipton Markakis Mehta ’03

Proof Sketch via Probabilistic Method In order to bound the probability of 𝜒1

𝑑 we define the following :

𝜒1𝑏 = { 𝑦′, 𝐵𝑧∗ − 𝑦∗, 𝐵𝑧∗ } <

𝜁 4

𝜒1𝑐 = { 𝑦′, 𝐵𝑧′ − 𝑦′, 𝐵𝑧∗ } <

𝜁 4

The expression (𝑦′, 𝐵𝑧∗) is a sum of 𝑙 independent random variables each of expected value (𝑦∗, 𝐵𝑧∗). Each such random variable takes value between 0 and 1. As a result we can apply Chernoff bounds: Pr[𝜒1𝑏

𝑑 ] ≤ 2𝑓−𝑙𝜁2

8

and similarly Pr[𝜒1𝑐

𝑑 ] ≤ 2𝑓−𝑙𝜁2

8

𝜒1𝑏 ∩ 𝜒1𝑐 ⊆ 𝜒1 Pr 𝜒1

𝑑 ≤ Pr 𝜒1𝑏 𝑑 ∪ 𝜒1𝑐 𝑑

≤ 4𝑓−𝑙𝜁2

8

Lipton Markakis Mehta ’03

Proof Sketch via Probabilistic Method Using the same toolbox we get the following bounds: Pr 𝜒1

𝑑 ≤ 4𝑓−𝑙𝜁2 8

Pr 𝜒2

𝑑 ≤ 4𝑓−𝑙𝜁2 8

Pr 𝜌1,𝑗

𝑑

≤ 4𝑓−𝑙𝜁2

8 +2𝑓−𝑙𝜁2 2

Pr 𝜌2,𝑘

𝑑

≤ 4𝑓−𝑙𝜁2

8 +2𝑓−𝑙𝜁2 2

Pr 𝐻𝑃𝑃𝐸𝑑 ≤ Pr 𝜒1

𝑑 + Pr 𝜒2 𝑑 + 𝑗=1

Pr 𝜌1,𝑗

𝑑

+

𝑘=1

Pr 𝜌1,𝑘

𝑑

≤ 8𝑓−𝑙𝜁2

8 + 2𝑜

𝑓−𝑙𝜁2

2 + 4𝑓−𝑙𝜁2 8

< 1

n n

Lipton Markakis Mehta ’03

Subexponential running time & 𝑛 −player games The main result implies the existence of subexponential algorithm (𝑜𝑃 𝑚𝑝𝑕𝑜 ) for computing all 𝑙 −uniform 𝜁 − equilibria for any 2 −player game(!) The main result can accommodate 𝑛 −player games although the dependence of 𝑙 to 𝑛 is polynomial.

Barman’s sparsification technique ‘14

Applying the approximate version of Caratheodory’s theorem Barman improved the previous results proving the following statement: (Assuming all entries of 𝐵, 𝐶 between 0,1) In any bimatrix game with 𝑜 × 𝑜 matrices 𝐵, 𝐶, if the number of non-zero entries in any column of 𝐵 + 𝐶 is at most 𝑡 then an 𝜁 −NE can be computed in time 𝑜𝑃 𝑚𝑝𝑕𝑡/𝜁2

Anonymous Games

In Anonymous Games a large population of players shares the same strategy set and, while players may have different payoff functions, the payoff of each depends on her

• wn choice of strategy and the number of the other players playing each strategy

(not the identity of these players). Canonical example: 500 citizens have to decide either to go to the cinema or to the theatre and they

• nly care about how crowded it will be.

Polynomial-Time Approximation Scheme

(PTAS) A PTAS is an algorithm which takes an instance of an optimization problem and a parameter 𝜁 > 0 and, in polynomial time, produces a solution that is within a factor 1 + 𝜁 of being optimal. Notice that an algorithm running in time 𝑃 𝑜𝜁−1 or even 𝑃(𝑜exp(𝜁−1)) counts as a PTAS.

PTAS for Anonymous Games

(Daskalakis, Papadimitriou ’14) There is a PTAS for the mixed Nash equilibrium problem for normalized anonymous games with a constant number of strategies. More precisely, there exists some function 𝑕 such that, for all 𝜁 > 0, an 𝜁 -Nash equilibrium of a normalized anonymous game of m players and n strategies can be computed in time 𝑛𝑕(𝑜,𝜁−1).

Wrapping up

• Computing exact NE in 𝟑 −player zero sum games belongs in P
• Computing exact NE in general 𝟑 −player games is PPAD complete
• Computing 𝜻 −NE in general 𝟑 −player games accepts subexponential time

algorithms

• Computing 𝜻 −NE in Anonymous Games accepts PTAS algorithms