Logit Dynamics with Concurrent Updates for Local Interaction Games - - PowerPoint PPT Presentation
Logit Dynamics with Concurrent Updates for Local Interaction Games Francesco Pasquale Sapienza Universit` a di Roma joint work with Vincenzo Auletta, Diodato Ferraioli, Paolo Penna, and Giuseppe Persiano ESA: European Symposium on
Francesco Pasquale
“Sapienza” Universit` a di Roma
joint work with Vincenzo Auletta, Diodato Ferraioli, Paolo Penna, and Giuseppe Persiano ESA: European Symposium on Algorithms Sophia Antipolis, September 2013
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Motivating example 2/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
The standard game-theoretic way
◮ Numbers are at most 100, so the average will be at most 100,
and half of the average will be at most 50
◮ I will not write a number larger than 50
Motivating example 3/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
The standard game-theoretic way
◮ Numbers are at most 100, so the average will be at most 100,
and half of the average will be at most 50
◮ I will not write a number larger than 50 ◮ If none writes a number larger than 50, then the average will
be at most 50, and half of the average will be at most 25
◮ I will not write a number larger than 25
Motivating example 3/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
The standard game-theoretic way
◮ Numbers are at most 100, so the average will be at most 100,
and half of the average will be at most 50
◮ I will not write a number larger than 50 ◮ If none writes a number larger than 50, then the average will
be at most 50, and half of the average will be at most 25
◮ I will not write a number larger than 25 ◮ If none writes a number larger than 25,. . . ◮ . . . ◮ Prediction: Everyone writes 1!
Motivating example 3/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
The standard game-theoretic way
◮ Numbers are at most 100, so the average will be at most 100,
and half of the average will be at most 50
◮ I will not write a number larger than 50 ◮ If none writes a number larger than 50, then the average will
be at most 50, and half of the average will be at most 25
◮ I will not write a number larger than 25 ◮ If none writes a number larger than 25,. . . ◮ . . . ◮ Prediction: Everyone writes 1!
Do you believe that prediction?
Motivating example 3/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
A previous experiment
STOC poster session at FCRC’11 Half of the average
Motivating example 4/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
A previous experiment
STOC poster session at FCRC’11 Half of the average
Standard game theoretic assumption Rationality common knowledge This is too strong assumption in several cases
◮ Limited knowledge ◮ Limited computational power ◮ Limited rationality
Motivating example 4/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Nash equilibria = Steady states of best-response dynamics
Idea
Relaxation of best-response dynamics
Logit dynamics and stationary distribution 5/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Nash equilibria = Steady states of best-response dynamics
Idea
Relaxation of best-response dynamics Best-response
Logit dynamics and stationary distribution 5/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Nash equilibria = Steady states of best-response dynamics
Idea
Relaxation of best-response dynamics Best-response
Logit dynamics and stationary distribution 5/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Nash equilibria = Steady states of best-response dynamics
Idea
Relaxation of best-response dynamics Best-response Randomized best-response
Logit dynamics and stationary distribution 5/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Nash equilibria = Steady states of best-response dynamics
Idea
Relaxation of best-response dynamics Best-response Randomized best-response
Logit dynamics and stationary distribution 5/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Nash equilibria = Steady states of best-response dynamics
Idea
Relaxation of best-response dynamics Best-response Randomized best-response
Logit Choice Function [McFadden, 1974]
From profile x = (x1, . . . , xn) player i chooses strategy y with probability proportional to eβui(x−i,y).
Logit dynamics and stationary distribution 5/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Logit choice function
Logit dynamics and stationary distribution 6/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Logit choice function
β = “Rationality level”
◮ β = 0 players play uniformly at random ◮ β → ∞ players best-respond
Logit dynamics and stationary distribution 6/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Logit choice function
β = “Rationality level”
◮ β = 0 players play uniformly at random ◮ β → ∞ players best-respond
Logit dynamics [Blume, GEB’93]
◮ Revision process: choose one player u.a.r. ◮ Update rule: logit choice function
Logit dynamics and stationary distribution 6/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
◮ Economics
[Blume, GEB’93]: Equilibrium selection when β → ∞ [Al´
Characterization of stochastically stable states
◮ Computer Science
[Montanari and Saberi, FOCS’09]: Hitting time of the best Nash equilibrium [Asadpour, Saberi, WINE’09]: Hitting time of the neighborhood of best Nash equilibria for Atomic Selfish Routing and Load Balancing.
◮ Statistical Mechanics
Logit dynamics vs Glauber dynamics
Logit dynamics and stationary distribution 7/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
it always exists? Is it unique?
Logit dynamics and stationary distribution 8/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
always exists? Is it unique? Stationary distribution of logit dynamics always exists and it is unique. [Auletta et al, SAGT’10]
Logit dynamics and stationary distribution 8/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
always exists? Is it unique? Stationary distribution of logit dynamics always exists and it is unique. [Auletta et al, SAGT’10]
does it take to reach equilibrium?
Logit dynamics and stationary distribution 8/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
always exists? Is it unique? Stationary distribution of logit dynamics always exists and it is unique. [Auletta et al, SAGT’10]
it take to reach equilibrium? Analysis of mixing time of logit dynamics for some classes of games [Auletta et al, SPAA’11]
Logit dynamics and stationary distribution 8/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
always exists? Is it unique? Stationary distribution of logit dynamics always exists and it is unique. [Auletta et al, SAGT’10]
it take to reach equilibrium? Analysis of mixing time of logit dynamics for some classes of games [Auletta et al, SPAA’11]
something about what happens in the meanwhile?
Logit dynamics and stationary distribution 8/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
always exists? Is it unique? Stationary distribution of logit dynamics always exists and it is unique. [Auletta et al, SAGT’10]
it take to reach equilibrium? Analysis of mixing time of logit dynamics for some classes of games [Auletta et al, SPAA’11]
something about what happens in the meanwhile? Metastability of logit dynamics [Auletta et al, SODA’12]
Logit dynamics and stationary distribution 8/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Logit choice function (pi(y | x) ∼ eβui(x−i,y)) Revision process (pick one single player at random)
All-logit reversibility 9/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Logit choice function (pi(y | x) ∼ eβui(x−i,y)) Revision process (pick one single player at random)
What happens when all players play simultaneously?
◮ All-logit ergodic (unique stationary distribution and
convergence)
All-logit reversibility 9/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Logit choice function (pi(y | x) ∼ eβui(x−i,y)) Revision process (pick one single player at random)
What happens when all players play simultaneously?
◮ All-logit ergodic (unique stationary distribution and
convergence)
◮ How do stationary distribution for all-logit differ from
stationary distribution for one-logit?
All-logit reversibility 9/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Logit choice function (pi(y | x) ∼ eβui(x−i,y)) Revision process (pick one single player at random)
What happens when all players play simultaneously?
◮ All-logit ergodic (unique stationary distribution and
convergence)
◮ How do stationary distribution for all-logit differ from
stationary distribution for one-logit?
◮ Are there any meaningful invariant quantities (that are the
same for the one-logit and the all-logit)?
All-logit reversibility 9/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Reversibility
What is the stationary distribution for the all-logit dynamics?
All-logit reversibility 10/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Reversibility
What is the stationary distribution for the all-logit dynamics?
Stationary distribution is easy for reversible chains. Reversibility: π(x)P(x, y) = π(y)P(y, x)
All-logit reversibility 10/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Reversibility
What is the stationary distribution for the all-logit dynamics?
Stationary distribution is easy for reversible chains. Reversibility: π(x)P(x, y) = π(y)P(y, x)
Kolmogorov criterion for reversibility
P is reversible if and only if for every cycle (x0, x1, . . . , xk, x0) P(x0, x1)P(x1, x2) · · · P(xk, x0) = P(x0, xk)P(xk, xk−1) · · · P(x1, x0)
x0 x1 x2 xk
All-logit reversibility 10/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Potential games and local interaction games
One-logit reversibility Theorem (Blume, GEB’93)
One-logit for game G is reversible if and only if G is a potential game.
All-logit reversibility 11/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Potential games and local interaction games
One-logit reversibility Theorem (Blume, GEB’93)
One-logit for game G is reversible if and only if G is a potential game.
All-logit reversibility
All-logit reversibility 11/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Potential games and local interaction games
One-logit reversibility Theorem (Blume, GEB’93)
One-logit for game G is reversible if and only if G is a potential game.
All-logit reversibility Theorem
All-logit for game G is reversible if and only is G is a local interaction game.
All-logit reversibility 11/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Potential Games
G = ([n], S, U). Φ : S1 × · · · × Sn → R exact potential if for every profile x, for every player i, and for every action y ui(x−i, y) − ui(x) = − [Φ(x−i, y) − Φ(x)]
Local interaction games 12/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Potential Games
G = ([n], S, U). Φ : S1 × · · · × Sn → R exact potential if for every profile x, for every player i, and for every action y ui(x−i, y) − ui(x) = − [Φ(x−i, y) − Φ(x)]
Local interaction games
◮ Players are nodes of a
graph
Local interaction games 12/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Potential Games
G = ([n], S, U). Φ : S1 × · · · × Sn → R exact potential if for every profile x, for every player i, and for every action y ui(x−i, y) − ui(x) = − [Φ(x−i, y) − Φ(x)]
Local interaction games
◮ Players are nodes of a
graph
◮ Edges are two-player
potential games
Local interaction games 12/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Potential Games
G = ([n], S, U). Φ : S1 × · · · × Sn → R exact potential if for every profile x, for every player i, and for every action y ui(x−i, y) − ui(x) = − [Φ(x−i, y) − Φ(x)]
Local interaction games
◮ Players are nodes of a
graph
◮ Edges are two-player
potential games
Observation
A local interaction game is a potential game.
Local interaction games 12/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Idea of proof
Theorem
Logit dynamics for game G is reversible if and only if G is a local interaction game. Idea of proof.
Local interaction games 13/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Idea of proof
Theorem
Logit dynamics for game G is reversible if and only if G is a local interaction game. Idea of proof.
[It follows from Monderer and Shapley characterization of potential games]
Local interaction games 13/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Idea of proof
Theorem
Logit dynamics for game G is reversible if and only if G is a local interaction game. Idea of proof.
[It follows from Monderer and Shapley characterization of potential games]
for every pair of profiles x, y K(x, y) = K(y, x) (1) where K(x, y) = n
i=1 Φ(x−i, yi) − (n − 2)Φ(x)
[It follows from the Kolmogorov criterion for reversibility applied to the all-logit for a potential game]
Local interaction games 13/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Stationary
interaction game [A potential function satisfies K(x, y) = K(y, x) if and only if it is a sum of 2-player potential functions]
Local interaction games 14/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Stationary
interaction game [A potential function satisfies K(x, y) = K(y, x) if and only if it is a sum of 2-player potential functions]
Stationary distribution
G local interaction game πall(x) ∼
e−βK(x,y) K(x, y) = n
i=1 Φ(x−i, yi) − (n − 2)Φ(x)
Local interaction games 14/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Stationary
interaction game [A potential function satisfies K(x, y) = K(y, x) if and only if it is a sum of 2-player potential functions]
Stationary distribution
G local interaction game πall(x) ∼
e−βK(x,y) K(x, y) = n
i=1 Φ(x−i, yi) − (n − 2)Φ(x)
For the one-logit it is πone(x) ∼ e−βΦ(x)
Local interaction games 14/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Example
F : {strategy profiles} → R
Question
◮ Local interaction game G ◮ πone, πall stationary distributions of one-logit and all-logit
Is there any meaningful observable F such that Eπone [F] = Eπall [F].
Observables 15/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Example
F : {strategy profiles} → R
Question
◮ Local interaction game G ◮ πone, πall stationary distributions of one-logit and all-logit
Is there any meaningful observable F such that Eπone [F] = Eπall [F].
Example (Ising model)
±1
◮ Φ(x) = − {i,j}∈E xixj
(Energy)
◮ F(x) = n i=1 xi
(Magnetization)
Observables 15/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Example
F : {strategy profiles} → R
Question
◮ Local interaction game G ◮ πone, πall stationary distributions of one-logit and all-logit
Is there any meaningful observable F such that Eπone [F] = Eπall [F].
Example (Ising model)
±1
◮ Φ(x) = − {i,j}∈E xixj
(Energy)
◮ F(x) = n i=1 xi
(Magnetization) Eπone [F] = Eπall [F]
Observables 15/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Decompositions
Local interaction game G, potential Φ, set of strategy profiles S
Decomposition
A permutation σ = (σ1, σ2) of S × S such that for every pair of profiles
◮ σ1(x, y) = σ2(y, x) ◮ K(x, y) = Φ(σ1(x, y)) + Φ(σ2(x, y))
Observables 16/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Decompositions
Local interaction game G, potential Φ, set of strategy profiles S
Decomposition
A permutation σ = (σ1, σ2) of S × S such that for every pair of profiles
◮ σ1(x, y) = σ2(y, x) ◮ K(x, y) = Φ(σ1(x, y)) + Φ(σ2(x, y))
Lemma
G local interaction game on a bipartite graph then G admits a decomposition.
Observables 16/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Decompositions
Decomposable observables
Observable F decomposable if decomposition σ exists such that for all x, y F(x) + F(y) = F(σ1(x, y)) + F(σ2(x, y))
Observables 17/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Decompositions
Decomposable observables
Observable F decomposable if decomposition σ exists such that for all x, y F(x) + F(y) = F(σ1(x, y)) + F(σ2(x, y))
Theorem
If F is a decomposable observable then Eπone [F] = Eπall [F]
Observables 17/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
rationality
Research directions 18/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
rationality
Research directions 18/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
rationality
Research directions 18/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
rationality
Open problems
◮ Game theoretic interpretation of K(x, y)
Research directions 18/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
rationality
Open problems
◮ Game theoretic interpretation of K(x, y) ◮ Mixing time and metastabilty of all-logit for local
interaction games
Research directions 18/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
rationality
Open problems
◮ Game theoretic interpretation of K(x, y) ◮ Mixing time and metastabilty of all-logit for local interaction
games
◮ Other invariant observables
Research directions 18/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
rationality
Open problems
◮ Game theoretic interpretation of K(x, y) ◮ Mixing time and metastabilty of all-logit for local interaction
games
◮ Other invariant observables ◮ Other revision processes: Players selected according to
some distribution
Research directions 18/ 19
Logit Dynamics with Concurrent Updates for Local Interaction Games ESA - Sophia Antipolis, September 2013
Research directions 19/ 19