Open problems in repeated games with finite automata Abraham Neyman - - PowerPoint PPT Presentation

Open problems in repeated games with finite automata

Abraham Neyman

Jerusalem, May 23, 2011

subject, date – p. 1/13

Two-person zero-sum games

subject, date – p. 2/13

Two-person zero-sum games

Quantify the advantage of larger automata in repeated games.

subject, date – p. 2/13

Two-person zero-sum games

Quantify the advantage of larger automata in repeated games.

GT (k1, k2) is the repeated game where

subject, date – p. 2/13

Two-person zero-sum games

Quantify the advantage of larger automata in repeated games.

GT (k1, k2) is the repeated game where

a stage game G is repeated T ≤ ∞ times, player i is confined to play a strategy that is implementable by a deterministic automaton of size

ki ≤ ∞,

the repeated game payoff is the average per-stage payoff.

subject, date – p. 2/13

Two-person zero-sum games

Quantify the advantage of larger automata in repeated games.

GT (k1, k2) is the repeated game where

a stage game G is repeated T ≤ ∞ times, player i is confined to play a strategy that is implementable by a deterministic automaton of size

ki ≤ ∞,

the repeated game payoff is the average per-stage payoff. Explicitly, what is the asymptotic behavior of VAL GT (k1, k2) as T, k1, k2 → ∞ .

subject, date – p. 2/13

Two-person zero-sum games

Quantify the advantage of larger automata in repeated games.

GT (k1, k2) is the repeated game where

a stage game G is repeated T ≤ ∞ times, player i is confined to play a strategy that is implementable by a deterministic automaton of size

ki ≤ ∞,

the repeated game payoff is the average per-stage payoff. Explicitly, what is the asymptotic behavior of VAL GT (k1, k2) as T, k1, k2 → ∞ .

subject, date – p. 2/13

Two-person zero-sum games

Quantify the advantage of larger automata in repeated games.

GT (k1, k2) is the repeated game where

a stage game G is repeated T ≤ ∞ times, player i is confined to play a strategy that is implementable by a deterministic automaton of size

ki ≤ ∞,

the repeated game payoff is the average per-stage payoff. Explicitly, what is the asymptotic behavior of VAL GT (k1, k2) as T, k1, k2 → ∞ .

subject, date – p. 2/13

Two-person zero-sum games

Quantify the advantage of larger automata in repeated games.

GT (k1, k2) is the repeated game where

a stage game G is repeated T ≤ ∞ times, player i is confined to play a strategy that is implementable by a deterministic automaton of size

ki ≤ ∞,

the repeated game payoff is the average per-stage payoff. Explicitly, what is the asymptotic behavior of VAL GT (k1, k2) as T, k1, k2 → ∞ .

subject, date – p. 2/13

Non-zero-sum games

subject, date – p. 3/13

Non-zero-sum games

Quantify the equilibrium payoff of non-zero-sum repeated games with large automata

subject, date – p. 3/13

Non-zero-sum games

Quantify the equilibrium payoff of non-zero-sum repeated games with large automata Explicitly, what is the asymptotic behavior of the equilibrium payoffs of GT (k1, k2, . . . , kn) as T, ki → ∞

subject, date – p. 3/13

Non-zero-sum games

Quantify the equilibrium payoff of non-zero-sum repeated games with large automata Explicitly, what is the asymptotic behavior of the equilibrium payoffs of GT (k1, k2, . . . , kn) as T, ki → ∞

subject, date – p. 3/13

Non-zero-sum games

Quantify the equilibrium payoff of non-zero-sum repeated games with large automata Explicitly, what is the asymptotic behavior of the equilibrium payoffs of GT (k1, k2, . . . , kn) as T, ki → ∞

subject, date – p. 3/13

Non-zero-sum games

Quantify the equilibrium payoff of non-zero-sum repeated games with large automata Explicitly, what is the asymptotic behavior of the equilibrium payoffs of GT (k1, k2, . . . , kn) as T, ki → ∞

subject, date – p. 3/13

Non-zero-sum games

Quantify the equilibrium payoff of non-zero-sum repeated games with large automata Explicitly, what is the asymptotic behavior of the equilibrium payoffs of GT (k1, k2, . . . , kn) as T, ki → ∞

subject, date – p. 3/13

Non-zero-sum games

Quantify the equilibrium payoff of non-zero-sum repeated games with large automata Explicitly, what is the asymptotic behavior of the equilibrium payoffs of GT (k1, k2, . . . , kn) as T, ki → ∞

subject, date – p. 3/13

Finite automata strategies

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i M = S, s1, α, τ – an automata of player i

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i M = S, s1, α, τ – an automata of player i S – states of the automaton, s0 ∈ S – the initial state

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i M = S, s1, α, τ – an automata of player i S – states of the automaton, s0 ∈ S – the initial state α : S → Ai – action function of the automaton M

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i M = S, s1, α, τ – an automata of player i S – states of the automaton, s0 ∈ S – the initial state α : S → Ai – action function of the automaton M τ : S × A → S – the transition function of M

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i M = S, s1, α, τ – an automata of player i S – states of the automaton, s0 ∈ S – the initial state α : S → Ai – action function of the automaton M τ : S × A → S – the transition function of M σ = σ(M) – the strategy defined by the automaton

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i M = S, s1, α, τ – an automata of player i S – states of the automaton, s0 ∈ S – the initial state α : S → Ai – action function of the automaton M τ : S × A → S – the transition function of M σ = σ(M) – the strategy defined by the automaton σ1 = f(s1), σ(a1, a2, . . . , ak) = f(sk) where

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i M = S, s1, α, τ – an automata of player i S – states of the automaton, s0 ∈ S – the initial state α : S → Ai – action function of the automaton M τ : S × A → S – the transition function of M σ = σ(M) – the strategy defined by the automaton σ1 = f(s1), σ(a1, a2, . . . , ak) = f(sk) where sk = τ(sk−1, ak−1) the state of the automaton before

play at stage k

subject, date – p. 4/13

Finite automata strategies

G = N, A = ×i∈NAi, g = (gi)i∈N – the stage game N – set of players Ai – stage actions of player i gi : A → R – payoff function of player i M = S, s1, α, τ – an automata of player i S – states of the automaton, s0 ∈ S – the initial state α : S → Ai – action function of the automaton M τ : S × A → S – the transition function of M σ = σ(M) – the strategy defined by the automaton σ1 = f(s1), σ(a1, a2, . . . , ak) = f(sk) where sk = τ(sk−1, ak−1) the state of the automaton before

play at stage k

subject, date – p. 4/13

Open Problem - I

subject, date – p. 5/13

Open Problem - I

I.1 Do the values of G(k, nk) := G∞(k, nk) converge as

nk ≥ k → ∞ and limk→∞

log nk k

= x?

subject, date – p. 5/13

Open Problem - I

I.1 Do the values of G(k, nk) := G∞(k, nk) converge as

nk ≥ k → ∞ and limk→∞

log nk k

= x?

I.2 For those values of x for which the limit exists, what is the limit as a function of the stage game G and x?

subject, date – p. 5/13

Open Problem - I

I.1 Do the values of G(k, nk) := G∞(k, nk) converge as

nk ≥ k → ∞ and limk→∞

log nk k

= x?

I.2 For those values of x for which the limit exists, what is the limit as a function of the stage game G and x? Exact asymptotic – x = 0: Ben-Porath 93,

x = log min(|A1|, |A2|): N-Spencer 10, x > log min(|A1|, |A2|): N 97

subject, date – p. 5/13

Open Problem - I

I.1 Do the values of G(k, nk) := G∞(k, nk) converge as

nk ≥ k → ∞ and limk→∞

log nk k

= x?

I.2 For those values of x for which the limit exists, what is the limit as a function of the stage game G and x? Exact asymptotic – x = 0: Ben-Porath 93,

x = log min(|A1|, |A2|): N-Spencer 10, x > log min(|A1|, |A2|): N 97

A lower bound – 0 < x < log min(|A1|, |A2|): N 08

subject, date – p. 5/13

Example

The stage game

L R T 2 B 2

subject, date – p. 6/13

Information Theoretic Tools

Example: Ω = {0, 1}, p(0) = p.

subject, date – p. 7/13

Information Theoretic Tools

Example: Ω = {0, 1}, p(0) = p. H(X) = −p log2 p − (1 − p) log2(1 − p).

subject, date – p. 7/13

Information Theoretic Tools

Example: Ω = {0, 1}, p(0) = p. H(X) = −p log2 p − (1 − p) log2(1 − p).

0.25 0.5 0.75 1.0 p 0.2 0.4 0.6 0.8 1.0

HX

subject, date – p. 7/13

A lower bound

subject, date – p. 8/13

A lower bound

subject, date – p. 8/13

A lower bound

0.2 0.4 0.6 0.8 1 1.2 1.4 x: log n

• k

0.2 0.4 0.6 0.8 1 Vx 2H11x

subject, date – p. 8/13

A lower bound

0.2 0.4 0.6 0.8 1 1.2 1.4 x: log n

• k

0.2 0.4 0.6 0.8 1 Vx 2H11x

x = 0: Ben-Porath x = 1: 93 N-Spencer 10 x > 1: N 97

subject, date – p. 8/13

A lower bound

0.2 0.4 0.6 0.8 1 1.2 1.4 x: log n

• k

0.2 0.4 0.6 0.8 1 Vx 2H11x

x = 0: Ben-Porath x = 1: 93 N-Spencer 10 x > 1: N 97 0 < x < 1 : N 08

subject, date – p. 8/13

Open Problem - II

subject, date – p. 9/13

Open Problem - II

Do the values of GTk(k, ∞) converge as k → ∞ and

limk→∞

Tk k log = x?

subject, date – p. 9/13

Open Problem - II

Do the values of GTk(k, ∞) converge as k → ∞ and

limk→∞

Tk k log = x?

For those values of x for which the limit exists, what is the limit as a function of the stage game G and x?

subject, date – p. 9/13

Open Problem - II

Do the values of GTk(k, ∞) converge as k → ∞ and

limk→∞

Tk k log = x?

For those values of x for which the limit exists, what is the limit as a function of the stage game G and x? Exact asymptotic (N-Okada 99),

x = ∞ = ⇒

VAL GTk(k, ∞) → v∗

subject, date – p. 9/13

Open Problem - II

Do the values of GTk(k, ∞) converge as k → ∞ and

limk→∞

Tk k log = x?

For those values of x for which the limit exists, what is the limit as a function of the stage game G and x? Exact asymptotic (N-Okada 99),

x = ∞ = ⇒

VAL GTk(k, ∞) → v∗ A conjecture (N 97)

x = 0 = ⇒

VAL GTk(k, ∞) → v

subject, date – p. 9/13

Open Problem - non-zero-sum

subject, date – p. 10/13

Open Problem - non-zero-sum

What is the limit behavior of the equilibrium payoffs of

GT(k1, k2, . . . , kn) as T, ki → ∞

subject, date – p. 10/13

Open Problem - non-zero-sum

What is the limit behavior of the equilibrium payoffs of

GT(k1, k2, . . . , kn) as T, ki → ∞

Known for two-player games

subject, date – p. 10/13

Open Problem - non-zero-sum

What is the limit behavior of the equilibrium payoffs of

GT(k1, k2, . . . , kn) as T, ki → ∞

Known for two-player games For T = ∞, the essential issue is the individual rational level.

subject, date – p. 10/13

Open Problem - non-zero-sum

What is the limit behavior of the equilibrium payoffs of

GT(k1, k2, . . . , kn) as T, ki → ∞

Known for two-player games For T = ∞, the essential issue is the individual rational level. For T < ∞ additional issues.

subject, date – p. 10/13

Example 2

The stage game

L R T −1, 1, 1 0, 0, 0 B 0, 0, 0 0, 0, 0 L R 0, 0, 0 0, 0, 0 0, 0, 0 −1, 1, 1

subject, date – p. 11/13

Example 2

The stage game

L R T 1, 1, 1 2, 0, 2 B 0, 2, 2 1, 1, 3 L R 2, 2, 0 3, 1, 1 1, 3, 1 2, 2, 2

subject, date – p. 12/13

Variations

Exact asymptotics VAL GT(k[obl], n) N 08, Peretz 10 VAL GT(k[rec], n[rec]) Peretz 10

subject, date – p. 13/13