Games Miheer Dewaskar Chennai Mathematical Institute April 27, - - PowerPoint PPT Presentation

Games

Miheer Dewaskar

Chennai Mathematical Institute April 27, 2016

1 / 19

Outline

Finite Duration Games Win-Lose Games Payoff Games Infinite Duration Games Parity Games Mean Payoff Games Simple Stochastic Games

2 / 19

Outline

Finite Duration Games Win-Lose Games Payoff Games Infinite Duration Games Simple Stochastic Games

3 / 19

Finite games

Win-Lose game

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Box wins

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Circle wins

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Algorithm for optimal play

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Algorithm for optimal play

Circle Wins Box Wins

4 / 19

Finite games

Win-Lose game

Algorithm for optimal play

Box can always win

Circle Wins Box Wins

4 / 19

Finite games

Payoff game

4

• 1
• 1

4 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

4

• 1
• 1

4 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

4

• 1
• 1

4 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

Payoff

Min pays 4 units to Max

4

• 1
• 1

4 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

4

• 1
• 1

4 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

Payoff

Min pays -1 units to Max

4

• 1
• 1

4 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

MinMax algorithm

4

• 1
• 1
• 1

4 1 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

MinMax algorithm

4 4

• 1

1

• 1
• 1

4 1 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

MinMax algorithm

Value = 1 Min can ensure a payoff ≤ 1 Max can ensure a payoff ≥ 1

1 4 4

• 1

1

• 1
• 1

4 1 1

• 2

Maximizer Minimizer

5 / 19

Finite games

Payoff game

MinMax algorithm

Value = 1 Min can ensure a payoff ≤ 1 Max can ensure a payoff ≥ 1 When both play optimally the payoff is exactly 1.

1 4 4

• 1

1

• 1
• 1

4 1 1

• 2

Maximizer Minimizer

5 / 19

Outline

Finite Duration Games Infinite Duration Games Parity Games Mean Payoff Games Simple Stochastic Games

6 / 19

Parity Games

Winning conditions

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 =

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 =

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π =

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π = 1

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π = 1 2

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π = 1 2 3

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π = 1 2 3 3

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π = 1 2 3 3 6

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π = 1 2 3 3 6 5

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π = 1 2 3 3 6 5 2

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Winning conditions

π1 = 1 5 2 1 2 1 2 . . . inf(π1) = {1, 2} max Inf(π1) = 2 Even wins π2 = 1 5 2 1 5 2 . . . inf(π2) = {1, 2, 5} max Inf(π2) = 5 Odd wins π = 1 2 3 3 6 5 2 1 . . . Parity(max Inf(π)) wins

5 6 1 2 3 Odd Even

7 / 19

Parity Games

Questions

Does either Even or Odd have a strategy to always win? If so, then how to compute the winning strategy?

8 / 19

Parity Games

Questions

Does either Even or Odd have a strategy to always win? Yes If so, then how to compute the winning strategy? By reduction to finite duration games

8 / 19

Parity Games

5 6 1 2 3 Odd Even 1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

9 / 19

Parity Games

5 6 1 2 3 Odd Even 1 5 6 5 2 3 6 5 3 1 2 1 3 3 6 5 2 6 Odd Wins Even Wins

9 / 19

Parity Games

5 6 1 2 3 Odd Even 1 5 6 5 2 3 6 5 3 1 2 1 3 3 6 5 2 6 Odd Wins Even Wins

9 / 19

Parity Games

5 6 1 2 3 Odd Even 1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Even has a winning strategy

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 Stack = 1

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 Stack = 1 2

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 Stack = 1 2 1

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 Stack = 1

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 Stack = 1 5

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 Stack = 1 5 6

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 5 Stack = 1 5 6 5

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 5 Stack = 1 5

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 5 2 Stack = 1 5 2

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 5 2 3 Stack = 1 5 2 3

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 5 2 3 6 Stack = 1 5 2 3 6

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 5 2 3 6 5 Stack = 1 5 2 3 6 5 Every eliminated cycle has max priority even

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 5 2 3 6 5 Stack = 1 5 . . . Every eliminated cycle has max priority even

9 / 19

Parity Games

5 6 1 2 3 Odd Even

Finite game

Every loop has max priority even

1 5 6 2 3 6 2 3 6 5 Odd Wins Even Wins

Extension to infinite plays

π = 1 2 1 5 6 5 2 3 6 5 Stack = 1 5 . . . Every eliminated cycle has max priority even Hence max Inf priority in π is Even

9 / 19

Parity Games

Better Algorithms

Marcin Jurdzinski and Jens Vöge. “A discrete strategy improvement algorithm for solving parity games”. In: Computer Aided Verification. Springer, 2000, pp. 202–215 Upper bound1 : O

• (n/d)d

1see also Friedmann, “Exponential Lower Bounds for Solving Infinitary Payoff Games

and Linear Programs”.

10 / 19

Parity Games

Better Algorithms

Marcin Jurdzinski and Jens Vöge. “A discrete strategy improvement algorithm for solving parity games”. In: Computer Aided Verification. Springer, 2000, pp. 202–215 Upper bound1 : O

• (n/d)d

Marcin Jurdzinski, Mike Paterson, and Uri Zwick. “A Deterministic Subexponential Algorithm for Solving Parity Games”. In: SIAM Journal on Computing 38.4 (Jan. 2008), pp. 1519–1532 nO(√n)

1see also Friedmann, “Exponential Lower Bounds for Solving Infinitary Payoff Games

and Linear Programs”.

10 / 19

Outline

Finite Duration Games Infinite Duration Games Parity Games Mean Payoff Games Simple Stochastic Games

11 / 19

Mean Payoff Games

Payoffs

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a −2 − − →b − 2 1

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a −2 − − →b +3 − − →a − 2 + 3 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a −2 − − →b +3 − − →a −2 − − →b − 2 + 3 − 2 3

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a − 2 + 3 − 2 + 3 4

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b − 2 + 3 − 2 + 3 − 2 5

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a − 2 + 3 − 2 + 3 − 2 + 3 6

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a −1 − − →c − 1 1

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a −1 − − →c −2 − − →b − 1 − 2 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a −1 − − →c −2 − − →b +3 − − →a − 1 − 2 + 3 3

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a −1 − − →c −2 − − →b +3 − − →a −1 − − →c − 1 − 2 + 3 − 1 4

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a −1 − − →c −2 − − →b +3 − − →a −1 − − →c −2 − − →b − 1 − 2 + 3 − 1 − 2 5

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω

a −1 − − →c −2 − − →b +3 − − →a −1 − − →c −2 − − →b +3 − − →a − 1 − 2 + 3 − 1 − 2 + 3 6

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω Min pays −1

3 units to Max

a −1 − − →c −2 − − →b +3 − − →a −1 − − →c −2 − − →b +3 − − →a . . . − 1 − 2 + 3 − 1 − 2 + 3 6 ∼ n(−1−2+3)

3n

→ 0

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω Min pays −1

3 units to Max

a −1 − − →c −2 − − →b +3 − − →a −1 − − →c −2 − − →b +3 − − →a . . . − 1 − 2 + 3 − 1 − 2 + 3 6 ∼ n(−1−2+3)

3n

→ 0 Min tries to minimize lim Max tries to maximize lim

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Payoffs (ab)ω Min pays 1

2 units to Max

a −2 − − →b +3 − − →a −2 − − →b +3 − − →a −2 − − →b +3 − − →a . . . − 2 + 3 − 2 + 3 − 2 + 3 6 ∼ n(−2+3)

2n

→ 1 2

(acb)ω Min pays −1

3 units to Max

a −1 − − →c −2 − − →b +3 − − →a −1 − − →c −2 − − →b +3 − − →a . . . − 1 − 2 + 3 − 1 − 2 + 3 6 ∼ n(−1−2+3)

3n

→ 0 Generally Min tries to minimize lim sup Max tries to maximize lim inf

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

12 / 19

Mean Payoff Games

Questions

Does the game have a value? i.e. is there a v so that

Max can ensure lim inf ≥ v Min can ensure lim sup ≤ v

13 / 19

Mean Payoff Games

Questions

Does the game have a value? i.e. is there a v so that

Max can ensure lim inf ≥ v Min can ensure lim sup ≤ v

Yes How to compute the optimal strategies?

13 / 19

Mean Payoff Games

Questions

Does the game have a value? i.e. is there a v so that

Max can ensure lim inf ≥ v Min can ensure lim sup ≤ v

Yes How to compute the optimal strategies? Solution using the finite game

13 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

14 / 19

Mean Payoff

Finite Game

a b c

• 1

3 1 2

c

• 1

b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

14 / 19

Mean Payoff

Finite Game

a b c

• 1

3 1 2

c

• 1

b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Max can ensure ≥ 0 in the finite game

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a Stack = a

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b Stack = a b

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c Stack = a b c

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b Stack = a b c b

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b Stack = a b

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b c Stack = a b c

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b c a Stack = a b c a

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b c a Stack = a

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b c a b Stack = a b

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b c a b c Stack = a b c

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b c a b c a Stack = a b c a Every time a cycle with average value ≤ 0 is eliminated

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b c a b c a Stack = a Every time a cycle with average value ≤ 0 is eliminated

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Min can ensure ≤ 0 in the mean payoff game too π = a b c b c a b c a Stack = a Hence limsup of averages of π is ≤ 0

14 / 19

Mean Payoff

Finite Game

a b c c b

• 1
• 1

3 1 2

a b c

• 2
• 1

2

• 2
• 1

3 Min Max

Max can ensure ≥ 0 Similarly Max can ensure liminf

• f the average is ≥ 0

Hence the value of Mean payoff game is 0

14 / 19

Outline

Finite Duration Games Infinite Duration Games Simple Stochastic Games

15 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

Circle Wins

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

Box Wins

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

Or

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

Circle can win from with probability 1

1 2 1 2 1 2 1 2 16 / 19

Simple Stochastic Game

Values

1 1

1 2

1

1 2 1 2 1 2 1 2 17 / 19

Simple Stochastic Game

Values

1 1

1 2 1 2

1

1 2 1 2 1 2 1 2 17 / 19

Simple Stochastic Game

Values

v( ) = 1 v( ) = 0 v( ) = 1 2(v( ) + v( )) v( ) = 1 2(v( ) + v( )) v( ) = max{v( ), v( )} v( ) = min{v( ), v( )}

1 1

1 2 1 2

1

1 2 1 2 1 2 1 2 17 / 19

Simple Stochastic Game

Values

v( ) = 1 v( ) = 0 v( ) = 1 2(v( ) + v( )) v( ) = 1 2(v( ) + v( )) v( ) = max{v( ), v( )} v( ) = min{v( ), v( )}

1 1

1 2 1 2

1

1 2 1 2 1 2 1 2

These equations have a unique solution.

17 / 19

Simple Stochastic Game

Values

v( ) = 1 v( ) = 0 v( ) = 1 2(v( ) + v( )) v( ) = 1 2(v( ) + v( )) v( ) = max{v( ), v( )} v( ) = min{v( ), v( )}

1 1

1 2 1 2

1

1 2 1 2 1 2 1 2

These equations have a unique solution. From state s - has a strategy to reach with probability ≥ v(s) has a strategy to reach with probability ≥ 1 − v(s)

17 / 19

Complexity of solving games

Does Even win the Parity Game? Is the value of the Mean Payoff Game ≥ 0 Is the value in the Simple Stochasic Game ≥ 1

2

NP∩coNP

2

18 / 19

Complexity of solving games

Does Even win the Parity Game? Is the value of the Mean Payoff Game ≥ 0 Is the value in the Simple Stochasic Game ≥ 1

2

NP∩coNP

2

18 / 19

Complexity of solving games

Does Even win the Parity Game? Is the value of the Mean Payoff Game ≥ 0 Is the value in the Simple Stochasic Game ≥ 1

2

NP∩coNP

2

2Chatterjee and Fijalkow, “A reduction from parity games to simple stochastic games”. 18 / 19

Complexity of solving games

Does Even win the Parity Game? Is the value of the Mean Payoff Game ≥ 0 Is the value in the Simple Stochasic Game ≥ 1

2

NP∩coNP

2

Open Problem

Is there a polynomial time algorithm for any of them?

2Chatterjee and Fijalkow, “A reduction from parity games to simple stochastic games”. 18 / 19

Timeline

Lloyd S. Shapley. “Stochastic games”. In: Proceedings of the National Academy of Sciences 39.10 (1953), pp. 1095–1100 E.A. Emerson and C.S. Jutla. “Tree automata, mu-calculus and determinacy”. In: IEEE Comput. Soc. Press, 1991, pp. 368–377 Anne Condon. “The complexity of stochastic games”. In: Information and Computation 96.2 (Feb. 1992), pp. 203–224 Uri Zwick and Mike Paterson. “The complexity of mean payoff games

• n graphs”.

In: Theoretical Computer Science 158.1 (May 1996),

• pp. 343–359

Marcin Jurdziski. “Deciding the winner in parity games is in UP co-UP”. . In: Information Processing Letters 68.3 (1998), pp. 119–124

19 / 19

Timeline

Lloyd S. Shapley. “Stochastic games”. In: Proceedings of the National Academy of Sciences 39.10 (1953), pp. 1095–1100 E.A. Emerson and C.S. Jutla. “Tree automata, mu-calculus and determinacy”. In: IEEE Comput. Soc. Press, 1991, pp. 368–377 Anne Condon. “The complexity of stochastic games”. In: Information and Computation 96.2 (Feb. 1992), pp. 203–224 Uri Zwick and Mike Paterson. “The complexity of mean payoff games

• n graphs”.

In: Theoretical Computer Science 158.1 (May 1996),

• pp. 343–359

Marcin Jurdziski. “Deciding the winner in parity games is in UP co-UP”. . In: Information Processing Letters 68.3 (1998), pp. 119–124

Thank you

19 / 19