The Multiple Dimensions of Mean-Payoff Games Laurent Doyen CNRS - - PowerPoint PPT Presentation

▶
the multiple dimensions of mean payoff games
SMART_READER_LITE
LIVE PREVIEW

The Multiple Dimensions of Mean-Payoff Games Laurent Doyen CNRS - - PowerPoint PPT Presentation

Oct 23, 2023 199 likes •1.17k views

The Multiple Dimensions of Mean-Payoff Games Laurent Doyen CNRS & LSV, ENS Paris-Saclay RP 2017 About Basics about mean-payoff games Algorithms & Complexity Strategy complexity Memory Strategy complexity Memory

slide-1
SLIDE 1

The Multiple Dimensions of Mean-Payoff Games

Laurent Doyen CNRS & LSV, ENS Paris-Saclay RP 2017

slide-2
SLIDE 2

About

Basics about mean-payoff games

  • Algorithms & Complexity
  • Strategy complexity – Memory
  • Strategy complexity – Memory

Focus

  • Equivalent game forms
  • Techniques for memoryless proofs
slide-3
SLIDE 3

Mean-Payoff Mean-Payoff

slide-4
SLIDE 4

Mean-Payoff

slide-5
SLIDE 5

Mean-Payoff

slide-6
SLIDE 6

Mean-Payoff

slide-7
SLIDE 7

Mean-Payoff

Switching policy to get average power (1,1) ?

slide-8
SLIDE 8

Mean-Payoff

Switching policy to get average power (1,1) ?

slide-9
SLIDE 9

Mean-Payoff

Mean-payoff value = limit-average of the visited weights

slide-10
SLIDE 10

Mean-Payoff

Switching policy

  • Infinite memory: (1,1) vanishing frequency in q0
  • Infinite memory: (1,1) vanishing frequency in q0
slide-11
SLIDE 11

Mean-Payoff

limit ?

slide-12
SLIDE 12

Mean-Payoff

limit ?

limsup liminf

Mean-payoff is prefix-independent

slide-13
SLIDE 13

Mean-Payoff

Switching policy

  • Infinite memory: (1,1) for liminf
  • Infinite memory: (1,1) for liminf
slide-14
SLIDE 14

Mean-Payoff

Switching policy

  • Infinite memory: (1,1) for liminf & (2,2) for limsup
  • Infinite memory: (1,1) for liminf & (2,2) for limsup
slide-15
SLIDE 15

Games Games

slide-16
SLIDE 16

Two-player games

slide-17
SLIDE 17

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

  • Turn-based
  • Infinite duration
slide-18
SLIDE 18

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-19
SLIDE 19

Two-player games

Player 1 (maximizer) Player 2 (minimizer)

  • Turn-based

Play:

  • Turn-based
  • Infinite duration
slide-20
SLIDE 20

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-21
SLIDE 21

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-22
SLIDE 22

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-23
SLIDE 23

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-24
SLIDE 24

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-25
SLIDE 25

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-26
SLIDE 26

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-27
SLIDE 27

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-28
SLIDE 28

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

Play:

  • Turn-based
  • Infinite duration
slide-29
SLIDE 29

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer) Strategies = recipe to extend the play prefix Player 1: Player 2:

  • Turn-based
  • Infinite duration
slide-30
SLIDE 30

Two-player games

  • Turn-based

Player 1 (maximizer) Player 2 (minimizer)

  • Turn-based
  • Infinite duration

Strategies = recipe to extend the play prefix

  • utcome of two

strategies is a play Player 1: Player 2:

slide-31
SLIDE 31

Mean-payoff games Mean-payoff games

slide-32
SLIDE 32

Mean-payoff games

positive and negative weights

(encoded in binary)

Decision problem: Mean-payoff game: Decide if there exists a player-1 strategy to ensure mean-payoff value ≥ 0 Value problem:

slide-33
SLIDE 33

Mean-payoff games

Key ingredients:

  • identify memory requirement:

infinite vs. finite vs. memoryless

  • solve 1-player games (i.e., graphs)

Key arguments for memoryless proof:

  • backward induction
  • shuffle of plays
  • nested memoryless objectives
slide-34
SLIDE 34

Reduction to Reachability Games

slide-35
SLIDE 35

Reduction to Reachability Games

Reachability objective: positive cycles (v ≥ 0) positive cycles (v ≥ 0)

slide-36
SLIDE 36

Reduction to Reachability Games

Reachability objective: positive cycles (v ≥ 0) positive cycles (v ≥ 0)

slide-37
SLIDE 37

Reduction to Reachability Games

Reachability objective: positive cycles (v ≥ 0) positive cycles (v ≥ 0)

slide-38
SLIDE 38

Reduction to Reachability Games

Reachability objective: positive cycles (v ≥ 0) positive cycles (v ≥ 0) If player 1 wins only positive cycles are formed mean-payoff value ≥ 0 If player 2 wins only negative cycles are formed mean-payoff value < 0

(Note: limsup vs. liminf does not matter)

slide-39
SLIDE 39

Reduction to Reachability Games

Reachability objective: positive cycles (v ≥ 0) positive cycles (v ≥ 0) Mean-payoff game Ensuring positive cycles Memoryless strategy transfers to finite-memory mean-payoff winning strategy

slide-40
SLIDE 40

Strategy Synthesis

Memoryless mean-payoff winning strategy ? winning strategy ?

slide-41
SLIDE 41

Memoryless mean-payoff winning strategy ?

Strategy Synthesis

winning strategy ?

slide-42
SLIDE 42

Memoryless mean-payoff winning strategy ?

Strategy Synthesis

winning strategy ? Progress measure: minimum initial credit to stay always positive

slide-43
SLIDE 43

Memoryless mean-payoff winning strategy ?

Strategy Synthesis

winning strategy ? Progress measure: minimum initial credit to stay always positive

slide-44
SLIDE 44

Memoryless mean-payoff winning strategy ?

Strategy Synthesis

winning strategy ? Progress measure: minimum initial credit to stay always positive

slide-45
SLIDE 45

Memoryless mean-payoff winning strategy ?

Strategy Synthesis

winning strategy ? Progress measure: minimum initial credit to stay always positive

slide-46
SLIDE 46

Memoryless mean-payoff winning strategy ?

Strategy Synthesis

winning strategy ? Choose successor to stay above minimum credit minimum credit such that Progress measure: minimum initial credit to stay always positive In choose such that

slide-47
SLIDE 47

Memoryless mean-payoff winning strategy ?

Strategy Synthesis

winning strategy ? Choose successor to stay above minimum credit minimum credit such that Progress measure: minimum initial credit to stay always positive In choose such that

slide-48
SLIDE 48

Memoryless proofs

Key arguments for memoryless proof:

  • backward induction
  • backward induction
  • shuffle of plays
  • nested memoryless objectives
slide-49
SLIDE 49

Energy Games

Mean-payoff: average-value of the cycle. Energy: min-value of the prefix.

(if positive cycle; otherwise ∞)

Mean-payoff: average-value of the cycle.

slide-50
SLIDE 50

Energy Games

Winning strategy ?

slide-51
SLIDE 51

Energy Games

Winning strategy ? Follow the minimum initial credit !

slide-52
SLIDE 52

Multi-dimension Multi-dimension games

slide-53
SLIDE 53
  • Energy: initial credit to stay always above (0,0)

Multiple resources

Multi-dimension games

  • Energy: initial credit to stay always above (0,0)
  • Mean-payoff:
slide-54
SLIDE 54
  • Energy: initial credit to stay always above (0,0)

Multiple resources

Multi-dimension games

  • Energy: initial credit to stay always above (0,0)
  • Mean-payoff:

same ? same as positive cycles ?

slide-55
SLIDE 55
  • Energy: initial credit to stay always above (0,0)

Multiple resources

Multi-dimension games

  • Energy: initial credit to stay always above (0,0)
  • Mean-payoff:

same ? same as positive cycles ? If player 1 can ensure positive simple cycles, then energy and mean-payoff are satisfied. Not the converse !

slide-56
SLIDE 56

If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient

Multi-dimension games

slide-57
SLIDE 57

Let σ1 be winning L On each branch Then σ’1 is winning and finite memory

If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient

Multi-dimension games

... ... ... ... ... ...

L1 L2 With L1≤L2 stop and play as from L1 !

... ... ... ... ... ...

wqo + Koenig’s lemma (ℕd,≤) is well-quasi ordered

slide-58
SLIDE 58

Multi-energy games

For player 2 ? If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient

slide-59
SLIDE 59

Multi-energy games

For player 2, memoryless strategies are sufficient

  • induction on player-2 states
  • if ∃ initial credit against all memoryless strategies,

then ∃ initial credit against all arbitrary strategies. ‘left’ game ‘right’ game

slide-60
SLIDE 60

Multi-energy games

For player 2, memoryless strategies are sufficient

  • induction on player-2 states
  • if ∃ initial credit against all memoryless strategies,

then ∃ initial credit against all arbitrary strategies.

cl cr

cl+cr ‘left’ game ‘right’ game

⇒

Play is a shuffle of left-game play and right-game play Energy is sum of them

slide-61
SLIDE 61

Multi-energy games

For player 2, memoryless strategies are sufficient

  • induction on player-2 states
  • if ∃ initial credit against all memoryless strategies,

then ∃ initial credit against all arbitrary strategies.

cl cr

cl+cr ‘left’ game ‘right’ game

⇒

Play is a shuffle of left-game play and right-game play Energy is sum of them In general, we need Value against memoryless strategies Value against arbitrary strategies

slide-62
SLIDE 62

Memoryless proofs

Key arguments for memoryless proof:

  • backward induction
  • backward induction
  • shuffle of plays
  • nested memoryless objectives
slide-63
SLIDE 63

Multi-energy games

If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient For player 2, memoryless strategies are sufficient coNP ?

slide-64
SLIDE 64

Multi-energy games

If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient For player 2, memoryless strategies are sufficient coNP ?

not necessarily not necessarily simple cycle!

  • guess a memoryless strategy π for Player 2
  • Construct Gπ
  • check in polynomial time that Gπ contains no cycle

with nonnegative effect in all dimensions

slide-65
SLIDE 65

Multi-weighted energy games

Detection of nonnegative cycles ⇒ polynomial-time

  • Flow constraints using LP
  • Divide and conquer algorithm
  • Divide and conquer algorithm
slide-66
SLIDE 66

Multi-weighted energy games

Detection of nonnegative cycles ⇒ polynomial-time

  • Flow constraints using LP
slide-67
SLIDE 67

Multi-weighted energy games

Detection of nonnegative cycles ⇒ polynomial-time

  • Flow constraints using LP

Not connected !

slide-68
SLIDE 68

Multi-weighted energy games

Detection of nonnegative cycles ⇒ polynomial-time

  • Flow constraints using LP
  • Divide and conquer algorithm
  • Divide and conquer algorithm

Mark the edges that belong to some (pseudo) solution. Solve the connected subgraphs.

slide-69
SLIDE 69

Multi-weighted energy games

Detection of nonnegative cycles ⇒ polynomial-time

  • Flow constraints using LP
  • Divide and conquer algorithm
  • Divide and conquer algorithm

Mark the edges that belong to some (pseudo) solution. Solve the connected subgraphs.

slide-70
SLIDE 70

If player 1 has initial credit to stay always positive (Energy) then finite-memory strategies are sufficient For player 2, memoryless strategies are sufficient

Multi-dimension games

Equivalent with mean-payoff games (under finite-memory): If player 1 wins positive cycles are formed mean-payoff value ≥ 0 Otherwise, for all finite-memory strategy of player 1 (with memory M), player 2 can repeat a negative cycle (in G x M)

slide-71
SLIDE 71

Multi-dimension games

Player 1 Energy MP - liminf MP - limsup Finite memory . coNP-complete . Player 2 memoryless . Infinite memory .

slide-72
SLIDE 72

Player 1 Energy MP - liminf MP - limsup Finite memory . coNP-complete . Player 2 memoryless . coNP-complete

Multi-dimension games

Infinite memory . coNP-complete

  • Pl. 2 memoryless
  • Player 2 memoryless (shuffle argument)
  • Graph problem in PTIME (LP argument)
  • True for
  • False for
slide-73
SLIDE 73

Player 1 wins from every state in R if and only if player 1 wins each from every state in R

Multi-mean-payoff games

The winning region R of player 1 has the following characterization: Proof idea: (without leaving R)

slide-74
SLIDE 74

Multi-mean-payoff games

The winning region R of player 1 has the following characterization: Player 1 wins from every state in R if and only if player 1 wins each from every state in R Proof idea: (without leaving R)

Attr2(L) L

Losing for player 1 for single objective Winning for player 2, with memoryless strategy By induction, player 2 is memoryless in the subgame

slide-75
SLIDE 75

Memoryless proofs

Key arguments for memoryless proof:

  • backward induction
  • backward induction
  • shuffle of plays
  • nested memoryless objectives
slide-76
SLIDE 76

Player 1 Energy MP - liminf MP - limsup Finite memory . coNP-complete . Player 2 memoryless . coNP-complete NP ∩ coNP

Multi-dimension games

Infinite memory . coNP-complete

  • Pl. 2 memoryless

NP ∩ coNP

  • Pl. 2 memoryless
slide-77
SLIDE 77

Window games

Issues with mean-payoff

  • limsup vs. liminf
  • limsup vs. liminf
  • limit-behaviour, unbounded delay
  • complexity
slide-78
SLIDE 78

Window games

  • limsup vs. liminf

Issues with mean-payoff

unbounded window

  • limsup vs. liminf
  • limit-behaviour, unbounded delay
  • complexity

Sliding window of size at most B At every step, MP ≥ 0 within the window

slide-79
SLIDE 79

Window games

Window objective: from some point on, at every step, MP ≥ 0 within window of B steps prefix-independent bounded delay Implies the mean-payoff condition Implies the mean-payoff condition

slide-80
SLIDE 80

Window games

Window objective: from some point on, at every step, MP ≥ 0 within window of B steps prefix-independent bounded delay Implies the mean-payoff condition Complexity, Algorithm ?

  • like coBüchi objective

min-max cost (for ≤B steps) stable set (safety) attractor & subgame iteration Implies the mean-payoff condition

slide-81
SLIDE 81

Window games

Window objective: from some point on, at every step, MP ≥ 0 within window of B steps prefix-independent bounded delay Implies the mean-payoff condition Complexity, Algorithm ?

  • like coBüchi objective
  • multi-dimension: EXPTIME-complete

Implies the mean-payoff condition

slide-82
SLIDE 82

Hyperplane Separation

Multi-dimension mean-payoff (liminf): coNP-complete Naive algorithm: exponential in number of states Hyperplane separation: reduction to single-dimension mean-payoff games

slide-83
SLIDE 83

Hyperplane Separation

Multi-dimension Single dimension

⇒

slide-84
SLIDE 84

Hyperplane Separation

slide-85
SLIDE 85

Hyperplane Separation

Player 1 loses the multi-dimension game Player 1 cannot ensure MPλ ≥ 0 for some λ ⇔

slide-86
SLIDE 86

Hyperplane Separation

Player 1 wins the multi-dimension game Player 1 wins MPλ ≥ 0 for all λ ∈ (R+)d ⇔

slide-87
SLIDE 87

Hyperplane Separation

Player 1 wins the multi-dimension game Player 1 wins MPλ ≥ 0 for all λ ∈ (R+)d ⇔

slide-88
SLIDE 88

Hyperplane Separation

Player 1 wins the multi-dimension game Player 1 wins MPλ ≥ 0 for all λ ∈ (R+)d ⇔

slide-89
SLIDE 89

Hyperplane Separation

Player 1 wins the multi-dimension game Player 1 wins MPλ ≥ 0 for all λ ∈ (R+)d ⇔

slide-90
SLIDE 90

Hyperplane Separation

Player 1 wins the multi-dimension game Player 1 wins MPλ ≥ 0 for all λ ∈ (R+)d ⇔

slide-91
SLIDE 91

Hyperplane Separation

  • Multi-dimension mean-payoff (liminf): coNP-complete
  • Naive algorithm: exponential in number of states
  • Hyperplane separation: reduction to single-dimension mean-payoff games

Player 1 wins MPλ ≥ 0 for all λ ∈ (R+)d ⇔ In fact, it is sufficient for player 1 to win for all λ ∈ {0,…,(d⋅n⋅W)d+1}d Player 1 wins the multi-dimension game ⇔ Fixpoint algorithm:

  • remove states if losing for some λ
  • remove attractor (for player 2) of losing states

M Solving O(n⋅Md) mean-payoff games in O(n⋅m⋅M) O(n2⋅m⋅Md+1)

slide-92
SLIDE 92

Conclusion

Multiple dimensions of mean-payoff games

  • Reachability game
  • Energy game
  • Cycle-forming game

Multi-dimension mean-payoff games Multi-dimension mean-payoff games Memoryless proofs Other directions: parity condition, stochasticity, imperfect information

slide-93
SLIDE 93

Credits

  • Energy/Mean-Payoff Games is joint work with Lubos Brim, Jakub

Chaloupka, Raffaela Gentilini, Jean-Francois Raskin.

  • Multi-dimension Games is joint work with Krishnendu Chatterjee, Jean-

Francois Raskin, Alexander Rabinovich, Yaron Velner.

  • Window games is joint work with Krishnendu Chatterjee, Michael

Randour, Jean-Francois Raskin.

[BFLMS08] Bouyer, Fahrenberg, Larsen, Markey, Srba. Infinite Runs in weighted timed automata with energy

  • constraints. FORMATS’08.

[BJK10] Brazdil, Jancar, Kucera. Reachability Games on extended vector addition systems with states. ICALP’10. [CV14] Chatterjee, Velner. Hyperplane Separation Technique for Multidimensional Mean-Payoff Games. FoSSaCS’14. [Kop06] Kopczynski. Half-Positional Determinacy of Infinite Games. ICALP’06. [KS88] Kosaraju, Sullivan. Detecting cycles in dynamic graphs in polynomial time. STOC’88.

Other important contributions:

slide-94
SLIDE 94
slide-95
SLIDE 95

Thank you !

The end

Questions ?