Two-Player Zero-sum Games Played on Graphs: -Regular and - PowerPoint PPT Presentation

Two-Player Zero-sum Games 
 Played on Graphs: 
 ω -Regular and Quantitative Objectives Jean-François Raskin Université Libre de Bruxelles MOVEP 2020 Grenoble

Introduction : Motivations and 
 Basic Definitions

The Basic Model

Two-player games 
 played on graphs 1 (Finite) directed graph 2 4 Two types of vertices (states) 
 (Player 1 and Player 2) 3 5

Two-player games 
 played on graphs How to play ? 1 One token is placed on initial state Players play ω rounds : 2 4 in each round: the player that owns the state moves the token to an 3 adjacent state Outcome= ω -path 5

1 2 4 3 5 1

1 2 4 3 5 1 ➞ 2

1 2 4 3 5 1 ➞ 2 ➞ 1

1 2 4 3 5 1 ➞ 2 ➞ 1 ➞ 4

1 2 4 3 5 1 ➞ 2 ➞ 1 ➞ 4 ➞ 5

1 2 4 3 5 1 ➞ 2 ➞ 1 ➞ 4 ➞ 5 ➞ 3

1 2 4 3 5 1 ➞ 2 ➞ 1 ➞ 4 ➞ 5 ➞ 3 ➞ 4

1 2 4 3 5 1 ➞ 2 ➞ 1 ➞ 4 ➞ 5 ➞ 3 ➞ 4 ➞ 5

1 2 4 3 5 1 ➞ 2 ➞ 1 ➞ 4 ➞ 5 ➞ 3 ➞ 4 ➞ 5 ➞ 4 ➞ ...

Motivations

Reactive system synthesis 
 as solving a game ☞ support the design process with automatic synthesis Env ? ⊨ ψ || Ø Sys is constructed by an algorithm Ø Sys is correct by construction Ø Underlying theory: 2-player zero-sum games Ø Env is adversarial (worst-case assumption) Winning strategy = Correct Sys

Games for solving 
 automata problems Nondeterministic finite tree automata emptiness =two-player zero-sum reachability game Language inclusion between 
 non-deterministic Büchi automata =two-player zero-sum Parity game with imperfect information … and many more …

Checking refinements between two systems Spec Implementation ≽ A B Simulation ? Safety game between Prover and Spoiler Spoiler proposes moves in B Prover tries to match them in A Prover wins if he never fails to match the proposed moves

Model-checking 
 Mu-Calculus Sys ⊨ ? φ The model-checking problem for the Mu-Calculus can be reduced to the problem of deciding the winner in a zero-sum two-player parity game

Controller synthesis with quantitative objectives Embedded Control Communica<on Protocols Parts of OS/Chipset Security Protocols In most of those examples, quantitative measures of performances are important : not only a matter of correctness !!!

Controller synthesis with quantitative objectives Env ? ⊨ ψ || both the model and the property exhibit quantities

Plan of the lectures • Introduction - Motivations - Basic definitions • Qualitative games • Reachability and safety games • Büchi, coBüchi and parity games • Quantitative games • Sup, Inf, LimSup, LimInf games one and multiple dimensions • Mean-payoff and energy games

Part I: Qualitative Games

Basic Definitions


 Game arena A finite (turn-based) two-player game arena is a tuple A=(S 1 ,S 2 ,E,s init ) where: • S 1 is the finite set of states owned by Player 1 (the protagonist ) • S 2 is the finite set of states owned by Player 2 (the antagonist ) 
 We write S for S 1 ∪ S 2 • E ⊆ S 1 ∪ S 2 × S 1 ∪ S 2 is the set of transitions . Unless otherwise stated, we assume that E is total , i.e. ∀ s ∈ S, ∃ s’ ∈ S:(s,s’) ∈ E • s init ∈ S 1 ∪ S 2 is the initial state of the game arena Sometimes, we consider arena without designated initial states.

Plays and histories • Let A=(S 1 ,S 2 ,E,s init ) be a two-player game arena • A play in A is an infinite sequence of states ρ =s 0 s 1 … s n … ∈ S ω 
 such that for all i ≥ 0, (s i ,s i+1 ) ∈ E • A play ρ =s 0 s 1 … s n … is initial if s 0 =s init • Let ρ =s 0 s 1 … s n …, we use the following notations: • ρ (i) for the state s i • ρ (i..j) for the infix s i s i+1 … s j • ρ (i..) for for the suffix s i s i+1 …s n … • visit ( ρ ) is the set of states that appear along ρ • inf ( ρ ) is the set of states that appear infinitely often along ρ

Plays and histories • We denote by Play (A) and InitPlay (A) the set of plays and the set of initial plays of A • A history π =s 0 s 1 … s n ∈ S* is a finite sequence of states, a history π =s 0 s 1 … s n belongs to Player 1 if its last state, noted last ( π )=s n , is in S 1 , and to Player 2 if last ( π ) ∈ S 2 • We note Hist 1 (A)=S*.S 1 and Hist 2 (A)=S*.S 2 the set of histories of Player 1 and Player 2 respectively

Rounds and play • The two players play on the game arena as follows • A game in A is played by the two players for an infinite number of rounds. Each round is as follows: • at the start of a round, a token is on a state s of the arena, the Player that owns the state s chooses a successor state s’ which is E-adjacent to s, i.e. (s,s’) ∈ E. The token is then moved to s’ and a new round is started from there • initially, the token is on s init • This interaction generates a play ρ ∈ Play (A) which is initial if the interaction is started from s init


 
 Strategies • Player plays according to strategies • A strategy for Player j, j ∈ {1,2}, is a function that maps histories to states: 
 λ j : S*.S j → S 
 with the restriction that for all histories π ∈ Hist j (A), ( last ( π ), λ j ( π )) ∈ E, i.e. the strategy always proposes a (next) state that is E-adjacent to the last state of the history.

Strategies Unfolding of the game graph ... ... ... ... ... ...

Strategies ... ... ... ... ... ... ... ... ... ... ... ...

Strategies ... ... ... ... ... ... ... ... ... ... ... ...

Strategies Strategy for Player 1 = One choice in each node of Player I in tree unfolding ... ... ... ... ... ... ... ... ... ... ... ...

Outcomes of a strategy • A play ρ ∈ S ω is compatible with a strategy λ j of Player j, if for all i ≥ 0 s.t. last ( ρ (0..i)) ∈ S j , ρ (i+1)= λ j ( ρ (0..i)), i.e. the position that follows a position that belongs to Player j is determined by λ j . • We note Outcome A (s, λ j ) the set of plays starting in s that are compatible with λ j . We write Outcome A (s, λ 1 , λ 2 ) the unique play that starts in s and is both compatible with λ 1 and λ 2 • We omit A if clear from the context, and when s is omitted then it is assumed to be equal to s init , the initial state of A

Types of strategies • A strategy λ j : S*.S j → S is memoryless if 
 for all histories π , π ’ ∈ S*.S j such that last ( π )= last ( π ’), λ j ( π )= λ j ( π ’), i.e. the strategy only depends on the last state of the history • So a memoryless strategy is equivalent to a function λ j : S j → S • A strategy λ j : S*.S j → S is finite memory if 
 there exists a equivalence relation ≈ on Hist j (A) of finite index such that for all π , π ’ ∈ S*.S j s.t. π≈π ’: λ j ( π )= λ j ( π ’) • If ≈ is computable by a finite state machine then a finite memory strategy can be represented as a synchronous transducer

Memoryless strategy An example A memoryless strategy for Player 1 1 1 2 2 4 4 3 3 5 5

Finite memory strategy An example 5 4 1 5 ε 4 4 3 1 2 4 4 3 4 ε 4 5 1 5 3 Strategy: -in 1 always play 4 5 -in 4 always play 5 -in 3 play 4 every odd visit 
 and 1 every even visit

Winning objective • A winning objective for player j Win j ⊆ S ω defines what are the « good » plays for Player j • A game (A,Win 1 ,Win 2 ) is specified by a game arena and two winning objectives, one for each player • A game is zero-sum if Win 2 =S ω \Win 1 . In this series of lectures, we will only consider zero-sum games and we will most often leave the winning objective of Player 2 implicit, because in this case Win 2 =S ω \Win 1

Winning strategy • In a game (A,Win 1 ,Win 2 ), a strategy for Player j is a winning strategy if Outcome A (s init , λ j ) ⊆ Win j • Clearly, in a zero-sum game, a play is either winning for one player or for the other • But does a zero-sum game always have a winner, i.e. one player that has a winning strategy ?

Determinacy • Let A=(S 1 ,S 2 ,E,s init ) be a two-player turn based arena, and Win 1 be the winning objective for Player 1 • The game (A,Win 1 ) is determined if 
 either there exists a strategy λ 1 for Player 1 s.t. Outcome ( λ 1 ) ⊆ Win 1 , 
 or there exists a strategy λ 2 for Player 2 s.t. Outcome ( λ 2 ) ⊆ S ω \Win 1 =Win 2 • Theorem [Martin 1975]. 
 For all two-player turn based arenas A=(S 1 ,S 2 ,E,s init ) with a zero-sum winning condition, for all Borel definable objectives Win 1 , (A,Win 1 ) is determined Borel set =is any set in a topological space that can be formed from open sets using countable union, countable intersection, and complement 
 Rule of thumb : any reasonable set included in S ω that you can think of is a Borel set

Controllable Predecessors and Attractors

Controllable predecessors • Let A=(S 1 ,S 2 ,E,s init ) be a two-player game arena. • Let T ⊆ S be a set of states. • CPre 1 (T)= { s ∈ S 1 | ∃ s’ ∈ T: (s,s’) ∈ E} 
 ∪ { s ∈ S 2 | ∀ s’ ∈ T: (s,s’) ∈ E} • CPre 2 (T)= { s ∈ S 2 | ∃ s’ ∈ T: (s,s’) ∈ E} 
 ∪ { s ∈ S 1 | ∀ s’ ∈ T: (s,s’) ∈ E} • if s ∈ CPre j (T) then from s, Player j has a strategy to enforce the set T in one step

CPre 1 (T) ✔ ✕ T ✕ ✔