Synthesis of Optimal Strategies Using H Y T ECH Patricia Bouyer 1 , - PowerPoint PPT Presentation

Synthesis of Optimal Strategies Using H Y T ECH Patricia Bouyer 1 , Franck Cassez 2 , Emmanuel Fleury 3 & Kim Guldstrand Larsen 3 1 LSV, ENS-Cachan, France 2 IRCCyN, Nantes, France 3 Comp. Science. Dept., Aalborg University, Danemark Games in Design and Verification Boston, Massachusetts, USA July 18, 2004 http://www.lsv.ens-cachan.fr/aci-cortos/ptga � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 1/35

Contents 1. Context & Related Work 2. Priced Timed Game Automata 3. From Optimal Control to Control � Computing The Optimal Cost � Computing Optimal Strategies 4. Implementation using H Y T ECH � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 2/35

Context Timed Automata x ≤ 2 ; a 1 ℓ 2 x ≥ 2 ; a 4 a 2 y := 0 ℓ 0 ℓ 1 Goal a 3 [ y = 0] x ≥ 2 ; a 5 ℓ 3 � Timed Automata + Reachability [AD94] � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 3-a/35

Context Timed Game Automata x ≤ 2 ; c 1 ℓ 2 x ≥ 2 ; c 2 u y := 0 ℓ 0 ℓ 1 Goal u [ y = 0] x ≥ 2 ; c 2 ℓ 3 � Timed Automata + Reachability [AD94] � Timed Game Automata: Control [MPS95, AMPS98] � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 3-b/35

Context As soon As Possible in Timed Automata 1 ≤ x ≤ 2 ; a 1 ℓ 2 x ≥ 2 ; a 4 u y := 0 ℓ 0 ℓ 1 Goal u [ y = 0] x ≥ 2 ; a 5 ℓ 3 � Timed Automata + Reachability [AD94] � Timed Game Automata: Control [MPS95, AMPS98] � Time Optimal Control (Reachability) [AM99] � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 3-c/35

Context Reachability in Priced Timed Automata x ≥ 2 ; a 4 cost = 1 x ≤ 2 ; a 1 ℓ 2 a 2 y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal a 3 cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; a 5 cost = 7 cost ( ℓ 3 ) = 1 � Timed Automata + Reachability [AD94] � Timed Game Automata: Control [MPS95, AMPS98] � Time Optimal Control (Reachability) [AM99] � Priced (or Weighted) Timed Automata [LBB + 01, ALTP01] � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 3-d/35

Context Priced Timed Game Automata x ≥ 2 ; c 2 cost = 1 x ≤ 2 ; c 1 ℓ 2 u y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal u cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; c 2 cost = 7 cost ( ℓ 3 ) = 1 � Timed Automata + Reachability [AD94] � Timed Game Automata: Control [MPS95, AMPS98] � Time Optimal Control (Reachability) [AM99] � Priced (or Weighted) Timed Automata [LBB + 01, ALTP01] � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 3-e/35

A Simple Example x ≥ 2 ; c 2 cost = 1 x ≤ 2 ; c 1 ℓ 2 u y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal u cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; c 2 cost = 7 cost ( ℓ 3 ) = 1 � Model = Game = Controller vs. Environment � What is the best cost whatever the environment does ? � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 4-a/35

A Simple Example x ≥ 2 ; c 2 cost = 1 x ≤ 2 ; c 1 ℓ 2 u y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal u cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; c 2 cost = 7 cost ( ℓ 3 ) = 1 � What is the best cost whatever the environment does ? 0 ≤ t ≤ 2 max { 5 t + 10(2 − t ) + 1 , 5 t + (2 − t ) + 7 } inf � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 4-b/35

A Simple Example x ≥ 2 ; c 2 cost = 1 x ≤ 2 ; c 1 ℓ 2 u y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal u cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; c 2 cost = 7 cost ( ℓ 3 ) = 1 � What is the best cost whatever the environment does ? 0 ≤ t ≤ 2 max { 5 t +10(2 − t )+1 , 5 t +(2 − t )+7 } at t = 4 3 inf = 141 inf 3 � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 4-c/35

A Simple Example x ≥ 2 ; c 2 cost = 1 x ≤ 2 ; c 1 ℓ 2 u y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal u cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; c 2 cost = 7 cost ( ℓ 3 ) = 1 � What is the best cost whatever the environment does ? ⇒ 14 1 3 at t = 4 = 3 � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 4-d/35

A Simple Example x ≥ 2 ; c 2 cost = 1 x ≤ 2 ; c 1 ℓ 2 u y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal u cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; c 2 cost = 7 cost ( ℓ 3 ) = 1 � What is the best cost whatever the environment does ? ⇒ 14 1 3 at t = 4 = 3 � Is there a strategy to achieve this optimal cost ? Yes: because see later � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 4-e/35

A Simple Example x ≥ 2 ; c 2 cost = 1 x ≤ 2 ; c 1 ℓ 2 u y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal u cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; c 2 cost = 7 cost ( ℓ 3 ) = 1 � What is the best cost whatever the environment does ? ⇒ 14 1 3 at t = 4 = 3 � Is there a strategy to achieve this optimal cost ? Yes: because see later � Can we compute such a strategy ? Yes: in ℓ 0 , x < 4 3 wait then do c 1 ; in ℓ 2 , 3 do c 2 when x ≥ 2 � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 4-f/35

Optimal Control Problems x ≥ 2 ; c 2 cost = 1 x ≤ 2 ; c 1 ℓ 2 u y := 0 cost ( ℓ 2 ) = 10 ℓ 0 ℓ 1 Goal u cost ( ℓ 0 ) = 5 [ y = 0] ℓ 3 x ≥ 2 ; c 2 cost = 7 cost ( ℓ 3 ) = 1 � Can we find algorithms for these problems on PTGA: 1. Compute the optimal cost 2. Decide if there is an optimal strategy 3. Compute an optimal strategy (if ∃ ) � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 4-g/35

Related Work � La Torre et al. [LTMM02] (IFIP TCS’02) • Acyclic Priced Timed Game Automata • Recursive definition of optimal cost [ = ⇒ La Torre et al. Def.] • Computation of the infimum of the optimal cost OptCost = 2 could be 2 or 2 + ε • No strategy synthesis � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 5-a/35

Related Work � La Torre et al. [LTMM02] (IFIP TCS’02) Acyclic Games, infimum, no strategy synthesis � Alur et al. [ABM04] (ICALP’04) • bounded optimality: optimal cost within k steps • complexity bound: exponential in k and #states of the PTGA • no bound for the more general optimal problem • Computation of the infimum of the optimal cost • no strategy synthesis � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 5-b/35

Related Work � La Torre et al. [LTMM02] (IFIP TCS’02) Acyclic Games, infimum, no strategy synthesis � Alur et al. [ABM04] (ICALP’04) bounded optimality, complexity bound, infimum, no strategy synthesis � Our work [BCFL04]: • Run-based definition of optimal cost • We can decide whether ∃ an optimal strategy • We can synthesize an optimal strategy (if ∃ ) • We can prove structural properties of optimal strategies • Applies to Linear Hybrid Game (Automata) � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 5-c/35

Contents 1. Context & Related Work 2. Priced Timed Game Automata 3. From Optimal Control to Control � Computing The Optimal Cost � Computing Optimal Strategies 4. Implementation using H Y T ECH � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 6/35

Priced Timed Game Automata A Timed Game Automaton (PTGA) G is a tuple ( L, ℓ 0 , Act , X, E, inv , cost ) where: � L is a finite set of locations; � ℓ 0 ∈ L is the initial location; � Act = Act c ∪ Act u is the set of actions (partitioned into controllable and uncontrollable actions); � X is a finite set of real-valued clocks; � E ⊆ L × B ( X ) × Act × 2 X × L is a finite set of transitions; � inv : L − → B ( X ) associates to each location its invariant; � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 7-a/35

Priced Timed Game Automata A Priced Timed Game Automaton (PTGA) G is a tuple ( L, ℓ 0 , Act , X, E, inv , cost ) where: � L is a finite set of locations; � E ⊆ L × B ( X ) × Act × 2 X × L is a finite set of transitions; � Priced Version: cost : L ∪ E − → N associates to each location a cost rate and to each discrete transition a cost value. [ = ⇒ Example] � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 7-b/35

Priced Timed Game Automata A Priced Timed Game Automaton (PTGA) G is a tuple ( L, ℓ 0 , Act , X, E, inv , cost ) where: � L is a finite set of locations; � E ⊆ L × B ( X ) × Act × 2 X × L is a finite set of transitions; � Priced Version: cost : L ∪ E − → N associates to each location a cost rate and to each discrete transition a cost value. [ = ⇒ Example] � we assume that PTGA are deterministic w.r.t. controllable actions (+ time-deterministic) � A reachability PTGA (RPTGA) = PTGA with distinguished set of states Goal ⊆ L . � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 7-c/35

Configurations, Runs, Costs � configuration: ( ℓ, v ) in L × R X ≥ 0 � step: t i = ( ℓ i , v i ) α i − → ( ℓ i +1 , v i +1 ) � α i ∈ R > 0 = ⇒ ℓ i +1 = ℓ i ∧ v i +1 = v i + α i α i ∈ Act = ⇒ ∃ ( ℓ i , g, α i , Y, ℓ i +1 ) ∈ E ∧ v i | = g ∧ v i +1 = v i [ Y ] � run ρ = t 0 t 1 t 2 · · · t n − 1 · · · finite or infinite sequence of t i � cost of a transition: � Cost ( t i ) = α i . cost ( ℓ i ) if α i ∈ R > 0 Cost ( t i ) = cost (( ℓ i , g, α i , Y, ℓ i +1 )) if α i ∈ Act � if ρ finite Cost ( ρ ) = � 0 ≤ i ≤ n − 1 Cost ( t i ) � winning run if States ( ρ ) ∩ Goal � = ∅ � IRCCyN/CNRS c Synthesis of Optimal Strategies Using H Y T ECH page 8/35