Synthesis from LTL Specifications with Mean-Payoff Objectives Aaron - - PowerPoint PPT Presentation

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Synthesis from LTL Specifications with Mean-Payoff Objectives

Aaron Bohy1 V´ eronique Bruy` ere1 Emmanuel Filiot2 Jean-Fran¸ cois Raskin3

1Universit´

e de Mons

2Universit´

e Paris-Est Cr´ eteil

3Universit´

e Libre de Bruxelles

ETAPS - TACAS 2013 Rome, Italy

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Overview

LTL realizability and synthesis with mean-payoff objectives:

• Qualitative objectives
• Quantitative objectives

Interest:

• Reactive systems
• Resource-constrained environments

Contributions:

• Incremental symbolic algorithm based on antichains
• Implementation in Acacia+

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

LTL Realizability game

Let φ be an LTL formula over the set P = I⊎O of atomic signals LTL Realizability game

• 2-player game:
• Player O, the system, controls the set O
• Player I, the environment, controls the set I
• Infinite play: at each round j,
• Player O gives a subset oj ⊆ O
• Player I responds by giving ij ⊆ I
• Outcome of the game: w = (i1∪o1)(i2∪o2). . . (ik∪ok). . .
• Player O wins the play if w satisfies φ, otherwise Player I wins

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

LTLMP Realizability game

Let φ be an LTL formula over the set P = I⊎O of atomic signals Let v : Lit(P) → Z be a cost function Let c ∈ Q be a threshold value LTLMP Realizability game

• 2-player infinite game
• Outcome of the game: w = (i1∪o1)(i2∪o2). . . (ik∪ok). . .
• Let

MP(w, v) = lim inf

n→∞

1 n

n

• j=1
• l∈Lit(ij∪oj)

v(l) be the mean-payoff value of w w.r.t. v

• Player O wins the play if (1) w satisfies φ and (2) MP(w, v) ≥ c,
• therwise Player I wins

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Problems

LTL realizability problem Decide whether the system has a winning strategy to satisfy φ against any strategy of the environment LTLMP realizability problem Given a threshold c ∈ Q, decide whether the system has a winning strategy (1) to satisfy φ and (2) to ensure a mean-payoff value ≥ c against any strategy of the environment LTL/LTLMP synthesis problem Produce such a winning strategy when φ is realizable

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Motivating example

2 clients sharing access to a resource

Desired properties: (1) every request must be eventually granted by the server (2) the server can grant only one request at a time (mutual exclusion) (3) the requests emitted by client 2 must be prioritized (3) is an optimality criterion ⇒ best modeled using a cost function and a mean-payoff objective

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Motivating example

2 clients sharing access to a resource

Let I = {r1, r2}, O = {g1, g2, w1, w2} (1) every request must be eventually granted by the server (r1 → X(w1Ug1)) ∧ (r2 → X(w2Ug2)) (2) the server can grant only one request at a time (mutual exclusion) (¬g1 ∨ ¬g2) (3) the requests emitted by client 2 must be prioritized Idea: Impose penalties to w1 and w2: Let v : Lit(I ∪ O) → Z s.t.      v(w1) = −1 v(w2) = −2 v(l) = 0, ∀l ∈ {w1, w2}

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Motivating example

2 clients sharing access to a resource

Winning strategies for Player O for properties:

• (1) and (2):

(g1, w2) (g2, w1) T T

• (1), (2) and (3) to ensure a mean-payoff value of −1.2:

(g2, w1) (g2, w1) (g2, w1) (g2, w1) (g1, w2) T T T T T

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

State of the art

Results of [PR89]:

• 2ExpTime-Complete
• Finite-memory strategies

Classical procedure:

• Reduction to Rabin games with Safra’s determinization

Safraless procedures:

• Reduction to B¨

uchi games [KV05]:

• Implemented in Lily [JB06]
• Reduction to safety games [SF07, FJR09]:
• Implemented in Unbeast [Ehl10] with BDDs
• Implemented in Acacia+ [BBF+12] with antichains

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Reduction to safety games

Method proposed by [SF07, FJR09]:

LTL ↓ 2O(n) Universal coB¨ uchi Word Automaton ↓ O(1) Universal K-coB¨ uchi Word Automaton ↓ 2O(m2) Deterministic K-coB¨ uchi Word Automaton ⇒ Solve a safety game

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Step 2: strengthening of the acceptance condition

Universal coB¨ uchi Word Aut. − → Universal K-coB¨ uchi Word Aut. New acceptance condition: Universal K-coB¨ uchi Word Automaton: w is accepted iff all runs labeled by w visit at most K accepting states Result from [FJR09]: Aφ is realizable ⇔ Aφ,K is realizable where K = 2n(n2n+2 + 1) and n the number of states of Aφ ⇒ Strenghten the acceptance condition

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Step 3: determinization

Universal K-coB¨ uchi Word Aut. − → Det. K-coB¨ uchi Word Aut. Determinization:

• Extension of subset construction with counters
• Counting functions f from Q to [−1, 0, . . . , K + 1]
• f (q) = −1: state q not reached
• f (q) = i ≥ 0: i is the maximal number of accepting states visited by

runs ending up in q

• Bad states are counting functions f such that ∃q: f (q) > K

⇒ Solve a safety game det(A, K), where the system tries to avoid bad states

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Goals

Solve det(A, K) efficiently:

• Avoid determinization
• Avoid bound K = 2n(n2n+2 + 1)
• Quickly find winning strategies with heuristics

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Properties of the safety game

Property 1: partial order on counting functions f f ′ ⇔ ∀q: f (q) ≤ f ′(q)

• Intuition: If we can avoid bad states from f ′, we can also avoid

them from f

• Compact symbolic representation of downward-closed sets by their

maximal elements with antichains Example: ⇒ Construction on the fly, with antichains

• Synthesis of compact strategies

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Properties of the safety game

Property 2: incremental algorithm

• If det(A, k1) is realizable, then det(A, k2) is realizable, ∀k2>k1

⇒ Incremental algorithm on the values of k = 0, 1, 2, . . .

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Back to motivating example

Desired properties: (1) every request must be eventually granted by the server (2) the server can grant only one request at a time (mutual exclusion) Let I = {r1, r2} and O = {g1, g2,w1, w2} (r1 → X(w1Ug1)) ∧ (r2 → X(w2Ug2))

• (1)

∧ (¬g1 ∨ ¬g2)

• (2)

Winning strategy for Player O output by Acacia+:

(g1, w2) (g2, w1) T T

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Complexity

Theorem: The LTLMP synthesis problem is 2ExpTime-Complete. ⇒ Reduction to a Mean-Payoff Parity Game (MPPG) of size 22O(nlogn) and with 2O(n) priorities

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Back to motivating example

Additional property: (3) the requests emitted by client 2 must be prioritized Let      v(w1) = −1 v(w2) = −2 v(l) = 0, ∀l ∈ {w1,w2} (r1 → X(w1Ug1)) ∧ (r2 → X(w2Ug2)) ∧ (¬g1 ∨ ¬g2) Optimal strategy for Player O requires infinite memory:

• Almost always immediately grant client 2
• Grant client 1 with decreasing frequency

⇒ Ensures a mean-payoff value of −1

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Finite memory ǫ-optimal strategies

Results from [CHJ05]: (1) Optimal strategies in MPPGs may require infinite memory (2) Finite memory ǫ-optimal strategies exist in MPPGs, ∀ǫ > 0 ⇒ Focus on ǫ-optimal strategies

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

From MPPGs to Energy Parity Games

Results of [CD10]: (1) MPPGs can be handled by a reduction to Energy Parity Games (EPGs) for ǫ-optimal stategies ⇒ • Simpler algorithm for solving EPGS than for MPPGs

• Basis of an efficient symbolic incremental algorithm

Energy objective: Given c0 ≥ 0, c0 +

n

• j=1
• l∈Lit(ij∪oj)

v ′(l) ≥ 0, ∀n ≥ 0 (2) If Player O wins in the EPG, then Player O has a winning strategy with initial credit (n − 1) · W , where

• n = number of state of the EPG
• W = largest absolute cost w.r.t. v

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Reduction to safety games with costs

LTL + v : Lit(P) → Z ↓ 2O(n) Universal coB¨ uchi Energy Word Automaton ↓ O(1) Universal K-coB¨ uchi Energy Word Automaton ↓ 2O(m2) Deterministic K-coB¨ uchi Energy Word Automaton ⇒ Solve a safety game det(A, K, C) with costs s.t.:

• States are pairs (f , c)
• Upper bound C = (n − 1) · W on credit values i.e. c ∈ [−1, 0, . . . , C]
• Bad states: (f , c) s.t. either ∃q: f (q) > K or c < 0

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Properties of the safety game with costs

Property 1: partial order on its set of states (f1, c1) MP (f2, c2) ⇔ f1 f2 and c1 ≥ c2 Intuition: If we can avoid bad states from (f2, c2), we can also avoid them from (f1, c1) ⇒ Symbolic algorithm based on antichains Property 2: incremental algorithm

• If det(A, k1, c1) is realizable, then det(A, k2, c2) is realizable,

∀k2≥k1 and ∀c2≥c1 ⇒ Incremental algorithm on the values of k = 0, 1, 2, . . . , K and c = 0, 1, 2, . . . , C

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Back to motivating example

Winning strategy for Player O to ensure a mean-payoff value of −1.2:

(g2, w1) (g2, w1) (g2, w1) (g2, w1) (g1, w2) T T T T T

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Extension to multi-dimensional weights

Let v : Lit(P) → Zm and c ∈ Qm, with m ≥ 1 Theorem: The multi-dimensional LTLMP synthesis problem under finite memory is in co-N2ExpTime. ⇒ Reduction to a Multi-Energy Game [CDHR10, CRR12] In practice:

• Reduction to safety games with multi-dimensional costs
• States are pairs (f , c) where c ∈ [−1, 0, . . . , C]m
• Incremental symbolic antichain-based algorithm

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Conclusion and future work

Implementation in Acacia+

• Try it online at http://lit2.ulb.ac.be/acaciaplus/

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Conclusion and future work

Implementation in Acacia+

• Try it online at http://lit2.ulb.ac.be/acaciaplus/

Future work: improvement on the quality of output strategies

• Optimal against the most challenging opponent
• Not always optimal against an opponent playing randomly
• Compute the set of worst-case winning strategies
• Add a probability distribution on the actions of Player I MDP
• Solve this MDP symbolically with antichains

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

Thank you! Questions?

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

References I

Aaron Bohy, V´ eronique Bruy` ere, Emmanuel Filiot, Naiyong Jin, and Jean-Fran¸ cois Raskin. Acacia+, a tool for LTL synthesis. In P. Madhusudan and Sanjit A. Seshia, editors, CAV, volume 7358 of Lecture Notes in Computer Science, pages 652–657. Springer, 2012. Patricia Bouyer, Nicolas Markey, J¨

• rg Olschewski, and Michael Ummels.

Measuring permissiveness in parity games: Mean-payoff parity games revisited. In Tevfik Bultan and Pao-Ann Hsiung, editors, ATVA, volume 6996 of Lecture Notes in Computer Science, pages 135–149. Springer, 2011. Krishnendu Chatterjee and Laurent Doyen. Energy parity games. In Samson Abramsky, Cyril Gavoille, Claude Kirchner, Friedhelm Meyer auf der Heide, and Paul G. Spirakis, editors, ICALP (2), volume 6199 of Lecture Notes in Computer Science, pages 599–610. Springer, 2010. Krishnendu Chatterjee, Laurent Doyen, Thomas A. Henzinger, and Jean-Fran¸ cois Raskin. Generalized mean-payoff and energy games. In Kamal Lodaya and Meena Mahajan, editors, FSTTCS, volume 8 of LIPIcs, pages 505–516. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2010.

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

References II

Krishnendu Chatterjee, Thomas A Henzinger, and Marcin Jurdzinski. Mean-payoff parity games. In Logic in Computer Science, 2005. LICS 2005. Proceedings. 20th Annual IEEE Symposium on, pages 178–187. IEEE, 2005. Krishnendu Chatterjee, Mickael Randour, and Jean-Fran¸ cois Raskin. Strategy synthesis for multi-dimensional quantitative objectives. In Maciej Koutny and Irek Ulidowski, editors, CONCUR, volume 7454 of Lecture Notes in Computer Science, pages 115–131. Springer, 2012. R¨ udiger Ehlers. Symbolic bounded synthesis. In Tayssir Touili, Byron Cook, and Paul Jackson, editors, CAV, volume 6174 of Lecture Notes in Computer Science, pages 365–379. Springer, 2010. Emmanuel Filiot, Naiyong Jin, and Jean-Fran¸ cois Raskin. An antichain algorithm for ltl realizability. In Ahmed Bouajjani and Oded Maler, editors, CAV, volume 5643 of Lecture Notes in Computer Science, pages 263–277. Springer, 2009.

Introduction LTL synthesis LTL synthesis with MP objectives Conclusion and future work

References III

• B. Jobstmann and R. Bloem.

Optimizations for LTL synthesis. In Proceedings of the 6th International Conference on Formal Methods in Computer Aided Design (FMCAD), pages 117–124. IEEE Computer Society, 2006. Orna Kupferman and Moshe Y Vardi. Safraless decision procedures. In Foundations of Computer Science, 2005. FOCS 2005. 46th Annual IEEE Symposium on, pages 531–540. IEEE, 2005.

• A. Pnueli and R. Rosner.

On the synthesis of a reactive module. In Proceedings of the ACM Symposium on Principles of Programming Languages (POPL), pages 179–190. ACM, ACM Press, 1989. Sven Schewe and Bernd Finkbeiner. Bounded synthesis. In Kedar S. Namjoshi, Tomohiro Yoneda, Teruo Higashino, and Yoshio Okamura, editors, ATVA, volume 4762 of Lecture Notes in Computer Science, pages 474–488. Springer, 2007.