Satisfiability Bounds for -Regular Properties in Bounded-Parameter - PowerPoint PPT Presentation

Satisfiability Bounds for ω-Regular Properties in Bounded-Parameter Markov Decision Processes M. Weininger T. Meggendorfer J. Kretinsky

Satisfiability Bounds for ω-Regular Properties in Bounded-Parameter Markov Decision Processes M. Weininger T. Meggendorfer J. Kretinsky

Bounded-Parameter Markov Decision Process Hills 0.8 0.2 Broken Probe Station 0.5 0.5 Valley

Bounded-Parameter Markov Decision Process Hills [0.1, 1] [0, 0.5] Broken Probe Station Station [0.1, 0.5] [0.2, 0.8] Valley

Bounded-Parameter Markov Decision Process Hills Hills [0.1, 1] 0.8 [0, 0.5] 0.2 Broken Broken Probe Station Station Probe Station Station [0.1, 0.5] [0.2, 0.8] 0.5 0.5 Valley Valley

Bounded-Parameter Markov Decision Process Hills Hills [0.1, 1] 1 [0, 0.5] 0 Broken Broken Probe Station Station Probe Station Station [0.1, 0.5] [0.2, 0.8] 0.5 0.5 Valley Valley

Satisfiability bounds for ω-Regular Properties Hills [0.1, 1] [0, 0.5] Broken Probe Station Station [0.1, 0.5] [0.2, 0.8] Valley

Satisfiability bounds for ω-Regular Properties “Eventually take a probe” Hills F (Probe) “Always take a probe in the future and [0.1, 1] bring it to the station” [0, 0.5] Broken G (F (Probe) ∧ Probe ⇒ X (Station)) Probe Station Station [0.1, 0.5] [0.2, 0.8] Valley

Satisfiability bounds for ω-Regular Properties “Eventually take a probe” Hills F (Probe) “Always take a probe in the future and [0.1, 1] bring it to the station” [0, 0.5] Broken G (F (Probe) ∧ Probe ⇒ X (Station)) Probe Station Station Find optimal controller [0.1, 0.5] [0.2, 0.8] 𝓜 ≤ ℙ (System ⊨ Property) ≤ 𝓥 Valley

Semantics of the intervals 𝓜 : Adversarial Environment 𝓥 : Design choice slideshare.net/jefffarias9 letsgetsciencey.com/best-microscope-for-kids/

Semantics of the intervals 𝓜 : Adversarial Environment 𝓥 : Design choice slideshare.net/jefffarias9 letsgetsciencey.com/best-microscope-for-kids/

Resolving intervals in ”Design choice” setting Hills [0.1, 1] [0, 0.5] Broken Station

Resolving intervals in ”Design choice” setting Hills Hills [0.1, 1] [0, 0.5] Design Broken Broken Choice Station Station Station

Resolving intervals in ”Design choice” setting Hills Hills 1 [0.1, 1] [0, 0.5] Design Broken Broken Choice Station Station Station

Resolving intervals in ”Design choice” setting Hills Hills 0.5 0.5 1 [0.1, 1] [0, 0.5] Design Broken Broken Choice Station Station Station

Resolving intervals in ”Design choice” setting 0.6983 Hills Hills 0.5 0.3127 0.5 1 [0.1, 1] [0, 0.5] Design Broken Broken Choice Station Station Station

Resolving intervals in ”Design choice” setting 0.6983 ... Hills Hills 0.5 0.3127 0.5 1 [0.1, 1] [0, 0.5] Design Broken Broken Choice Station Station Station

Resolving intervals in ”Design choice” setting Hills Hills 0.5 0.5 1 [0.1, 1] [0, 0.5] Design Broken Broken Choice Station Station Station

Resolving intervals in ”Design choice” setting Hills Hills Basic Feasible 0.5 Solutions [HM18] 0.5 1 [0.1, 1] [0, 0.5] Design Broken Broken Choice Station Station Station

Resolving intervals in ”Design choice” setting Hills Hills Basic Feasible 0.5 Solutions [HM18] 0.5 1 [0.1, 1] [0, 0.5] Design Broken Broken Choice Solving MDP e.g. [Put94] yields controller and probability Station Station Station

Idea in short 1. New state for every action 2. Basic feasible solutions as its actions 3. Solve MDP bpMDP Design choice MDP

Idea in short 1. New state for every action (other player!) 2. Basic feasible solutions as its actions 3. Solve MDP Stochastic Game bpMDP A d v e r s Design a r i a l choice MDP SG

Idea in short 1. New state for every action (other player!) 2. Basic feasible solutions as its actions 3. Solve MDP Stochastic Game Solving SG e.g. [CH06] yields controller and probability bpMDP A d v e r s Design a r i a l choice MDP SG

The bigger picture bpMDP A d v e r Design s a r i a l choice MDP SG

The bigger picture IMC bpMDP A d v e r Design s a r i a l choice MDP SG

The bigger picture IMC bpMDP A d A v d e v r Design e s r a s [ r H a i a M r l i a choice 1 l 8 ] E X P MDP SG

The bigger picture IMC bpMDP A d A v d e v r Design e s r a s [ EXP r H a E i a M r X l i a choice P 1 l 8 ] E X P MDP SG

The bigger picture IMC bpMDP A d A v d e Design [DC18] v r Design e s r a s [ EXP r H a E i a choice POL M r X l i a choice P 1 l 8 ] E X P MC MDP SG

The bigger picture IMC bpMDP A d A v d e Design [DC18] v r Design e s r a s [ POL r H a E i a choice POL M r X l i a choice P 1 l 8 ] E X P MC MDP SG

The bigger picture IMC bpMDP A d A v d e Design [DC18] v r Design e s r a s [ POL r H a E i a choice POL M r X l i a choice P 1 l 8 [ ] D E C X 1 P 8 ] P O L MC MDP SG

The bigger picture IMC bpMDP bpSG A d A v d e Design [DC18] v r Design e s r a s [ POL Design r H a E i a choice POL M r X l i a Adversarial choice P 1 l 8 [ choice ] D E C X 1 P 8 ] P O L MC MDP SG

Future work - Practical implementation (using our previous work)

Future work - Practical implementation (using our previous work) - Other imprecisions in system model, e.g. parametrized MDPs - Multiple objectives