SLIDE 1
Approximating Values of Generalized-Reachability Stochastic Games - - PowerPoint PPT Presentation
Approximating Values of Generalized-Reachability Stochastic Games - - PowerPoint PPT Presentation
Approximating Values of Generalized-Reachability Stochastic Games Maximilian Weininger joint work with Pranav Ashok, Krishnendu Chatterjee, Jan Kretnsk, Tobias Winkler HIGHLIGHTS 2020 (Paper appeared at LICS 2020) Model The problem The
SLIDE 2
SLIDE 3
The problem
SLIDE 4
The problem
SLIDE 5
The problem
SLIDE 6
The problem
SLIDE 7
The problem
SLIDE 8
The problem
Want: Pareto frontier
SLIDE 9
The problem
Want: Pareto frontier How: Value iteration from below [CFK+13]
[CFK+13] Chen, T., Forejt, V., Kwiatkowska, M., Simaitis, A., & Wiltsche, C. On stochastic games with multiple objectives. MFCS 2013.
SLIDE 10
The problem
Want: Pareto frontier How: Value iteration from below [CFK+13]
[CFK+13] Chen, T., Forejt, V., Kwiatkowska, M., Simaitis, A., & Wiltsche, C. On stochastic games with multiple objectives. MFCS 2013.
SLIDE 11
The problem
Want: Pareto frontier How: Value iteration from below [CFK+13]
[CFK+13] Chen, T., Forejt, V., Kwiatkowska, M., Simaitis, A., & Wiltsche, C. On stochastic games with multiple objectives. MFCS 2013.
SLIDE 12
The problem
Want: Pareto frontier How: Value iteration from below [CFK+13]
[CFK+13] Chen, T., Forejt, V., Kwiatkowska, M., Simaitis, A., & Wiltsche, C. On stochastic games with multiple objectives. MFCS 2013.
SLIDE 13
The problem
Want: Pareto frontier How: Value iteration from below [CFK+13]
[CFK+13] Chen, T., Forejt, V., Kwiatkowska, M., Simaitis, A., & Wiltsche, C. On stochastic games with multiple objectives. MFCS 2013.
SLIDE 14
The problem
Want: Pareto frontier How: Value iteration from below [CFK+13]
[CFK+13] Chen, T., Forejt, V., Kwiatkowska, M., Simaitis, A., & Wiltsche, C. On stochastic games with multiple objectives. MFCS 2013.
SLIDE 15
The problem
Want: Pareto frontier How: Value iteration from below [CFK+13] Problem: When to stop?
[CFK+13] Chen, T., Forejt, V., Kwiatkowska, M., Simaitis, A., & Wiltsche, C. On stochastic games with multiple objectives. MFCS 2013.
SLIDE 16
The problem
Want: Pareto frontier How: Value iteration from below [CFK+13] Problem: When to stop? Solution: Convergent over-approximation
SLIDE 17
Want: Pareto frontier How: Value iteration from below [CFK+13] Problem: When to stop? Solution: Convergent over-approximation
The problem
SLIDE 18
Want: Pareto frontier How: Value iteration from below [CFK+13] Problem: When to stop? Solution: Convergent over-approximation
The problem
SLIDE 19
Want: Pareto frontier How: Value iteration from below [CFK+13] Problem: When to stop? Solution: Convergent over-approximation
The problem
SLIDE 20
Want: Pareto frontier How: Value iteration from below [CFK+13] Problem: When to stop? Solution: Convergent over-approximation
The problem
SLIDE 21
Want: Pareto frontier How: Value iteration from below [CFK+13] Problem: When to stop? Solution: Convergent over-approximation
The problem
SLIDE 22
Want: Pareto frontier How: Value iteration from below [CFK+13] Problem: When to stop? Solution: Convergent over-approximation
Approximate values of generalized-reachability stochastic games for arbitrarily small precision.
The problem
SLIDE 23
Our contribution
Over-approximation need not converge (multiple fixpoints)
SLIDE 24
Over-approximation need not converge (multiple fixpoints)
- Consider single directions
- Apply single-dimensional solution
Our contribution
SLIDE 25
Over-approximation need not converge (multiple fixpoints)
- Consider single directions
- Apply single-dimensional solution
- Group directions into regions
Our contribution
SLIDE 26
Context
Single-dim SG Multi-dim MDP Multi-dim SG Computational complexity
NP ∩ coNP [Con92] PSPACE-complete [RRS15]
Strategy complexity
[Con92] Condon, A. (1992). The complexity of stochastic games [RRS15] Randour, M., Raskin, J. F., & Sankur, O. Percentile queries in multi-dimensional Markov decision processes. CAV 2015.
SLIDE 27
Context
Single-dim SG Multi-dim MDP Multi-dim SG Computational complexity
NP ∩ coNP [Con92] PSPACE-complete [RRS15] Decidability open
Strategy complexity
[Con92] Condon, A. (1992). The complexity of stochastic games [RRS15] Randour, M., Raskin, J. F., & Sankur, O. Percentile queries in multi-dimensional Markov decision processes. CAV 2015.
SLIDE 28
Context
Single-dim SG Multi-dim MDP Multi-dim SG Computational complexity
NP ∩ coNP [Con92] PSPACE-complete [RRS15] Decidability open
Strategy complexity
Memoryless deterministic [Con92] Randomized memoryless (absorbing) [EKVY07]; Finite mem. in general [RRS15]
[Con92] Condon, A. (1992). The complexity of stochastic games [RRS15] Randour, M., Raskin, J. F., & Sankur, O. Percentile queries in multi-dimensional Markov decision processes. CAV 2015. [EKVY07] Etessami, K., Kwiatkowska, M., Vardi, M. Y., & Yannakakis, M. Multi-objective model checking of Markov decision processes. TACAS 2007.
SLIDE 29
Context
Single-dim SG Multi-dim MDP Multi-dim SG Computational complexity
NP ∩ coNP [Con92] PSPACE-complete [RRS15] Decidability open
Strategy complexity
Memoryless deterministic [Con92] Randomized memoryless (absorbing) [EKVY07]; Finite mem. in general [RRS15]
- Inf. mem. (absorbing)
[CFK+13]
[Con92] Condon, A. (1992). The complexity of stochastic games [RRS15] Randour, M., Raskin, J. F., & Sankur, O. Percentile queries in multi-dimensional Markov decision processes. CAV 2015. [EKVY07] Etessami, K., Kwiatkowska, M., Vardi, M. Y., & Yannakakis, M. Multi-objective model checking of Markov decision processes. TACAS 2007. [CFK+13] Chen, T., Forejt, V., Kwiatkowska, M., Simaitis, A., & Wiltsche, C. On stochastic games with multiple objectives. MFCS 2013.
SLIDE 30
Context
Over-approximation need not converge (multiple fixpoints)
Single-dim SG Multi-dim MDP Multi-dim SG Computational complexity
NP ∩ coNP [Con92] PSPACE-complete [RRS15] Decidability open
Strategy complexity
Memoryless deterministic [Con92] Randomized memoryless (absorbing) [EKVY07]; Finite mem. in general [RRS15]
- Inf. mem. (absorbing)
[CFK+13]