Satisfiability Bounds for -Regular Properties in Bounded-Parameter - - PowerPoint PPT Presentation

▶

Dec 05, 2023 132 likes •469 views

Satisfiability Bounds for -Regular Properties in Bounded-Parameter Markov Decision Processes M. Weininger T. Meggendorfer J. Kretinsky Satisfiability Bounds for -Regular Properties in Bounded-Parameter Markov Decision Processes M.

SLIDE 1

Satisfiability Bounds for ω-Regular Properties in Bounded-Parameter Markov Decision Processes

M. Weininger T. Meggendorfer J. Kretinsky

SLIDE 2

Satisfiability Bounds for ω-Regular Properties in Bounded-Parameter Markov Decision Processes

M. Weininger T. Meggendorfer J. Kretinsky

SLIDE 3

Bounded-Parameter Markov Decision Process

Station Broken Valley Hills Probe

0.8 0.2 0.5 0.5

SLIDE 4

Bounded-Parameter Markov Decision Process

Station Broken Valley Hills Probe

[0.1, 1] [0, 0.5] [0.1, 0.5] [0.2, 0.8]

Station

SLIDE 5

Bounded-Parameter Markov Decision Process

Station Broken Valley Hills Probe

[0.1, 1] [0, 0.5] [0.2, 0.8]

Station Broken Valley Hills Probe

0.8 0.2 0.5 0.5 [0.1, 0.5]

Station Station

SLIDE 6

Bounded-Parameter Markov Decision Process

Station Broken Valley Hills Probe

[0.1, 1] [0, 0.5] [0.2, 0.8]

Station Broken Valley Hills Probe

1 0.5 0.5 [0.1, 0.5]

Station Station

SLIDE 7

Satisfiability bounds for ω-Regular Properties

Station Broken Valley Hills Probe

[0.1, 1] [0, 0.5] [0.2, 0.8] [0.1, 0.5]

Station

SLIDE 8

Satisfiability bounds for ω-Regular Properties

Station Broken Valley Hills Probe

[0.1, 1] [0, 0.5] [0.2, 0.8]

“Eventually take a probe” F (Probe) “Always take a probe in the future and bring it to the station” G (F (Probe) ∧ Probe ⇒ X (Station))

[0.1, 0.5]

Station

SLIDE 9

Satisfiability bounds for ω-Regular Properties

Station Broken Valley Hills Probe

[0.1, 1] [0, 0.5] [0.2, 0.8]

Find optimal controller 𝓜 ≤ ℙ(System ⊨ Property) ≤ 𝓥 “Eventually take a probe” F (Probe) “Always take a probe in the future and bring it to the station” G (F (Probe) ∧ Probe ⇒ X (Station))

[0.1, 0.5]

Station

SLIDE 10

Semantics of the intervals

𝓜: Adversarial Environment 𝓥: Design choice

slideshare.net/jefffarias9 letsgetsciencey.com/best-microscope-for-kids/

SLIDE 11

Semantics of the intervals

𝓜: Adversarial Environment 𝓥: Design choice

slideshare.net/jefffarias9 letsgetsciencey.com/best-microscope-for-kids/

SLIDE 12

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station

SLIDE 13

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station Broken Hills Design Choice Station Station

SLIDE 14

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station Broken Hills Design Choice Station Station

SLIDE 15

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station Broken Hills Design Choice

0.5 0.5

Station Station

SLIDE 16

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station Broken Hills Design Choice

0.5 0.5 0.3127 0.6983

Station Station

SLIDE 17

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station Broken Hills Design Choice

0.5 0.5 0.3127 0.6983

Station Station

...

SLIDE 18

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station Broken Hills Design Choice

0.5 0.5 1

Station Station

SLIDE 19

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station Broken Hills Design Choice

0.5 0.5 1

Station Station

Basic Feasible Solutions [HM18]

SLIDE 20

Resolving intervals in ”Design choice” setting

Broken Hills

[0.1, 1] [0, 0.5]

Station Broken Hills Design Choice

0.5 0.5 1

Station Station

Basic Feasible Solutions [HM18] Solving MDP e.g. [Put94] yields controller and probability

SLIDE 21

Idea in short

1. New state for every action 2. Basic feasible solutions as its actions 3. Solve MDP

bpMDP MDP

Design choice

SLIDE 22

Idea in short

1. New state for every action (other player!) 2. Basic feasible solutions as its actions 3. Solve MDP Stochastic Game

bpMDP MDP SG

Design choice A d v e r s a r i a l

SLIDE 23

Idea in short

1. New state for every action (other player!) 2. Basic feasible solutions as its actions 3. Solve MDP Stochastic Game

bpMDP MDP SG

Design choice A d v e r s a r i a l

Solving SG e.g. [CH06] yields controller and probability

SLIDE 24

The bigger picture

bpMDP MDP SG

Design choice A d v e r s a r i a l

SLIDE 25

The bigger picture

bpMDP MDP IMC SG

Design choice A d v e r s a r i a l

SLIDE 26

The bigger picture

bpMDP MDP IMC SG

A d v e r s a r i a l Design choice A d v e r s a r i a l [ H M 1 8 ] E X P

SLIDE 27

The bigger picture

bpMDP MDP IMC SG

A d v e r s a r i a l Design choice A d v e r s a r i a l [ H M 1 8 ] E X P EXP E X P

SLIDE 28

The bigger picture

bpMDP MDP IMC SG

A d v e r s a r i a l Design choice A d v e r s a r i a l [ H M 1 8 ] E X P

MC

Design choice EXP [DC18] POL E X P

SLIDE 29

The bigger picture

bpMDP MDP IMC SG

A d v e r s a r i a l Design choice A d v e r s a r i a l [ H M 1 8 ] E X P

MC

Design choice POL [DC18] POL E X P

SLIDE 30

The bigger picture

bpMDP MDP IMC SG

A d v e r s a r i a l Design choice A d v e r s a r i a l [ H M 1 8 ] E X P

MC

Design choice POL [DC18] POL E X P [ D C 1 8 ] P O L

SLIDE 31

The bigger picture

bpMDP MDP IMC SG

A d v e r s a r i a l Design choice A d v e r s a r i a l [ H M 1 8 ] E X P

MC

Design choice POL [DC18] POL E X P [ D C 1 8 ] P O L

bpSG

Adversarial Design choice

SLIDE 32

Future work

Practical implementation (using our previous work)

SLIDE 33

Future work

Practical implementation (using our previous work)
Other imprecisions in system model, e.g. parametrized MDPs
Multiple objectives