Formal verification of complex systems: model-based and data-driven - - PowerPoint PPT Presentation

Formal verification of complex systems: model-based and data-driven methods Alessandro Abate

Department of Computer Science, University of Oxford

Alan Turing Institute - Jan 12, 2018

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 1 /20

Automated formal verification: successes and frontiers

automated, sound, formal

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 2 /20

Automated formal verification: successes and frontiers

automated, sound, formal industrial impact in verification of protocols, hardware circuits, and software

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 2 /20

Automated formal verification: successes and frontiers

automated, sound, formal industrial impact in verification of protocols, hardware circuits, and software asserts properties over given model of a system scalable and useful on “unsophisticated” models

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 2 /20

Automated formal verification: pushing the envelope

verification of physical systems (cyber-physical systems)

dynamical models with uncertainty, noise (for CPS) bridging the gap between data and models principled integration of learning and verification

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 3 /20

Building automation systems: an exemplar of CPS

cyber-physical systems: integration of physical/analogue with cyber/digital building automation systems as a CPS exemplar

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 4 /20

Building automation systems: an exemplar of CPS

cyber-physical systems: integration of physical/analogue with cyber/digital building automation systems as a CPS exemplar smart energy initiatives at Oxford CS

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 4 /20

Building automation systems - a CPS exemplar

Building automation system setup in rooms 478/9 at Oxford CS advanced modelling for smart buildings application: certifiable energy management

1

control of temperature, humidity, CO2

2

model-based predictive maintenance of devices

3

fault-tolerant control

4

demand-response over smart grids

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 5 /20

Building automation systems - a CPS exemplar

Building automation system setup in rooms 478/9 at Oxford CS advanced modelling for smart buildings application: certifiable energy management

1

control of temperature, humidity, CO2

2

model-based predictive maintenance of devices

3

fault-tolerant control

4

demand-response over smart grids

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 5 /20

Building automation systems - a CPS exemplar

Building automation system setup in rooms 478/9 at Oxford CS advanced modelling for smart buildings application: certifiable energy management

1

control of temperature, humidity, CO2

2

model-based predictive maintenance of devices

3

fault-tolerant control

4

demand-response over smart grids

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 5 /20

Building automation systems - problem setup

model CO2 dynamics, under the effect of

1

• ccupants: room full (F)/empty (E)

2

window: open (O)/closed (C)

3

air circulation: ON/OFF

(F,C) (F,O) (E,C) (E,O)

xk+1 = xk + ∆ V

• −1ONmxk + µ{O,C}(Cout − xk)
• + 1FCocc

x - zone CO2 level ∆ - sampling time V - zone volume m - air inflow (when ON) µO - air exchange with outside (when O) µC - air leakage with outside (when C) Cout - outside CO2 level Cocc - CO2 by occupants (when F)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 6 /20

Building automation systems - problem setup

model CO2 dynamics, under the effect of

1

• ccupants: room full (F)/empty (E)

2

window: open (O)/closed (C)

3

air circulation: ON/OFF

(F,C) (F,O) (E,C) (E,O)

xk+1 = xk + ∆ V

• −1ONmxk + µ{O,C}(Cout − xk)
• + 1FCocc

Parameter Value ∆ 15 min V 288 m3 m 0.25 m3/min µO 0.1667 m3/min µC 0.01 m3/min Cout 375 ppm Cocc 0.4 ppm/min

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 6 /20

Building automation systems - problem setup

model CO2 dynamics, under the effect of

1

• ccupants: room empty E

2

window: closed C

3

air circulation: ON

(F,C) (F,O) (E,C) (E,O)

xk+1 = xk + ∆ V (−mxk + µC(Cout − xk)) + 0 · Cocc

0 12 0 12 0 12 0 12 0 100 200 300 400 500 600

CO2 levels

0 12 0 12 0 12 0 12 0 1

Fan (on, off)

0 12 0 12 0 12 0 12 0 1

Occupancy (occupied, empty)

0 12 0 12 0 12 0 12 0 1

Windows (open, closed)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 6 /20

Building automation systems - problem setup

model CO2 dynamics, under the effect of

1

• ccupants: room full F

2

window: closed C

3

air circulation: ON

(F,C) (F,O) (E,C) (E,O)

xk+1 = xk + ∆ V (−mxk + µC(Cout − xk)) + Cocc

0 12 0 12 0 12 0 12 0 100 200 300 400 500 600

CO2 levels

0 12 0 12 0 12 0 12 0 1

Fan (on, off)

0 12 0 12 0 12 0 12 0 1

Occupancy (occupied, empty)

0 12 0 12 0 12 0 12 0 1

Windows (open, closed)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 6 /20

Building automation systems - problem setup

model CO2 dynamics, under the effect of

1

• ccupants: room full F

2

window: open O

3

air circulation: ON

(F,C) (F,O) (E,C) (E,O)

xk+1 = xk + ∆ V (−mxk + µO(Cout − xk)) + Cocc

0 12 0 12 0 12 0 12 0 100 200 300 400 500 600

CO2 levels

0 12 0 12 0 12 0 12 0 1

Fan (on, off)

0 12 0 12 0 12 0 12 0 1

Occupancy (occupied, empty)

0 12 0 12 0 12 0 12 0 1

Windows (open, closed)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 6 /20

Building automation systems - problem setup

model CO2 dynamics, under the effect of

1

• ccupants: room empty E

2

window: closed C

3

air circulation: ON

(F,C) (F,O) (E,C) (E,O)

xk+1 = xk + ∆ V (−mxk + µO(Cout − xk))

0 12 0 12 0 12 0 12 0 100 200 300 400 500 600

CO2 levels

0 12 0 12 0 12 0 12 0 1

Fan (on, off)

0 12 0 12 0 12 0 12 0 1

Occupancy (occupied, empty)

0 12 0 12 0 12 0 12 0 1

Windows (open, closed)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 6 /20

Building automation systems - problem setup

model CO2 dynamics, under the effect of

1

• ccupants: room full (F)/empty (E)

2

window: open (O)/closed (C)

3

air circulation: ON

(F,C) (F,O) (E,C) (E,O)

model with hybrid dynamics

0 12 0 12 0 12 0 12 0 100 200 300 400 500 600

CO2 levels

0 12 0 12 0 12 0 12 0 1

Fan (on, off)

0 12 0 12 0 12 0 12 0 1

Occupancy (occupied, empty)

0 12 0 12 0 12 0 12 0 1

Windows (open, closed)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 6 /20

Building automation systems - problem setup

model CO2 dynamics, under the effect of

1

• ccupants: room full (F)/empty (E)

2

window: open (O)/closed (C)

3

air circulation: OFF

(F,C) (F,O) (E,C) (E,O)

model with hybrid dynamics

0 12 0 12 0 12 0 12 0 600 800 1,000 1,200 1,400

CO2 levels

0 12 0 12 0 12 0 12 0 1

Fan (on, off)

0 12 0 12 0 12 0 12 0 1

Occupancy (occupied, empty)

0 12 0 12 0 12 0 12 0 1

Windows (open, closed)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 6 /20

Learning and verification: state of art and objective

inputs noise system noise

• utputs

data

data-driven analysis

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 7 /20

Learning and verification: state of art and objective

inputs noise system noise

• utputs

data model

data-driven analysis model learning (with data), and model-based verification

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 7 /20

Learning and verification: state of art and objective

inputs noise system noise

• utputs

data model

disconnect between data-driven learning and model-based verification

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 7 /20

Learning and verification: state of art and objective

inputs noise system noise

• utputs

data model

disconnect between data-driven learning and model-based verification principled integration of learning and verification

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 7 /20

Overview of method

parameter synthesis property φ model pMC Bayesian inference

• ver parameters

data from system S C = P(S | = φ) confidence computation Θφ D p(θ|D)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 8 /20

Parametric Markov chains

parameter synthesis property φ model pMC Bayesian inference

• ver parameters

data from system S C = P(S | = φ) confidence computation Θφ D p(θ|D)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 9 /20

Parametric Markov chains

G = (Θ, S, Tθ, →, AP, L) S – set of states Tθ – mapping S × S → [0, 1] expressed in terms of θ ∈ Θ Θ – set of all possible valuations of θ, vector of parameters → – starting states

(F,C) (F,O) (E,C) (E,O) . 7 5 1 − 0.75 − θ1 θ1 0.25 1 − 0.25 − θ1 θ2 θ2 1 − θ2 1 − θ2 θ1

Θ = [0, 0.25] × [0, 1]

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 9 /20

Parametric Markov chains

G = (Θ, S, Tθ, →, AP, L) S – set of states Tθ – mapping S × S → [0, 1] expressed in terms of θ ∈ Θ Θ – set of all possible valuations of θ, vector of parameters → – starting states L – labelling function, mapping states into 2AP, AP alphabet denote by M(θ) ∈ G a model parameterised by θ ∈ Θ

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 9 /20

parameter synthesis property φ model pMC Bayesian inference

• ver parameters

data from system S C = P(S | = φ) confidence computation Θφ D p(θ|D)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 10 /20

property φ specified in PCTL, e.g. φ = P≥0.99(≤20 safe), φ = P>0.5(safe U reach), safe, reach ∈ AP probabilistic model checking PCTL properties over Markov chains

input: Markov chain (S, T), PCTL formula φ

• utput: Sat(φ) = {z ∈ S : z |

= φ}

tools: PRISM, STORM, . . .

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 10 /20

← set Θφ

Parameter synthesis

property φ specified in PCTL, e.g. φ = P≥0.99(≤20 safe), φ = P>0.5(safe U reach), safe, reach ∈ AP classify models in Θ according to property of interest φ, that is synthesise parameters θ ∈ Θ s.t. M(θ) satisfies φ: Θφ = {θ ∈ Θ : M(θ) | = φ} ⊆ Θ

1 1

parameter set Θ

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 10 /20

parameter synthesis property φ model pMC Bayesian inference

• ver parameters

data from system S C = P(S | = φ) confidence computation Θφ D p(θ|D)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 11 /20

Bayesian inference

p(θj | D) = P(D | θj)p(θj) P(D) = ∏s′∈S Tθ(sj, s′)

Ds′

sj p(θj)

P(Dsj) D – overall data gathered (traces) Dsj – traces crossing state sj, where θj = θsj p(θj) – prior distribution ∏s′∈S Tθ(sj, s′)

Ds′

sj – likelihood, multinomial distribution at state sj

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 11 /20

Bayesian inference

p(θj | D) = P(D | θj)p(θj) P(D) = ∏s′∈S Tθ(sj, s′)

Ds′

sj p(θj)

P(Dsj) D – overall data gathered (traces) Dsj – traces crossing state sj, where θj = θsj select as conjugate prior the Dirichlet distribution p(θj) = Dir(θj | α) ∝ θα1−1

j

(1 − θj)α2−1 for pair (θj, 1 − θj), with α = (α1, α2) hyperparameters

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 11 /20

Bayesian inference

p(θj | D) = P(D | θj)p(θj) P(D) = ∏s′∈S Tθ(sj, s′)

Ds′

sj p(θj)

P(Dsj) D – overall data gathered (traces) Dsj – traces crossing state sj, where θj = θsj under Dirichlet prior, posterior update is analytic p(θj | D) ∝ θ

D

s′ 1 sj

j

(1 − θj)

D

s′ 2 sj θα1−1

j

(1 − θj)α2−1 and obtained updating hyperparameters of Dirichlet distribution, as p(θj|D) = Dir(θj | Dsj + α)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 11 /20

parameter synthesis property φ model pMC Bayesian inference

• ver parameters

data from system S C = P(S | = φ) confidence computation Θφ D p(θ|D)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 12 /20

Confidence computation

1 1

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 12 /20

Confidence computation

1 1

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 12 /20

Confidence computation

compute confidence C on whether system S satisfies property φ as C = P(S | = φ | D) =

• Θφ

p(θ | D)dθ

1

x x x x x x x x x x x x x x x x x x x

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 12 /20

Case study: setup

goal: benchmark against statistical model checking (SMC) pMC model:

(F,C) (F,O) (E,C) (E,O) . 7 5 1 − 0.75 − θ1 θ1 0.25 1 − 0.25 − θ1 θ2 θ2 1 − θ2 1 − θ2 θ1

specification: φ = P>0.3(≤20 ¬(E, O))

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 13 /20

Case study: setup

goal: benchmark against statistical model checking (SMC) pMC model:

(F,C) (F,O) (E,C) (E,O) . 7 5 1 − 0.75 − θ1 θ1 0.25 1 − 0.25 − θ1 θ2 θ2 1 − θ2 1 − θ2 θ1

1 · 10−2 2 · 10−2 3 · 10−2 4 · 10−2 5 · 10−2 6 · 10−2 7 · 10−2 8 · 10−2 9 · 10−2 0.1 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.2 0.21 0.22 0.23 0.24 0.25 5 · 10−2 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

θ1 θ2

specification: φ = P>0.3(≤20 ¬(E, O)) for selected pMC and property, synthesis yields Θφ (yellow set)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 13 /20

Case study: experiments

data: state trajectories of different length SMC this work attains confidence closer to “true” value than SMC extracts information from data more efficiently is more robust with limited data

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 13 /20

Parametric Markov decision processes

G = (Θ, S, A, Tθ, →, AP, L) Θ, S, →, L – as before A – set of actions Tθ – mapping S × A × S → [0, 1] expressed in terms of θ ∈ Θ

(E, O) (F, O) (F, C) (E, C)

1 − θ2 1 − θ2 (1 − 0.75 −θ1) (1 − 0.25 −θ1) 1 − 0.35 1 − 0.15 0.35 0.75 0.15 0.25 θ1 θ1 θ2 θ2 fon fon foff foff Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 14 /20

Dual role of actions in pMDP

actions can be employed to shape set Θφ

1 1

shape set Θφ actions can be chosen to affect confidence level C

1

x x x x x x x x x x x x x x x x x x x

integral = confidence level

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 15 /20

Dual role of actions in pMDP

actions can be employed to shape set Θφ

1 1

shape set Θφ actions can be chosen to affect confidence level C

1

x x x x x x x x x x x x x x x x x x x

integral = confidence level

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 15 /20

Dual role of actions in pMDP

actions can be employed to shape set Θφ

1 1

shape set Θφ actions can be chosen to affect confidence level C

1

integral → confidence level

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 15 /20

Dual role of actions in pMDP

actions can be employed to shape set Θφ

1 1

shape set Θφ actions can be chosen to affect confidence level C

1

integral → confidence level reminiscent of exploration/exploitation tradeoff in RL

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 15 /20

Overview of method

parameter synthesis property φ model pMC Bayesian inference

• ver parameters

data from system S C = P(S | = φ) confidence computation Θφ D p(θ|D)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 16 /20

Overview of method

parameter synthesis property φ model pMDP strategy synthesis generate data from system S Bayesian inference

• ver parameters

confidence computation C = P(S | = φ) Θφ π D p(θ|D) p(θ|D)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 16 /20

Strategy synthesis for experiment design

design experiments to affect confidence calculation max {P(S | = φ | D), P(S | = φ | D)}

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 17 /20

Strategy synthesis for experiment design

design experiments to affect confidence calculation max {P(S | = φ | D), P(S | = φ | D)} expected confidence gain at state-action (s, α) (and corresp. parameter) Cs,α =

• Θφ ∏

θi∈θ

p(θi | Es,α(Di))dθ

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 17 /20

Strategy synthesis for experiment design

design experiments to affect confidence calculation max {P(S | = φ | D), P(S | = φ | D)} expected confidence gain at state-action (s, α) (and corresp. parameter) Cs,α =

• Θφ ∏

θi∈θ

p(θi | Es,α(Di))dθ use Cs,α as a reward for (s, α) synthesise optimal strategy π for experiment design

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 17 /20

Case study: setup

goal: compare optimally synthesised policies vs. random/deterministic ones pMDP model:

(E, O) (F, O) (F, C) (E, C)

1 − θ2 1 − θ2 (1 − 0.75 −θ1) (1 − 0.25 −θ1) 1 − 0.35 1 − 0.15 0.35 0.75 0.15 0.25 θ1 θ1 θ2 θ2 fon fon foff foff

specification: φ = P>0.3(≤20 ¬(E, O))

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 18 /20

Case study: setup

goal: compare optimally synthesised policies vs. random/deterministic ones pMDP model:

(E, O) (F, O) (F, C) (E, C)

1 − θ2 1 − θ2 (1 − 0.75 −θ1) (1 − 0.25 −θ1) 1 − 0.35 1 − 0.15 0.35 0.75 0.15 0.25 θ1 θ1 θ2 θ2 fon fon foff foff

1 · 10−2 2 · 10−2 3 · 10−2 4 · 10−2 5 · 10−2 6 · 10−2 7 · 10−2 8 · 10−2 9 · 10−2 0.1 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.2 0.21 0.22 0.23 0.24 0.25 5 · 10−2 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

θ1 θ2

specification: φ = P>0.3(≤20 ¬(E, O)) for selected pMDP and given φ, Θφ is shown in yellow

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 18 /20

Case study: experiments

1 3 5 7 9 11 13 15 17 19 21 23 −5 · 10−2 5 · 10−2 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 1.05

System Parameter Confidence

1 3 5 7 9 11 13 15 17 19 21 23 −5 · 10−2 5 · 10−2 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 1.05

System Parameter Confidence

Synthesised strategy π Fully random strategy

1 2 3 4 5 6 7 8 9 10 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

Number of Traces Confidence

1 2 3 4 5 6 7 8 9 10 0.64 0.66 0.68 0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.84 0.86 0.88 0.9 0.92 0.94 0.96 0.98 1

Number of Traces Confidence

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 18 /20

Case study: experiments

1 3 5 7 9 11 13 15 17 19 21 23 −5 · 10−2 5 · 10−2 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 1.05

System Parameter Confidence

1 3 5 7 9 11 13 15 17 19 21 23 −5 · 10−2 5 · 10−2 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 1.05

System Parameter Confidence

Synthesised strategy π Deterministic strategy

1 2 3 4 5 6 7 8 9 10 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

Number of Traces Confidence

1 3 5 7 9 11 13 15 17 19 21 5 · 10−2 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

Number of Traces Confidence

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 18 /20

Extensions to other model classes

model CO2 dynamics, under the effect of

1

• ccupants: room full (F)/empty (E)

2

window: open (O)/closed (C)

3

air circulation: ON/OFF

(F,C) (F,O) (E,C) (E,O)

xk+1 = xk + ∆ V

• −1ONmxk + µ{O,C}(Cout − xk)
• + 1FCocc

x - zone CO2 level ∆ - sampling time V - zone volume m - air inflow (when ON) µO - air exchange with outside (when O) µC - air leakage with outside (when C) Cout - outside CO2 level Cocc - CO2 by occupants (when F)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 19 /20

Extensions to other model classes

parameter synthesis property φ model pLTI strategy synthesis generate data from system S Bayesian inference

• ver parameters

confidence computation C = P(S | = φ) Θφ π D p(θ|D) p(θ|D)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 19 /20

Extensions to other model classes

parametrised LTI model u(t) – input y(t) – system output ˜ y(t) – measured output e(t) – measurement noise, e(t) ∼ N(0, σ2

e )

model set G = {M(θ) | θ ∈ Θ}, where M(θ) : x(t + 1) = Ax(t) + Bu(t) y(t) = θTx(t)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 19 /20

[red text: new theory needed]

Applications of method

B + E A + C A B D

models for chemical reaction networks , with known stoichiometry, but with uncertain rates, expressed as pMDP

1

CRN can be excited by external input, pCT-MDP

2

limited data access (only to some states) to analyse known property

3

quantify confidence

4

synthesise optimal experiments

5

study actions tradeoff

6

if stoichiometry is not perfectly known, do network synthesis?

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 19 /20

Take away message

integration of learning and verification verification and policy synthesis for Cyber-Physical Systems (CPS) application in Building Automation Systems (BAS)

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 20 /20

Acknowledgments My students: V. Wijesurija, N. Cauchi, E. Polgreen, A. Peruffo, K. Lesser, M. Zamani, S. Haesaert, I. Tkachev, D. Adzkiya, S. Soudjani and collaborators Selected journal references

• E. Polgreen, V. Wijesuriya, S. Haesaert and A. Abate, “Automated Experiment Design for Efficient Verification of Parametric Markov Decision

Processes,” QEST17, 2017.

• E. Polgreen, V. Wijesuriya, S. Haesert and A. Abate, “Data-efficient Bayesian verification of parametric Markov chains,” QEST16, LNCS 9826,
• pp. 35–51, 2016.
• S. Haesaert, S.E.Z. Soudjani, and A. Abate, “Verification of general Markov decision processes by approximate similarity relations and policy

refinement,” SIAM Journal on Control and Optimisation, vol. 55, nr. 4, pp. 2333-2367, 2017.

• I. Tkachev, A. Mereacre, J.-P. Katoen, and A. Abate, “Quantitative Model Checking of Controlled Discrete-Time Markov Processes,” Information

and Computation, vol. 253, nr. 1, pp. 1–35, 2017.

• S. Haesaert, at al., P.M.J. V.d. Hof, and A. Abate, “Data-driven and Model-based Verification via Bayesian Identification and Reachability

Analysis,” Automatica, vol. 79, pp. 115–126, 2017. S.E.Z. Soudjani and A. Abate, “Aggregation and Control of Populations of Thermostatically Controlled Loads by Formal Abstractions,” IEEE Transactions on Control Systems Technology. vol. 23, nr. 3, pp. 975–990, 2015. S.E.Z. Soudjani and A. Abate, “Quantitative Approximation of the Probability Distribution of a Markov Process by Formal Abstractions,” Logical Methods in Computer Science, Vol. 11, nr. 3, Oct. 2015.

• M. Zamani, P. Mohajerin Esfahani, R. Majumdar, A. Abate, and J. Lygeros, “Symbolic control of stochastic systems via approximately bisimilar

finite abstractions,” IEEE Transactions on Automatic Control, vol. 59 nr. 12, pp. 3135-3150, Dec. 2014.

• I. Tkachev and A. Abate, “Characterization and computation of infinite horizon specifications over Markov processes,” Theoretical Computer

Science, vol. 515, pp. 1-18, 2014.

• S. Soudjani and A. Abate, “Adaptive and Sequential Gridding for Abstraction and Verification of Stochastic Processes,” SIAM Journal on Applied

Dynamical Systems, vol. 12, nr. 2, pp. 921-956, 2013.

• A. Abate, et al., “Approximate Model Checking of Stochastic Hybrid Systems,” European Journal of Control, 16(6), 624-641, 2010.
• A. Abate, et al., “Probabilistic Reachability and Safety Analysis of Controlled Discrete-Time Stochastic Hybrid Systems,” Automatica, 44(11),

2724-2734, Nov. 2008. Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 20 /20

Thank you for your attention For more info: aabate@cs.ox.ac.uk

Alessandro Abate, CS, Oxford Model-based and data-driven verification slide 20 /20