Verified Runtime Monitoring: From Foundations to Practice Presenter: - - PowerPoint PPT Presentation

Verified Runtime Monitoring: From Foundations to Practice

Presenter: Brandon Bohrer1, based on works with: Yong Kiam Tan1, Stefan Mitsch1, Andrew Sogokon1, Edward Ahn1, David Held1, John Dolan1, Aman Khurana1, Magnus O. Myreen2, and Andr´ e Platzer1

Carnegie Mellon University1 Chalmers University of Technology2

CMU V&V Workshop, Dec 12 2018

A Real Cyber-Physical System

2

A Scary Cyber-Physical System

2

VeriPhy: Automatic, Verified EXEs from Controllers

3

VeriPhy: Automatic, Verified EXEs from Controllers

3

VeriPhy: Automatic, Verified EXEs from Controllers

3

VeriPhy: Automatic, Verified EXEs from Controllers

3

VeriPhy: Automatic, Verified EXEs from Controllers

3

VeriPhy: Automatic, Verified EXEs from Controllers

3

VeriPhy: Automatic, Verified EXEs from Controllers

3

VeriPhy: Automatic, Verified EXEs from Controllers

3

HPs Model Control and Environment

4

α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

HPs Model Control and Environment

4

α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

Far Enough?

HPs Model Control and Environment

4

α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

Far Enough? Velocity Envelope

HPs Model Control and Environment

4

α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

Far Enough? Velocity Envelope Fallback

HPs Model Control and Environment

4

α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

Far Enough? Velocity Envelope Fallback Physics

HPs Model Control and Environment

4

α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

Far Enough? Velocity Envelope Fallback Physics Constraint

KeYmaera X Enables Model Verification

5

ModelPlex: Provably Correct Monitors

6

Monitor whether transitions from previous state x to next state x+ are consistent with control, environment models. α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

ModelPlex: Provably Correct Monitors

6

Monitor whether transitions from previous state x to next state x+ are consistent with control, environment models. α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

ModelPlex: Provably Correct Monitors

6

Monitor whether transitions from previous state x to next state x+ are consistent with control, environment models. α ≡

• (

drive

• ?d ≥ εV ; v := ∗; ?0 ≤ v ≤ V ∪

stop

v := 0); t := 0; {

env.

• d′ = −v, t′ = 1 & t ≤ ε}

∗

Provable Monitor Provable Sandbox

7

Sandboxed controller uses external controller when decision is safe, else uses verified fallback. Detects non-compliant plants. V := ∗; ε := ∗; d := ∗; t := ∗; // x := ∗ ?d ≥ 0 ∧ V ≥ 0 ∧ ε ≥ 0; // ?φ

t+ := ∗; v+ := ∗; d+ := d;

// x+ := extCtrl ( ?ctrlMon(d, t, v, d+, t+, v+) ∪ t+ := 0; v+ := 0 ); // x+ := fallback t := t+; v := v+; // x := x+ d+ := ∗; t+ := ∗; // x+ := ∗ ?plantMon( x, x+); d := d+; t := t+ // x := x+∗

Intervals Make ctrlMon and plantMon Computable

8

Example: Check whether π < e, efficiently. Solution: Conservative interval approximation

Example

Let νI = {pi → [3, 4], e → [2, 3]}, then

• pi <w e is false (⊥)

Intervals Make ctrlMon and plantMon Computable

8

Example: Check whether π < e, efficiently. Solution: Conservative interval approximation

Example

Let νI = {pi → [3, 4], e → [2, 3]}, then

• pi <w e is false (⊥)
• pi <w e + 3 is true (⊤)

Intervals Make ctrlMon and plantMon Computable

8

Example: Check whether π < e, efficiently. Solution: Conservative interval approximation

Example

Let νI = {pi → [3, 4], e → [2, 3]}, then

• pi <w e is false (⊥)
• pi <w e + 3 is true (⊤)
• pi <w e + 1 is ???

Intervals Make ctrlMon and plantMon Computable

8

Example: Check whether π < e, efficiently. Solution: Conservative interval approximation

Example

Let νI = {pi → [3, 4], e → [2, 3]}, then

• pi <w e is false (⊥)
• pi <w e + 3 is true (⊤)
• pi <w e + 1 is a known unknown (U)

When truth values can be unknown, resulting logic is 3-valued

Interval dL is 3-Valued

9

∧ ⊤ U ⊥ ⊤ ⊤ U ⊥ U U U ⊥ ⊥ ⊥ ⊥ ⊥ ∨ ⊤ U ⊥ ⊤ ⊤ ⊤ ⊤ U ⊤ U U ⊥ ⊤ U ⊥ ωI[ (θ1 + θ2) ] = [l1 ˇ +wl2, u1 ˆ +wu2] where ωI[ (θi) ] = [li, ui] ωI[ (θ1<θ2) ] =

      

⊤ if ωI[ (θi) ] = (li, ui) and u1 < l2 ⊥ if ωI[ (θi) ] = (li, ui) and l1 ≥ u2 U

• therwise

(ωI, νI) ∈ [ (α ∪ β) ] iff (ωI, νI) ∈ [ (α) ] or (ωI, νI) ∈ [ (β) ]

Interval dL is a Sound Approximation

10

Theorem (Interval Soundness for Formulas)

• If ω ∈ ωI and ωI[

(φ) ]=⊤ then ω ∈ [ [φ] ]

• If ω ∈ ωI and ωI[

(φ) ]=⊥ then ω / ∈ [ [φ] ]

• No claims when ωI[

(φ) ]=U Generalizes naturally to programs, but CakeML sandbox only runs simpler formula case

Sandbox HP Already Verified

11 V := ∗; ε := ∗; d := ∗; t := ∗; // x := ∗ ?d ≥ 0 ∧ V ≥ 0 ∧ ε ≥ 0; // ?φ

• t+ := ∗; v + := ∗; d+ := d;

// x+ := extCtrl ( ?ctrlMon(d, t, v, d+, t+, v +) ∪ t+ := 0; v + := 0 ); // x+ := fallback t := t+; v := v +; // x := x+ d+ := ∗; t+ := ∗; // x+ := ∗ ?

• 0≤t+≤ε ∧ d+≥v(ε − t+)
• ;

// ?plantMon( x, x+) d := d+; t := t+ // x := x+∗

Verified CakeML Source is Generated

11

CakeML source incorporates external control, actuation, sensing

fun cmlSandboxBody state = if not (stop ()) then state.ctrl+:= extCtrl state; state.ctrl := if intervalSem ctrlMon state = ⊤ then state.ctrl+ else fallback state; actuate state.ctrl; state.sensors+:= sense (); if intervalSem plantMon state = ⊤ then Runtime.fullGC (); state.sensors := state.sensors+; cmlSandboxBody state else violation "Plant Violation"

CakeML Sandbox is Sound

12

Theorem (Soundness for CakeML Sandbox, Main Case)

If

[

{ω} ], [ {ν} ]

∈ [

{cmlSandbox} ] then ([ (ω) ], [ (ν) ]) ∈ [ (sandbox) ]

CakeML Compiler Preserves Guarantees

13

Code Executed on Sim, Soon Bot

14

Speed Ctrl Fail. Phys Fail. Collide World Sim Human Sim Human Sim Human Sim Human 1 6.69 17.4 .431 .913 .045 .377 2 5.78 10.7 .632 .890 .011 .417 3 7.89 29.9 1 .996 .01 .151

Table : Average speed, Monitor failure rates, safety violation rates, for AirSim, F1/10, and human driver in Rectangular World, NeighborHood, and Free-Range for Patrol and Goto missions

Proof Chain Justifies Transformations

15

ν | = ψ ⇑ (ω, ν) ∈ [ [sandbox] ]

dL (KeYmaera X)

Real arithmetic, nondeterministic

⇑

ωI, νI ∈

[ (sandbox) ]

dL (Isabelle/HOL)

Interval word arithmetic, nondeterministic

⇑

[

{ω} ], [ {ν} ]

∈ [

{cmlSandbox} ]

CakeML (HOL4)

Interval word arithmetic, deterministic

⇑

{

|ω| }, { |ν| }

∈ {

|CML(cmlSandbox)| }

ARM/x64

Interval word arithmetic, machine-executable

Takeaway Metaphor

16

Takeaway Metaphor

16

References I

17

Brandon Bohrer, Vincent Rahli, Ivana Vukotic, Marcus V¨

• lp,

and Andr´ e Platzer, Formally verified differential dynamic logic, Certified Programs and Proofs - 6th ACM SIGPLAN Conference, CPP 2017, Paris, France, January 16-17, 2017 (Yves Bertot and Viktor Vafeiadis, eds.), ACM, 2017,

• pp. 208–221.

Joe Hurd, The OpenTheory standard theory library, NFM (Mihaela Gheorghiu Bobaru, Klaus Havelund, Gerard J. Holzmann, and Rajeev Joshi, eds.), LNCS, vol. 6617, Springer, 2011, pp. 177–191. Magnus O. Myreen and Scott Owens, Proof-producing synthesis of ML from higher-order logic, ICFP (Peter Thiemann and Robby Bruce Findler, eds.), ACM, 2012,

• pp. 115–126.

Isabelle/HOL Cross-Checks KeYmaera X

18

Problem: Later pipeline stages need understanding of dL semantics, which KeYmaera X lacks

Isabelle/HOL Cross-Checks KeYmaera X

18

Problem: Later pipeline stages need understanding of dL semantics, which KeYmaera X lacks Solution: Import soundly into Isabelle/HOL from KeYmaera X

• Proof term exported from KeYmaera X, serialized
• Proof checker verified in Isabelle/HOL, extending [BRV+17]

Isabelle/HOL Cross-Checks KeYmaera X

18

Problem: Later pipeline stages need understanding of dL semantics, which KeYmaera X lacks Solution: Import soundly into Isabelle/HOL from KeYmaera X

• Proof term exported from KeYmaera X, serialized
• Proof checker verified in Isabelle/HOL, extending [BRV+17]
• Executable checker code-generated [MO12]
• Scales to 100K’s of proof steps (≈6 seconds)
• Eliminates KeYmaera X core from trusted base!

Isabelle/HOL → HOL4 Translation is Trusted

19

Isabelle/HOL Strength: Library Access

• Analysis libraries (absolute must for dL soundness)
• Machine word libraries (must for interval arithmetic)

Isabelle/HOL → HOL4 Translation is Trusted

19

Isabelle/HOL Strength: Library Access

• Analysis libraries (absolute must for dL soundness)
• Machine word libraries (must for interval arithmetic)

Isabelle/HOL Weakness: Weaker Verified Compiler Support

• This is a problem: need to generate source code!

Isabelle/HOL → HOL4 Translation is Trusted

19

Isabelle/HOL Strength: Library Access

• Analysis libraries (absolute must for dL soundness)
• Machine word libraries (must for interval arithmetic)

Isabelle/HOL Weakness: Weaker Verified Compiler Support

• This is a problem: need to generate source code!

We jump to HOL4 for access to verified CakeML compiler:

• Manually translate Isabelle/HOL definitions to HOL4
• Justification: Similar logical foundation
• Could be automated in principle, see OpenTheory [Hur11]

Future Work

20

Improve pipeline components:

• Reduce trusted base: OpenTheory, arithmetic witnesses in

KeYmaera X

• Floating-point, mixed precision interval arithmetic
• Generalize proof-driven monitor synthesis

Exploit pipeline in case studies:

• UAVs
• High-speed robots
• Your favorite CPS