Parameterized Systems (via Effectively Propositional Reasoning) - - PowerPoint PPT Presentation

▶
parameterized systems via
SMART_READER_LITE
LIVE PREVIEW

Parameterized Systems (via Effectively Propositional Reasoning) - - PowerPoint PPT Presentation

Aug 18, 2022 745 likes •1.03k views

Ivy: Interactive Verification of Parameterized Systems (via Effectively Propositional Reasoning) Oded Padon Joint work with: Ken McMillan and Mooly Sagiv Dagstuhl Seminar, 3-Nov-2015 Motivation Many real-world verification efforts use

slide-1
SLIDE 1

Ivy: Interactive Verification of Parameterized Systems (via Effectively Propositional Reasoning)

Oded Padon Joint work with: Ken McMillan and Mooly Sagiv Dagstuhl Seminar, 3-Nov-2015

slide-2
SLIDE 2

Motivation

  • Many real-world verification efforts use little proof automation:
  • CompCert, seL4, Verdi, IronClad/IronFleet, Intel, AMD
  • Automated tools are brittle and opaque
  • Fail unpredictably and completely
  • Unavoidable since problems are intractable / undecidable
  • Diagnosing and correcting failures is hard because tools do

not effectively communicate with the user

  • With interactive tools, the system always makes progress
  • Goal: Put the user “in the loop” of more automated verification
  • Use powerful invariant generation

heuristics interactively

  • Bidirectional feedback between

user and machine heuristics

slide-3
SLIDE 3

Effectively Propositional Logic – EPR

a.k.a. Bernays-Schönfinkel-Ramsey class

  • Fragment of first-order logic
  • Restricted quantifier prefix: ** φQ.F.
  • No function symbols
  • Small model property
  • x1,…, xn. y1,…,ym.φQ.F. has a model iff

it has a model of at most n+k elements (k - number of constant symbols)

  • Satisfiability is decidable
  • Support from Z3
  • F. Ramsey. On a problem in formal logic. Proc. London Math. Soc. 1930
slide-4
SLIDE 4
  • I is an inductive invariant iff I∧TR∧¬I’ is unsatisfiable
  • EPR transition system:
  • States – finite structures
  • Transition relation – two vocabulary ** formula
  • Checking inductiveness of alternation-free invariants is

decidable

  • Expressible in EPR:
  • Partial function
  • Injective function
  • Partial order
  • Total order
  • Not expressible in EPR:
  • Total function
  • Surjective function

Using EPR for Verification

slide-5
SLIDE 5

Prior Work on EPR based Verification

  • Itzhaky et al. showed EPR can be used to reason about

reachabilty in linked lists by using n* instead of n

  • Karbyshev et al. showed a version of IC3/PDR that infers

universal invariants for EPR transition systems

  • Padon et al. showed that under certain conditions (e.g. linked

lists) inferring universal inductive invariants is decidable

  • S. Itzhaky et al. Effectively-Propositional Reasoning about Reachability in Linked

Data Structures. CAV 2013

  • S. Itzhaky et al. Modular reasoning about heap paths via effectively propositional
  • formulas. POPL 2014
  • A. Karbyshev et al. Property-Directed Inference of Universal Invariants or Proving

Their Absence. CAV 2015

  • O. Padon et al. Decidability of Inferring Inductive Invariants. POPL 2016
slide-6
SLIDE 6

Ivy Overview

Elements of Interactive Verification

  • EPR based verification
  • Graphical visualization of proof state and proof failures
  • Visualization of sets of structures - concept graph
  • Visualization of carefully chosen concrete transitions
  • Web-based UI (embedded in IPython/Jupyter notebook)
  • Interaction modes
  • Bounded verification with BMC
  • Abstract reachability graph based
  • Counterexample to induction based
  • Scriptable interface
slide-7
SLIDE 7

Ivy Overview

Elements of Interactive Verification

  • EPR based verification
  • Graphical visualization of proof state and proof failures
  • Visualization of sets of structures - concept graph
  • Visu

isuali lization of

  • f car

arefull lly ch chosen con

  • ncrete tr

tran ansit itions

  • Web-based UI (embedded in IPython/Jupyter notebook)
  • Interaction modes
  • Bounded verification with BMC
  • Abstract reachability graph based
  • Cou
  • unterexample

le to

  • in

induction base ased

  • Scriptable interface
slide-8
SLIDE 8

CTI based Interaction Mode

Model Candidate Inductive Invariant Inductive? Yes No Display “minimal” CTI Modify candidate invariant Diagnose CTI User Heuristics Inductive Invariant Found

slide-9
SLIDE 9

CTI based Interaction Mode

Model Candidate Inductive Invariant Inductive? Yes No Display “minimal” CTI Modify candidate invariant Diagnose CTI User Heuristics Inductive Invariant Found Z3 (EPR)

slide-10
SLIDE 10

Example: Leader Election in a Ring

  • Nodes are organized in a ring
  • Each node has a unique numeric id
  • Protocol:
  • Each node sends its id to the left
  • A node that receives a massage passes it (to the left) if

the id in the message is higher than the node’s own id

  • A node that receives its own id becomes the leader
  • Theorem:
  • The protocol selects at most one leader
  • E. Chang and R. Roberts. An improved algorithm for decentralized extrema-finding

in circular configurations of processes. CACM 1979

3 5 2 4 1 6

slide-11
SLIDE 11
  • Nodes are organized in a ring
  • Each node has a unique numeric id
  • Protocol:
  • Each node sends its id to the left
  • A node that receives a massage passes it (to the left) if

the id in the message is higher than the node’s own id

  • A node that receives its own id becomes the leader
  • Theorem:
  • The protocol selects at most one leader
  • E. Chang and R. Roberts. An improved algorithm for decentralized extrema-finding

in circular configurations of processes. CACM 1979

Example: Leader Election in a Ring

3 5 2 4 1 6

slide-12
SLIDE 12

Leader Protocol Model

  • Sorts: Node, ID
  • Constants:
  • rin

ring_head: Node

  • rin

ring_tail il: Node

  • zero: ID
  • le

lead ader_id id: ID

  • Relations:
  • le

le(ID, ID) – total order on node id’s

  • reach(Node, Node) – total order, describes ring topology
  • id

id(Node, ID) – relate a node to its id

  • pendin

ing(ID, Node) – models pending messages

  • le

lead ader(Node) – leader(n) means n is the leader

slide-13
SLIDE 13

Leader Protocol Model

action send: n1 := * n2 := next_in_ring(n1) i1 := id(n1) pending.insert(i1, n2)

  • le

le(ID, ID) – total order on node id’s

  • reach(Node, Node) – total order, describes ring topology
  • id

id(Node, ID) – relate a node to its id

  • pen

endin ing(ID, Node) – models pending messages

  • lea

leader(Node) – leader(n) means n is the leader action receive: m, n1 := pending.nd_pop() i1 := id(n1) if i1 = m: # found leader leader.insert(n1) leader_id := i1 else if le(i1, m): # pass message n2 := next_in_ring(n1) pending.insert(m, n2)

slide-14
SLIDE 14

Leader Protocol Model

action send: n1 := * n2 := next_in_ring(n1) i1 := id(n1) pending.insert(i1, n2) init ¬leader(N) ∧ ¬pending(I,N) ∧ leader_id = zero conjecture (leader(X) ∧ leader(Y))  X = Y conjecture leader(X) -> id(X, leader_id) conjecture (id(X, I) ∧ id(Y, J) ∧ leader(X))  le(J, I) action receive: m, n1 := pending.nd_pop() i1 := id(n1) if i1 = m: # found leader leader.insert(n1) leader_id := i1 else if le(i1, m): # pass message n2 := next_in_ring(n1) pending.insert(m, n2)

slide-15
SLIDE 15

DEMO

slide-16
SLIDE 16

Heuristics for User Interaction

  • CTI

CTI min inim imization

  • Small number of elements
  • “Small” relations according to

user’s choice / model hints

  • Dia

Diagnosis is in interactions

  • Select visible relations
  • Gather facts from user selection
  • BMC
  • Check conjecture
  • Minimize conjecture
  • Sufficiency for current failure
  • Relative inductiveness

Model Invariant Inductive? Yes No

Display “minimal” CTI

Modify Invariant

Diagnose CTI User Heuristics 

slide-17
SLIDE 17

Observations

  • User intuition and machine heuristics complement each other:
  • User has better bias towards relevant and inductive facts that

leads to better generalizations

  • Machine is better at finding bugs and corner cases
  • Challenge is finding useful interaction mode and heuristics
  • Might not be the same as fully automated heuristics (e.g. PDR)
  • Heuristics will sometimes fail, so must leave a “manual” mode
slide-18
SLIDE 18

Observations

  • User intuition and machine heuristics complement each other:
  • User has better bias towards relevant and inductive facts that

leads to better generalizations

  • Machine is better at finding bugs and corner cases
  • Challenge is finding useful interaction mode and heuristics
  • Might not be the same as fully automated heuristics (e.g. PDR)
  • Heuristics will sometimes fail, so must leave a “manual” mode
  • EPR allows predictability of automated analysis
  • EPR small model property makes BMC more powerful
  • Bound is only on TR unrolling, not on universe size
  • EPR verification sometimes requires additional ghost state
slide-19
SLIDE 19

Applications

Application Status Distributed spanning tree Modeled and verified Network learning switch Modeled and verified Leader in Ring Modeled and verified Chord Modeled and verified Flash Modeled and verified* Copy Garbage Collection Modeled and verified* Bakery Algorithm Modeled, work in progress Distributed Database Work in progress (with Aurojit Panda, UC Berkeley)

We are looking for more applications, please let me know your suggestions! *Not yet full functional correctness, still work in progress

slide-20
SLIDE 20
  • Ivy is a framework for exploring interactive verification:
  • EPR based verification using Z3
  • Graphical visualization (concrete structures, concept graphs)
  • Web-based UI (embedded in IPython/Jupyter notebook)
  • Scriptable interface

Ivy as a Research Framework

slide-21
SLIDE 21

CTI Mode

  • Ivy is a framework for exploring interactive verification:
  • EPR based verification using Z3
  • Graphical visualization (concrete structures, concept graphs)
  • Web-based UI (embedded in IPython/Jupyter notebook)
  • Scriptable interface

M Inv Ind? “minimal” CTI Modify Inv Diagnose CTI User Heuristics 

Ivy as a Research Framework

slide-22
SLIDE 22

CTI Mode

  • Ivy is a framework for exploring interactive verification:
  • EPR based verification using Z3
  • Graphical visualization (concrete structures, concept graphs)
  • Web-based UI (embedded in IPython/Jupyter notebook)
  • Scriptable interface

M Inv Ind? “minimal” CTI Modify Inv Diagnose CTI User Heuristics 

BMC

M Spec BMC Fix model / spec 

Ivy as a Research Framework

slide-23
SLIDE 23

CTI Mode

  • Ivy is a framework for exploring interactive verification:
  • EPR based verification using Z3
  • Graphical visualization (concrete structures, concept graphs)
  • Web-based UI (embedded in IPython/Jupyter notebook)
  • Scriptable interface

M Inv Ind? “minimal” CTI Modify Inv Diagnose CTI User Heuristics 

BMC

M Spec BMC Fix model / spec 

Abstract Reachability & Concept Graphs

User Heuristics 

Ivy as a Research Framework

slide-24
SLIDE 24

CTI Mode

  • Ivy is a framework for exploring interactive verification:
  • EPR based verification using Z3
  • Graphical visualization (concrete structures, concept graphs)
  • Web-based UI (embedded in IPython/Jupyter notebook)
  • Scriptable interface

M Inv Ind? “minimal” CTI Modify Inv Diagnose CTI User Heuristics 

BMC

M Spec BMC Fix model / spec 

Abstract Reachability & Concept Graphs

User Heuristics 

???

 User Heuristics

Ivy as a Research Framework

slide-25
SLIDE 25

Conclusion

  • Putting the user “in the loop” of automated verification
  • Let the machine solve a decidable problem
  • Graphical visualization
  • Useful interaction modes & heuristics (we’re still experimenting)
  • User and machine complement each other via bidirectional

feedback

  • Machine computes counterexamples and proven facts
  • User diagnoses proof failures and guides generalization
  • Machine provides heuristics to assist generalization
  • Future directions:
  • More interaction modes and heuristics
  • Forward-looking heuristics: is a conjecture causally related

to the failure?

  • “EPR++”: recovering  information via ghost state,

differencing, and manual/heuristic quantifier instantiation

  • Large scale real-world verification problems
slide-26
SLIDE 26

Conclusion

  • Putting the user “in the loop” of automated verification
  • Let the machine solve a decidable problem
  • Graphical visualization
  • Useful interaction modes & heuristics (we’re still experimenting)
  • User and machine complement each other via bidirectional

feedback

  • Machine computes counterexamples and proven facts
  • User diagnoses proof failures and guides generalization
  • Machine provides heuristics to assist generalization
  • Future directions:
  • More interaction modes and heuristics
  • Forward-looking heuristics: is a conjecture causally related

to the failure?

  • “EPR++”: recovering  information via ghost state,

differencing, and manual/heuristic quantifier instantiation

  • Large scale real-world verification problems