Qualification of PVS for Systematic Design Verification of a Nuclear - - PowerPoint PPT Presentation

Qualification of PVS for Systematic Design Verification of a Nuclear Shutdown System

Mark Lawford & Many Others

McMaster Centre for Software Certification (McSCert) McMaster University Hamilton, ON, Canada

Dagstuhl Seminar 15182 “Qualification of Formal Methods Tools” April 26-29, 2014 Dagstuhl, Germany

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 1 / 40

Outline

Outline

1

Prelude

2

Background: How did we get to qualification of PVS?

3

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

4

Current work on Tools and Conclusions

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 2 / 40

Prelude

It was In 1997, the year before IEC-61508-3 Ed 1.0 came out.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 3 / 40

Prelude

It was In 1997, the year before IEC-61508-3 Ed 1.0 came out. I turned down an NSERC Postdoctoral award to go work on an industrial problem related to my thesis work - verification of real-time controls software. For the next two years I wound up learning about theorem provers and had to qualify and use PVS for the Systematic Design Verification of the Nuclear Generating Station that was on the way to my parent’s place. If it was 2010 or 2011, I could have used IEC-61508-3 (ed 2.0) and IEC 61508-4 (ed 2.0) and life would have been much easier.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 3 / 40

Prelude

It was In 1997, the year before IEC-61508-3 Ed 1.0 came out. I turned down an NSERC Postdoctoral award to go work on an industrial problem related to my thesis work - verification of real-time controls software. For the next two years I wound up learning about theorem provers and had to qualify and use PVS for the Systematic Design Verification of the Nuclear Generating Station that was on the way to my parent’s place. If it was 2010 or 2011, I could have used IEC-61508-3 (ed 2.0) and IEC 61508-4 (ed 2.0) and life would have been much easier. My memory of events may be fuzzy.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 3 / 40

Background: How did we get to qualification of PVS?

In the late 1980s industry kicked the first two soft- ware based SDS com- puter controlled safety systems in Canadia Nu- clear industry over the wall to the regulator and said “approve it!” Each system had ap- prox. 84 monitored variables 27 controlled variables

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 4 / 40

Background: How did we get to qualification of PVS?

In the late 1980s industry kicked the first two soft- ware based SDS com- puter controlled safety systems in Canadia Nu- clear industry over the wall to the regulator and said “approve it!” Each system had ap- prox. 84 monitored variables 27 controlled variables Regulator response:

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 4 / 40

Background: How did we get to qualification of PVS?

In the late 1980s industry kicked the first two soft- ware based SDS com- puter controlled safety systems in Canadia Nu- clear industry over the wall to the regulator and said “approve it!” Each system had ap- prox. 84 monitored variables 27 controlled variables Regulator response: Software?

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 4 / 40

Background: How did we get to qualification of PVS?

In the late 1980s industry kicked the first two soft- ware based SDS com- puter controlled safety systems in Canadia Nu- clear industry over the wall to the regulator and said “approve it!” Each system had ap- prox. 84 monitored variables 27 controlled variables Regulator response: Software? WTF?

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 4 / 40

Background: How did we get to qualification of PVS?

Regulator Reaction to the Originial Darlington SDS

Regulator brought in David Parnas as a consultant. He concluded: The software was developed (and documented), but the requirements documents were not detailed enough. Despite this deficiency, a surprisingly rigorous and comprehensive testing regime had already been developed.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 5 / 40

Background: How did we get to qualification of PVS?

Regulator Reaction to the Originial Darlington SDS

Regulator brought in David Parnas as a consultant. He concluded: The software was developed (and documented), but the requirements documents were not detailed enough. Despite this deficiency, a surprisingly rigorous and comprehensive testing regime had already been developed. Dave’s recommendation:

1

develop tabular requirements specification/design for each existing code module from requirements documents and domain experts

2

create tables from the code for each program function by eliminating intermediate variables

3

independent verification team then tries to (manually) prove tables are equivalent

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 5 / 40

Background: How did we get to qualification of PVS?

Outcome of the First Darlington Licensing

How OPG met regulator expectations Tables were created from both requirements & code in excel spreadsheets Independent team tried to show their were equivalent by manually transforming and composing tables to create new tables in excel spreadsheets. Extensive review of the transformations was used to eliminate single point of failure in process. Regulator was satisfied that system would operate safely but was not maintainable System was licensed for fixed number of years on condition that the software would be redesigned to be maintainable

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 6 / 40

Background: How did we get to qualification of PVS?

Outcome of the First Darlington Licensing

How OPG met regulator expectations Tables were created from both requirements & code in excel spreadsheets Independent team tried to show their were equivalent by manually transforming and composing tables to create new tables in excel spreadsheets. Extensive review of the transformations was used to eliminate single point of failure in process. Regulator was satisfied that system would operate safely but was not maintainable System was licensed for fixed number of years on condition that the software would be redesigned to be maintainable OPG realized they needed a rigorous approach in order to cost effectively develop safety critical software.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 6 / 40

Background: How did we get to qualification of PVS?

What Original Darlington Experience Taught Us

Dialog with the regulator is essential:

To make the process affordable and deterministic for industry To make it possible for the regulator to “assessing” a system with a reasonable amount of effort.

Verification in one step from code to (low-level) requirements was

• ften too large and complex.

reverse engineering tabular specifications for requirements and the code was very expensive and labour intensive

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 7 / 40

Background: How did we get to qualification of PVS?

What Original Darlington Experience Taught Us

Dialog with the regulator is essential:

To make the process affordable and deterministic for industry To make it possible for the regulator to “assessing” a system with a reasonable amount of effort.

Verification in one step from code to (low-level) requirements was

• ften too large and complex.

reverse engineering tabular specifications for requirements and the code was very expensive and labour intensive Automation and tool support is essential!

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 7 / 40

Background: How did we get to qualification of PVS?

What Original Darlington Experience Taught Us

Dialog with the regulator is essential:

To make the process affordable and deterministic for industry To make it possible for the regulator to “assessing” a system with a reasonable amount of effort.

Verification in one step from code to (low-level) requirements was

• ften too large and complex.

reverse engineering tabular specifications for requirements and the code was very expensive and labour intensive Automation and tool support is essential! ⇒ Tool qualification on the redesign!

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 7 / 40

The process & associated implicit “assurance case”

The Darlington SDS Redesign Project

OPG worked with the regulator to get agreement on:

what would be an acceptable development process what evidence would be sufficient for licensing

For the SDS Redesign Project formal techniques were integrated in the forward development process. Forward development process produced evidence for certification/evaluation

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 8 / 40

The process & associated implicit “assurance case”

Using Information Hiding 60 modules 280 access programs 40,000 lines of code (including comments) 33,000 FORTRAN 7,000 assembler 84 monitored variables 27 controlled variables

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 9 / 40

The process & associated implicit “assurance case”

The Assurance Case Implicit in the CANDU Standard

A part of the assurance case implicitly embodied in the standard employed in developing the SDS was as follows:

1

The requirements are specified mathematically and checked for completeness and consistency. A hazards analysis is required to document risks and especially to identify sources of single point

• failures. These hazards have to be mitigated in the specified

requirements.

2

Compliance between requirements and software design is mathematically verified.

3

Compliance between the code and software design is verified through both mathematical verification and testing. Compliance between code and requirements is shown explicitly through

• testing. However, there is an implicit argument of compliance

between code and requirements through the transitive closure of the mathematical verification - code to design, and design to requirements.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 10 / 40

The process & associated implicit “assurance case”

Idealized Development Process

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 11 / 40

The process & associated implicit “assurance case”

Mathematical System Requirements

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 12 / 40

The process & associated implicit “assurance case”

Tabular Expressions

In order for a table to be proper it must satisfy two properties. f(x1, . . . , xm) = c1 c2 . . . cn e1 e2 . . . en Here each ci is a Boolean expression, when ci is true f returns ei

1

Disjointness - i = j → (ci ∧ cj ↔ ⊥)

2

Completeness - (c1 ∨ c2 ∨ . . . ∨ cn) ↔ ⊤

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 13 / 40

The process & associated implicit “assurance case”

Why Tables Work

Tables give “Freedom from choice” to quote Lee quoting Sangiovanni-Vincentelli. f(x, y)

df

=                x + y if x > 1 ∧ y < 0 x − y if x ≤ 1 ∧ y < 0 x if x > 1 ∧ y = 0 xy if x ≤ 1 ∧ y = 0 y if x > 1 ∧ y > 0 x/y if x ≤ 1 ∧ y > 0 (1) f(x, y)

df

= x > 1 x ≤ 1 y < 0 x + y x − y y = 0 x xy y > 0 y x/y (2) Oh, yeah. You can actually read them.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 14 / 40

The process & associated implicit “assurance case”

TCDD Example - NOP Trip

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 15 / 40

The process & associated implicit “assurance case”

TCDD Example - NOP Parameter Trip

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 16 / 40

The process & associated implicit “assurance case”

TCDD Example - NOP Sensor Trip

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 17 / 40

The process & associated implicit “assurance case”

Module Interface Spec for NOP

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 18 / 40

The process & associated implicit “assurance case”

Module Internal Design

Each access program needs to be specified. We specify details of the required behaviour without going to the level of sequential code statements (most of the time) Two very simple rules make mathematical verification much more tractable:

1

Control accessing and updating with heuristic:

Get Process Set

2

Value of an asynchronous variables (e.g. a timer) stored on input - stored value used throughout module

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 19 / 40

The process & associated implicit “assurance case”

Module Internal Design - NOP

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 20 / 40

The process & associated implicit “assurance case”

4-Variable Model (Parnas &Madey)

REQ SOF IN OUT M C I O

M - Monitored Variable statespace C - Controlled Variable statespace I - Input Variable statespace O - Output Variable statespace M, C, I, O are time series vectors and REQ, SOF, IN, OUT are relations. We use a special case where all relations are functional resulting in proof

• bligation:

REQ = OUT ◦ SOF ◦ IN (3) Here REQ and SOF are the one step transition functions of the requirements and design respectively.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 21 / 40

The process & associated implicit “assurance case”

“Vertical” Decomposition of Proof Obligations

M I IN C O OUT Mp Cp SOFreq SOFin SOFout REQ AbstM AbstC

AbstC ◦ REQ = SOFreq ◦ AbstM (4) AbstM = SOFin ◦ IN (5) idC = OUT ◦ SOFout ◦ AbstC . (6) (5) and (6) represent verification of hardware hiding modules Mp is pseudo-monitored variables Cp is pseudo-controlled variables

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 22 / 40

The process & associated implicit “assurance case”

“Horizontal” Decomposition of Proof Obligations

M Mp C Cp REQ . . . SOF1 SOFn . . . REQ1 REQ2 REQn SOF2 V1 V2 Vn−1 AbstC AbstM SOFreq V1p AbstV1 V2p V(n−1)p AbstV2 AbstVn−1

Main block comparison can be sequentially decomposed into sequence of simpler obligations of the form: SOFi ◦ AbstVi−1 = AbstVi ◦ REQi (7) Cost of decomposition? Verifier must provide cross reference in form

• f AbstVi : Vi → Vip. Now we see benefit of "wrong way" arrow: Same

AbstVi can be used on output then input of successive blocks. Note: Only need to check invertibility of AbstC to satisfy (6).

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 23 / 40

The process & associated implicit “assurance case”

Idealized Development Process & Tools

Requirements Documents Software Design Document Code

Requirments Review Report Design Review and Verification Reports Code Review and Verification Reports Unit Test Report Software Integration Test Report Validation Test and Reliability Qual. Reports Legend: Documents produced in the forward going development Documents produced for verifications, reviews and testing Tools connected to documents/activities Activities and data flow Table Tools Table Tools Table Tools Table Tools Table Tools Theorem prover

• Id. Extraction

Tool Code editor & Compiler Logic analyzer Requirements Tool Design Tool Design Veri- fication Tool Design Tool Code Veri- fication Tool Simulation Tool Change Request Tool Config.

• Mgmt. Tool

Test Oracles Unit Test Oracle

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 24 / 40

The process & associated implicit “assurance case”

Idealized Development Process & Tools

Requirements Documents Software Design Document Code

Requirments Review Report Design Review and Verification Reports Code Review and Verification Reports Unit Test Report Software Integration Test Report Validation Test and Reliability Qual. Reports Legend: Documents produced in the forward going development Documents produced for verifications, reviews and testing Tools connected to documents/activities Activities and data flow Table Tools Table Tools Table Tools Table Tools Table Tools Theorem prover

• Id. Extraction

Tool Code editor & Compiler Logic analyzer Requirements Tool Design Tool Design Veri- fication Tool Design Tool Code Veri- fication Tool Simulation Tool Change Request Tool Config.

• Mgmt. Tool

Test Oracles Unit Test Oracle

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 25 / 40

The process & associated implicit “assurance case”

Tool Supported Formal Methods

A formal method should be tightly integrated with the software development process - i.e. it is directly applied to project documents used by all parties as part of the process.

SRS.rtf SDD.rtf DVR.rtf PVS processor Word + SESM Tools SDD.doc DVR.doc SRS.doc block proofs comp. Document flow Information flow b001.pvs b002.pvs b003.pvs etc... Tool SDV SESM

SOFi ◦ AbstVi−1 = AbstVi ◦ REQi

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 26 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

IEC 61508-4 Tool Classification

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 27 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

61508-3 style qualification was used for PVS

PVS classified as T2 Requirements are similar to DO-330 Criterion 3 tool (implies TQL-5 requirements) but not all explicitly mentioned While specification or documentation is expicitly required, you only need to show conformance evidence for T3 tools!

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 28 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

How was PVS used/evaluate

How you use it:

• nly use it with "small" hand checkable steps

(GRIND$) is your new best friend!

No axioms allowed! (Conservative extension) predicate logic with limit predicate subtyping Used to: show completeness & disjointness of tables perform block comparison proofs not for “general” theorem proving Created test cases of how we would use PVS: Sample tables - Some complete & disjoint, some not Sample block verifications We also collected "In use" evidence (thanks NASA!), version,

• perating environment, etc.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 29 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

Properties Claimed

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 30 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

Properties Claimed

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 31 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

Every Formal Verification was done manually too!

Say what? Tools are great, but they don’t buy you as much as you think if they can be a single point of failure. At the time the regulator wanted to mitigate a failure of PVS with a known method, manual proof. Latest version of IEC-61508-3 now provides better guidance here.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 32 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

Solving the Tool Qualification Problem

The bad news: You will, in all likelihood, need two different tools in order to avoid having to do it manually because “demonstrating soundness of the tools” will likely be difficult or impossible The good news: Its not as hard as you might think to knock the tool qualification requirements down a level by doing the same thing with 2+ tools. DSLs can be used to generate code for multiple theorem provers,

• r SMT solvers, or model checkers

There is often more than one way to get a result This can help avoid vendor lock-in Consider this in developing your tools and process

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 33 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

Formal Verification does not replace test

Don’t sell formal verification as a way to eliminating testing, it shouldn’t. Formal verification is done on models of the system. Testing is done on the real system. Theorem proving and other formal verification tools can help: generate test cases (e.g. creative use of PVS random test, proofs → tests, etc.) SMT solvers to hit all cells of a table & model checkers to generate longer test sequences with specific properties formal models as oracles - e.g. PVSio can be used to execute most of the Darlington tables FM can certainly help reduce the cost of testing!

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 34 / 40

The process & associated implicit “assurance case” Qualification of PVS for Darlington SDS Redesign

Formal Verification & Assessment

Regulatory authorities audit - be it process or product based, by looking at samples or checking parts of the work The regulator on Darlington (the Atomic Energy Control Board - now the CNSC) only audited the verification results by checking samples But . . . Automated tools let you “audit everything” - just rerun all the tools.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 35 / 40

Current work on Tools and Conclusions

New Development Process & Tools Replacement

Requirements Documents Software Design Document Code

Requirments Review Report Design Review and Verification Reports Code Review and Verification Reports Unit Test Report Software Integration Test Report Validation Test and Reliability Qual. Reports Legend: Documents produced in the forward going development Documents produced for verifications, reviews and testing Tools connected to documents/activities Activities and data flow Table Tools Table Tools Table Tools Table Tools Table Tools Theorem prover

• Id. Extraction

Tool Code editor & Compiler Logic analyzer Requirements Tool Design Tool Design Veri- fication Tool Design Tool Code Veri- fication Tool Simulation Tool Change Request Tool Config.

• Mgmt. Tool

Test Oracles Unit Test Oracle

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 36 / 40

Current work on Tools and Conclusions

Tabular Expression Toolbox Architecture

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 37 / 40

Current work on Tools and Conclusions

Model Based design of FPGA Based SDS

1

Create tabular specification of function blocks in Matlab/Simulink

2

Use Simulink HDL Coder to generate VHDL from Matlab/Simulink

3

Use equivalence checkers to verify synthesized code against model Also Create processes with tool support with Formal Verification in forward going development process for MBD of PLCs based systems using IEC 61131-3 Function Blocks [Pang et al., 2014].

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 38 / 40

Current work on Tools and Conclusions

How times have changed!

The unprovable sequent was all I could get out of PVS in 1998 as a verifier. This is a very simple one, but still not very user friendly. Now as a developer I can get in Matlab/Simulink [Eles and Lawford, 2010]:

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 39 / 40

Current work on Tools and Conclusions

Making the assumptions explicit

Problem occurred because developer implicitly assumed that Kout < Kin. We can easily make it explicit with PVS subtypes. Note: We still need to a tool like SimCheck to check input typing is satisfied when table with PVS subtyping is used!

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 40 / 40

Current work on Tools and Conclusions

Eles, C. and Lawford, M. (2010). A tabular expression toolbox for matlab/simulink. In 3rd NASA Formal Methods Symposium, volume 6617 of LNCS, pages 494–499. Springer-Verlag. Pang, L., Wang, C.-W., Lawford, M., and Wassyng, A. (2014). Formalizing and verifying function blocks using tabular expressions and pvs. In Second International Workshop on Formal Techniques for Safety-Critical Systems (FTSCS 2013), volume 419 of Communications in Computer and Information Science, pages 163–178. Springer.

Lawford et al. (McSCert) Dagstuhl 15182 2015/04/28 40 / 40