Outline Outline Review of PSP Levels Overview Selecting - PDF document

Outline Outline Review of PSP Levels Overview Selecting Verification Methods Design Verification Design Standards Design Verification Verification Methods • Approaches • State Machines • Program Tracing • Program Correctness Etc. AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 1 Humphrey Ch. 12 - slide 1 AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 2 Humphrey Ch. 12 - slide 2 Review of PSP Levels (Humphrey, 1995, p. 11) Overview (cf. Humphrey, 1995, p. 373-374) Review of PSP Levels (Humphrey, 1995, p. 11) Overview (cf. Humphrey, 1995, p. 373-374) To build high-quality software you must ensure PSP3 Cyclic that your designs are correct. Cyclic development Thus, the question is not whether, but how, to verify your programs. PSP2 PSP2.1 Quality Mgt Code reviews Design templates • These approaches are not foolproof. Design reviews • They are prone to human error. PSP1.1 PSP1 • However, their structure facilitates accuracy and Planning Task planning Size estimating Schedule planning reliability. Test report This chapter discusses a number of methods for PSP0.1 PSP0 doing this. Coding standard Size measurement Current process Baseline Process improvement Time recording • Formal methods can sometimes be used. proposal (PIP) Defect recording Defect type standard • However, this book presents “semi-formal” methods. AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 3 Humphrey Ch. 12 - slide 3 AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 4 Humphrey Ch. 12 - slide 4 Verification Methods: Verification Methods: Selecting Verification Methods Selecting Verification Methods Design Standards (cf. Humphrey, 1995, p. 376-378) Design Standards (cf. Humphrey, 1995, p. 376-378) (cf. Humphrey, 1995, p. 374-376) (cf. Humphrey, 1995, p. 374-376) V e r i f i c a t i o n M e t h o d s Design standards do not seem like a verification method. H u m p h r e y ( 1 9 9 5 , p . 3 7 5 ) M e t h o d A p p l i c a t i o n C o m m e n t s However, they provide criteria against which to evaluate a L o o p P r o g r a m U s e o n l o o p l o g i c w h e n e v e r V e r i f i c a t i o n L o o p s p r a c t i c a l . design. P r o p e r S t a t e S t a t e U s e d u r i n g d e s i g n a n d i n r e v i e w s a n d M a c h i n e s M a c h i n e s i n s p e c t i o n s o n e v e r y s t a t e m a c h i n e . Some standards you should use are: O n l y S y m b o l i c A l g o r i t h m i c U s e w h e n e v e r i t a p p l i e s . E x e c u t i o n L o g i c • Product conventions P r o o f b y L o o p s & U s e i n c o n j u n c t i o n w i t h t r a c e t a b l e s . – “Conceptual integrity” I n d u c t i o n R e c u r s i o n T r a c e T a b l e s C o m p l e x U s e f o r s m a l l p r o g r a m e l e m e n t s a n d • Product design standards L o g i c w i t h p r o o f b y i n d u c t i o n a n d / o r s y m b o l i c e x e c u t i o n w h e n e v e r p o s s i b l e . U s e i f o t h e r v e r i f i c a t i o n – Calling & naming conventions m e t h o d s d o n o t a p p l y . – Header, test, and documentation standards & formats, … E x e c u t i o n C o m p l e x U s e f o r s m a l l p r o g r a m e l e m e n t s a n d , T a b l e s L o g i c a s a l a s t r e s o r t , w h e n n o o t h e r – May be arbitrary, but you need a standard. m e t h o d s a p p l y . F o r m a l E n t i r e U s e w h e n e v e r y o u k n o w h o w t o • Reuse standards V e r i f i c a t i o n P r o g r a m a p p l y t h e v e r i f i c a t i o n m e t h o d s , t h e y a p p e a r f e a s i b l e , a n d t h e y a r e c o s t – Components must be well-documented, available, meet needs, e f f e c t i v e . and be reliable Select appropriate methods based on: – IBM’s German lab’s “OS components catalog” parts have never • Your defect profile: Use verification where you have problems. received a user defect report • Effectiveness of your current methods: Use methods you know and are – Toshiba’s control system, which achieved 90% reuse, had the effective with. “lowest defect content of any software [that users] had ever seen.” • Economics of your methods: Use the most cost-effective methods. AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 5 Humphrey Ch. 12 - slide 5 AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 6 Humphrey Ch. 12 - slide 6 1

Verification Methods: Verification Methods: Verification Methods: Verification Methods: Symbolic Execution Proof by Induction Symbolic Execution Proof by Induction (cf. Humphrey, 1995, p. 378-379, & lecture slides) (cf. Humphrey, 1995, p. 379-380, and lecture notes) (cf. Humphrey, 1995, p. 378-379, & lecture slides) (cf. Humphrey, 1995, p. 379-380, and lecture notes) In symbolic execution, the approach is to: Proof by induction states that: • assign algebraic symbols to the program variables 1. if f(n) is true for n = k • restate the program as one or more equations in these 2. and if symbols • when n = z where z > k • analyze the behavior of these equations • and f(z) is true Some questions to ask are: • you can show that f(z+1) is true • does the program converge on a result? 3. then • how does the program behave for both normal and • f(n) is true for all values of n larger than k abnormal input values? • does the program always produce the desired results? Look for places where there would be problems at z+1 (logical or hardware limits, memory, etc.) cf. Example, p. 379, and Lect13.Ppt, p. 9+ cf. Example, p. 380 (Function call) cf. Example, Lect12.Ppt, p. 39 (Factorial) AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 7 Humphrey Ch. 12 - slide 7 AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 8 Humphrey Ch. 12 - slide 8 Verification Methods: Rules for Checking for a Verification Methods: Rules for Checking for a State Machines Proper State Machine State Machines Proper State Machine (cf. Humphrey, 1995, p. 380-397) (cf. Humphrey, 1995, p. 381) (cf. Humphrey, 1995, p. 380-397) (cf. Humphrey, 1995, p. 381) A program is likely a state machine if, with Check for hidden traps or loops. identical inputs, it behaves differently at different • It cannot get stuck in an endless loop and never reach a times. return state. See if all possible states have been identified. Example: LOC counter • A state is defined for every possible combinations of • comments attributes. • non-comments (program, executable) Check for state orthogonality. In a proper state machine: • For every set of conditions there is one and only one • it is possible to reach a program return state from every possible state. other state Check for transition completeness and • all state conditions are complete and orthogonal orthogonality. • all transitions from each state are complete and orthogonal • From every state, a unique next state is defined for every possible combination of state machine input values. AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide Humphrey Ch. 12 - slide 9 9 AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide Humphrey Ch. 12 - slide 10 10 Verification Methods: Two Examples of Checking Verification Methods: Two Examples of Checking Program Tracing Program Tracing State Machines (cf. Humphrey, 1995, p. 381-397) State Machines (cf. Humphrey, 1995, p. 381-397) (cf. Humphrey, 1995, p. 397) (cf. Humphrey, 1995, p. 397) BSet Program tracing is performed with two general methods: • cf. Fig 12.1 (state machine) and Table 12.3 (state specification), p. 382, 383 • Execution Tables • Do checks • Trace Tables CData • cf. Fig 12.2 (state machine) and Table 12.5 (state specification), p. 385, 387- 389 • Do checks AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide 11 Humphrey Ch. 12 - slide 11 AU INSY 560, Winter 1997, Dan Turk AU INSY 560, Winter 1997, Dan Turk Humphrey Ch. 12 - slide Humphrey Ch. 12 - slide 12 12 2