Regression Verification: Project Proposal Presentation by Dennis - PowerPoint PPT Presentation

Regression Verification: Project Proposal Presentation by Dennis Felsing within the Projektgruppe Formale Methoden der Softwareentwicklung SS 2013 1 / 18

Introduction How to prevent regressions in software development? 2 / 18

Introduction Formal Verification Formally prove correctness of software ⇒ Requires formal specification Regression Testing Discover new bugs by testing for them ⇒ Requires test cases 3 / 18

Introduction Formal Verification Formally prove correctness of software ⇒ Requires formal specification Regression Testing Discover new bugs by testing for them ⇒ Requires test cases Regression Verification Formally prove there are no new bugs 3 / 18

Regression Verification Overview Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Uninterpreted Functions Function f Function g without recursions without recursions Static Single Static Single Assignment S f Assignment S g ( n = x ∧ S f ∧ S g ) → r = y SMT Solver Valid / Invalid 4 / 18

Regression Verification Formally prove there are no new bugs • Goal: Proving the equivalence of two closely related programs • No formal specification or test cases required • Instead use old program version • Make use of similarity between programs 5 / 18

Regression Verification Formally prove there are no new bugs • Goal: Proving the equivalence of two closely related programs • No formal specification or test cases required • Instead use old program version • Make use of similarity between programs f ( int n) { int g ( int x ) { int int r = 0; int y = 0; (n ≤ 0) { (x ≤ 1) { i f i f r = n ; y = x ; } else { } else { y = x + g ( x − 1 ) ; r = n + f (n − 1 ) ; } } return r ; return y ; } } 5 / 18

Regression Verification Formally prove there are no new bugs • Goal: Proving the equivalence of two closely related programs • No formal specification or test cases required • Instead use old program version • Make use of similarity between programs f ( int n) { int g ( int x ) { int int r = 0; int y = 0; (n ≤ 0) { (x ≤ 1) { i f i f r = n ; y = x ; } else { } else { y = x + g ( x − 1 ) ; r = n + f (n − 1 ) ; } } return r ; return y ; } } 5 / 18

Uninterpreted Functions Overview Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Uninterpreted Functions Function f Function g without recursions without recursions Static Single Static Single Assignment S f Assignment S g ( n = x ∧ S f ∧ S g ) → r = y SMT Solver Valid / Invalid 6 / 18

Uninterpreted Functions • Given the same inputs an Uninterpreted Function always returns the same outputs. • Motivation: Proof by Induction, to prove f ( n ) = g ( n ) assume f ( n − 1) = g ( n − 1) f ( int n) { int g ( int x ) { int int r = 0; int y = 0; i f (n ≤ 0) { i f (x ≤ 1) { r = n ; y = x ; } else { } else { r = n + U (n − 1 ) ; y = x + U ( x − 1 ) ; } } r ; return y ; return } } 7 / 18

Static Single Assignment Overview Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Uninterpreted Functions Function f Function g without recursions without recursions Static Single Static Single Assignment S f Assignment S g ( n = x ∧ S f ∧ S g ) → r = y SMT Solver Valid / Invalid 8 / 18

Static Single Assignment • Translate program functions to formulas • Recursions: Abstraction by Uninterpreted Function • In assignments x = exp replace x with a new variable x 1 • Represents the states of the program 9 / 18

Static Single Assignment • Translate program functions to formulas • Recursions: Abstraction by Uninterpreted Function • In assignments x = exp replace x with a new variable x 1 • Represents the states of the program int f ( int n) {  r 0 = 0 r = 0; int  i f (n ≤ 0) { S f =   r = n ; } else { r = n + U (n − 1 ) ; } r ; return } 9 / 18

Static Single Assignment • Translate program functions to formulas • Recursions: Abstraction by Uninterpreted Function • In assignments x = exp replace x with a new variable x 1 • Represents the states of the program int f ( int n) {  r 0 = 0 ∧ r = 0; int n ≤ 0 → r 1 = n  i f (n ≤ 0) { S f =   r = n ; } else { r = n + U (n − 1 ) ; } r ; return } 9 / 18

Static Single Assignment • Translate program functions to formulas • Recursions: Abstraction by Uninterpreted Function • In assignments x = exp replace x with a new variable x 1 • Represents the states of the program int f ( int n) {  r 0 = 0 ∧ r = 0; int n ≤ 0 → r 1 = n ∧  i f (n ≤ 0) { S f =  n > 0 → r 1 = n + U ( n − 1)  r = n ; } else { r = n + U (n − 1 ) ; } r ; return } 9 / 18

Static Single Assignment • Translate program functions to formulas • Recursions: Abstraction by Uninterpreted Function • In assignments x = exp replace x with a new variable x 1 • Represents the states of the program int f ( int n) {   r 0 = 0 ∧ r = 0; int n ≤ 0 → r 1 = n ∧   i f (n ≤ 0) { S f =   n > 0 → r 1 = n + U ( n − 1) ∧   r = n ; r = r 1 } else { r = n + U (n − 1 ) ; } r ; return } 9 / 18

Formula Overview Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Uninterpreted Functions Function f Function g without recursions without recursions Static Single Static Single Assignment S f Assignment S g ( n = x ∧ S f ∧ S g ) → r = y � �� � � �� � Equal inputs Equal outputs 10 / 18

SMT Solver Overview Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Uninterpreted Functions Function f Function g without recursions without recursions Static Single Static Single Assignment S f Assignment S g ( n = x ∧ S f ∧ S g ) → r = y SMT Solver Valid / Invalid 11 / 18

Extensions • SMT solver still complains: � − 1 if n = 0 f ( n ) = g ( n ) otherwise 12 / 18

Extensions • SMT solver still complains: � − 1 if n = 0 f ( n ) = g ( n ) otherwise int f ( int n) { int g ( int x ) { r = 0; y = 0; int int i f (n ≤ 0) { i f ( x ≤ 1) { r = n ; y = x ; } else { } else { r = n + f (n − 1 ) ; y = x + g ( x − 1 ) ; } } r ; return y ; return } } 12 / 18

Extensions • SMT solver still complains: � − 1 if n = 0 f ( n ) = g ( n ) otherwise int f ( int n) { int g ( int x ) { r = 0; y = 0; int int i f (n ≤ 0) { i f ( x ≤ 1) { r = n ; y = x ; } else { } else { r = n + f (n − 1 ) ; y = x + g ( x − 1 ) ; } } r ; return y ; return } } • But we can fix it: f (0) = 0 12 / 18

Extensions Finding Counter Examples Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Uninterpreted Functions Function f Function g without recursions without recursions Static Single Static Single Assignment S f Assignment S g ( n = x ∧ S f ∧ S g ) → r = y SMT Solver Execute Valid / Invalid Counter Example 13 / 18

Extensions Determining Corner Cases Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Uninterpreted Functions Function f Function g without recursions without recursions Static Single Static Single Assignment S f Assignment S g ( n = x ∧ S f ∧ S g ∧ U (0) = 0 ) → r = y Add SMT Solver Execute Valid / Invalid Counter Example 14 / 18

Extensions Functional Condition Extraction Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Function f Function g Functional Condition without recursions without recursions Add Static Single Static Single Assignment S f Assignment S g ( n = x ∧ S f ∧ S g ∧ α ) → r = y SMT Solver Valid / Invalid 15 / 18

Extensions Relational Equivalence Equivalent? Function f Function g (val n ; ret r ) (val x ; ret y ) Uninterpreted Functions Function f Function g without recursions without recursions Static Single Static Single Assignment S f Assignment S g ( n ≥ 0 ∧ n = x ∧ S f ∧ S g ) → r ∼ y SMT Solver Valid / Invalid 16 / 18

Example Catalog • Collect examples: Papers, Refactoring Rules, ... • 51 program pairs so far • Test how well approach and extensions work 17 / 18

Conclusion Regression Verification • Better chance of being adopted than Formal Verification • More powerful than Regression Testing • Extensions to cover more cases • Example Catalog for evaluation 18 / 18