Raluca Borca-Mure san ETHZ June, 14, 2006 Field study and - PowerPoint PPT Presentation

Field study and lassi� ation of faults in Ei�el Raluca Borca-Mure¸ san ETHZ June, 14, 2006 Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 1

Pro je t des ription Project description ❖ Problem: ❖ Solution ❖ Solution ❖ Solution ❖ Solution ❖ Field of study Classification scheme Results Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 2

Problem: ● Testing - an important step in the software development Project description process ❖ Problem: ❖ Solution ❖ Solution ❖ Solution ❖ Solution ❖ Field of study Classification scheme Results Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 3

Problem: ● Testing - an important step in the software development Project description process ❖ Problem: ❖ Solution ❖ Solution ● BUT testing is : ❖ Solution ✦ time consuming ❖ Solution ❖ Field of study ✦ tiresome ✦ boring Classification scheme ✦ . . . Results Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 3

Problem: ● Testing - an important step in the software development Project description process ❖ Problem: ❖ Solution ❖ Solution ● BUT testing is : ❖ Solution ✦ time consuming ❖ Solution ❖ Field of study ✦ tiresome ✦ boring Classification scheme ✦ . . . Results ● use automatic testing tools (AutoTest) that: ✦ provide a list of test cases that generate bugs ✦ NO debugging and interpretation of results Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 3

Problem: ● Testing - an important step in the software development Project description process ❖ Problem: ❖ Solution ❖ Solution ● BUT testing is : ❖ Solution ✦ time consuming ❖ Solution ❖ Field of study ✦ tiresome ✦ boring Classification scheme ✦ . . . Results ● use automatic testing tools (AutoTest) that: ✦ provide a list of test cases that generate bugs ✦ NO debugging and interpretation of results ● HOW TO BUILD A CLASSIFICATION SCHEME? Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 3

Solution AutoTest Project description ❖ Problem: ❖ Solution ❖ Solution ❖ Solution ❖ Solution ❖ Field of study Classification scheme Results Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 4

Solution AutoTest Project description ❖ Problem: KIKS ❖ Solution ❖ Solution ❖ Solution ❖ Solution ❖ Field of study Software Classification scheme Bugs Results Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 5

Solution FINAL PRODUCT Fault classification: AutoTest Project description Category1: ❖ Problem: KIKS ❖ Solution Subcategory 1.1 ❖ Solution Subcategory 1.2 ❖ Solution ... ❖ Solution ❖ Field of study Category 2: Software Classification Subcategory 2.1 scheme Subcategory 2.2 Bugs Results ... ... Category n: Subcategory n.1 Subcategory n.2 ... Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 6

Solution FINAL PRODUCT Fault classification: AutoTest Project description Category1: ❖ Problem: KIKS ❖ Solution Subcategory 1.1 ❖ Solution Subcategory 1.2 ❖ Solution *Large field ... ❖ Solution ❖ Field of study case study Category 2: Software *Analysis and Classification Subcategory 2.1 scheme Interpretation Subcategory 2.2 Bugs Results *Debugging ... ... Category n: Subcategory n.1 Subcategory n.2 ... Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 7

Field of study AutoTest was run on the following libraries and applications: Project description ● EiffelBase ❖ Problem: ❖ Solution ❖ Solution ● Gobo ❖ Solution ❖ Solution ❖ Field of study ● PerfectDeveloper - mathematical library Classification scheme ● DrC Results ● EWG Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 8

Classi� ation s heme Project description Classification scheme ❖ Bug Types ❖ Implementation/ Specification bug ❖ Implementation/ Specification bug ❖ Implementation/ Specification bug ❖ Implementation/ Specification bug ❖ Specification bug- example ❖ Implementation bugs ❖ Implementation bug -example ❖ Dunno bug example ❖ Supplier-induced (SI) bug ❖ Inheritance contract bug ❖ Invariant inheritance bug ❖ Other bugs in Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 9 Category 2

Bug T yp es FINAL PRODUCT Fault classification: AutoTest Project description Category1: KIKS Classification Implementation scheme Specification ❖ Bug Types *Large field ❖ Implementation/ Dunno Specification bug case study ❖ Implementation/ Category 2: Software Specification bug *Analysis and ❖ Implementation/ Faulty supplier Interpretation Specification bug Inheritance ❖ Implementation/ Bugs *Debugging Specification bug Wrong export status ❖ Specification bug- External fault example ❖ Implementation Void target bugs Other ❖ Implementation bug -example ❖ Dunno bug example ❖ Supplier-induced (SI) bug ❖ Inheritance contract bug ❖ Invariant inheritance bug ❖ Other bugs in Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 10 Category 2

Implemen tation/ Sp e i� ation bug Intended Specification Project description ANALYSIS Classification scheme Real ❖ Bug Types Specification ❖ Implementation/ Specification bug ❖ Implementation/ IMPLEMENTATION Specification bug ❖ Implementation/ Specification bug Program ❖ Implementation/ Specification bug ❖ Specification bug- example ❖ Implementation bugs ❖ Implementation bug -example ❖ Dunno bug example ❖ Supplier-induced (SI) bug ❖ Inheritance contract bug ❖ Invariant inheritance bug ❖ Other bugs in Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 11 Category 2

Implemen tation/ Sp e i� ation bug Definitions: Intended Specification ● A specification bug appears VALIDATION Project description because of the discrepancy Specification Bug ANALYSIS between the intended Classification scheme specification and the real Real ❖ Bug Types Specification specification. ❖ Implementation/ Specification bug ❖ Implementation/ IMPLEMENTATION Specification bug ❖ Implementation/ Specification bug Program ❖ Implementation/ Specification bug ❖ Specification bug- example ❖ Implementation bugs ❖ Implementation bug -example ❖ Dunno bug example ❖ Supplier-induced (SI) bug ❖ Inheritance contract bug ❖ Invariant inheritance bug ❖ Other bugs in Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 12 Category 2

Implemen tation/ Sp e i� ation bug Definitions: Intended Specification ● A specification bug appears VALIDATION Project description because of the discrepancy Specification Bug ANALYSIS between the intended Classification scheme specification and the real Real ❖ Bug Types Specification specification. ❖ Implementation/ Specification bug VERIFICATION ● An implementation bug ❖ Implementation/ Implementation IMPLEMENTATION Specification bug appears because the Bug ❖ Implementation/ implementation of the Specification bug Program ❖ Implementation/ routine does not fulfill the Specification bug real specification of the ❖ Specification bug- example routine. ❖ Implementation bugs ❖ Implementation bug -example ❖ Dunno bug example ❖ Supplier-induced (SI) bug ❖ Inheritance contract bug ❖ Invariant inheritance bug ❖ Other bugs in Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 13 Category 2

Implemen tation/ Sp e i� ation bug Definitions: Intended Specification ● A specification bug appears VALIDATION Project description because of the discrepancy Specification Bug ANALYSIS between the intended Classification scheme specification and the real Real ❖ Bug Types Specification specification. ❖ Implementation/ Specification bug VERIFICATION ● An implementation bug ❖ Implementation/ Implementation IMPLEMENTATION Specification bug appears because the Bug ❖ Implementation/ implementation of the Specification bug Program ❖ Implementation/ routine does not fulfill the Specification bug real specification of the ❖ Specification bug- example routine. ❖ Implementation bugs ● Dunno : prove that a bug ❖ Implementation can be interpreted as being bug -example ❖ Dunno bug a specification and an example interpretation bug. ❖ Supplier-induced (SI) bug ❖ Inheritance contract bug ❖ Invariant inheritance bug ❖ Other bugs in Field study and classification of faults in Eiffel ETHZ - June 2006, Raluca Borca-Mure¸ san - p. 14 Category 2