Calculation and Optimization of Thresholds for Sets of Software - PDF document

Calculation and Optimization of Thresholds for Sets of Software Metrics Steffen Herbold, Jens Grabow ski , Stephan Waack Georg-August-Universität Göttingen Institute of Computer Science 1 Contents  Motivation  Software Metrics  Classification with Thresholds  Optimization of Sets of Metrics  Applications  Case Studies  Summary and Outlook Optimization of Metric Sets with Thresholds 2 1

What is Software Engineering?  Software Engineering is  development,  maintenance, and  deployment of high-quality Software in consideration of  scientific methods,  economic principles,  planned development models, and  quantifiable goals.  B. Kahlbrandt: Software-Engineering: Objektorientierte Software- Entwicklung mit der Unified Modeling Language , Springer Verlag (1998) Optimization of Metric Sets with Thresholds 3 What is Software Engineering?  Software Engineering is  development,  maintenance, and  deployment of high-quality Softw are in consideration of  scientific methods,  economic principles,  planned development models, and  quantifiable goals .  B. Kahlbrandt: Software-Engineering: Objektorientierte Software- Entwicklung mit der Unified Modeling Language , Springer Verlag (1998) Optimization of Metric Sets with Thresholds 4 2

What is Software Quality? Maintenance Quality Process Quality Project Quality Usage Quality time Project Start System Retirement Start of Operation Optimization of Metric Sets with Thresholds 5 Quality Assessment using ISO 9126 External and Internal Quality Functionality Reliability Usability Efficiency Maintainability Portability Suitability Maturity Understand- Time Analysability Adaptability ability Behaviour Accuracy Fault-Tolerance Changeability Installability Resource Learnability Interoperability Recoverability Utilisation Stability Co-Existence Operability Security Testability Replaceability Attractiveness Metrics Optimization of Metric Sets with Thresholds 6 3

Contents  Motivation  Softw are Metrics  Classification with Thresholds  Optimization of Sets of Metrics  Applications  Case Studies  Summary and Outlook Optimization of Metric Sets with Thresholds 7 What do Software Metrics Measure?  “You cannot control what you cannot measure” (Tom DeMarco)  “To measure is to know” (Clerk Maxwell) Engine Power 100 PS Fuel usage 5,8 l Max. speed 176 km/h W eight 1458 kg ??? Optimization of Metric Sets with Thresholds 8 4

Properties of Software Metrics  Modes of measurement  internal  external  Objects of measurement  products  processes  resources  W e perform an internal m easurem ent of products by m eans of static analysis of source code. Optimization of Metric Sets with Thresholds 9 Metrics for Methods and Classes Modules, Files Classes Coupling Between Objects (CBO) Methods Response For a Class (RFC) Weighted Method per Class (WMC) Number of Statements (NST) Number of Overriden Methods (NORM) McCabe‘s Cyclomatic Number (VG) Lines Of Code (LOC) Nested Block Depth (NBD) Number Of Methods (NOM) Number of Function Calls (NFC) Number of Static Methods (NSM) Optimization of Metric Sets with Thresholds 10 5

Software Metrics for Methods  Number of Statements (NST)  McCabe’s Cyclomatic Number (VG)  Number of branches in the control flow  Nested Block Depth (NBD)  Max. depth of nested statement blocks  Number of Function Calls (NFC)  Number of methods invoked by the method under investigation Optimization of Metric Sets with Thresholds 11 Metrics for Classes 1(2)  Coupling Between Objects (CBO)  Number of associations with other classes  Response For a Class (RFC)  Number of methods that can be called when the methods of the class under investigation are invoked  Weighted Methods per Class (WMC)  Sum of complexities of all methods  Complexity: McCabe’s Cyclomatic Number (VG) Optimization of Metric Sets with Thresholds 12 6

Metrics for Classes 2(2)  Number of Overridden Methods (NORM)  Number of redefined methods inherited from a superclass  Lines Of Code (LOC)  Lines of source code without empty lines and comments  Number Of Methods (NOM)  Number of Static Methods (NSM) Optimization of Metric Sets with Thresholds 13 Quality Assessment using ISO 9126 (revisited) External and Internal Quality Functionality Reliability Usability Efficiency Maintainability Portability Suitability Maturity Understand- Time Analysability Adaptability ability Behaviour Accuracy Fault-Tolerance Changeability Installability Resource Learnability Interoperability Recoverability Utilisation Stability Co-Existence Operability Security Testability Replaceability Attractiveness Metrics Optimization of Metric Sets with Thresholds 14 7

Contents  Motivation  Software Metrics  Classification w ith Thresholds  Optimization of Sets of Metrics  Applications  Case Studies  Summary and Outlook Optimization of Metric Sets with Thresholds 15 Thresholds  Mechanism to classify values Threshold  Metrics with upper and lower bound  Only upper bounds are considered Optimization of Metric Sets with Thresholds 16 8

Thresholds for Methods Nam e of Metric Program m ing Threshold Language McCabe’s Cyclomatic Number C 24 (VG) C++/C# 10 Nested Block Depth (NBD) C/C++/C# 5 Number of Function Calls (NFC) C/C++/C# 5 Number of Statements (NST) C/C++/C# 50 Optimization of Metric Sets with Thresholds 17 Thresholds for Java Classes Nam e of Metric Threshold Weighted Methods per Class (WMC) 100 Coupling Between Objects (CBO) 5 Response For a Class (RFC) 100 Number of Overriden Methods (NORM) 3 Lines of Code (LOC) 500 Number of Methods (NOM) 20 Number of Static Methods (NSM) 4 Optimization of Metric Sets with Thresholds 18 9

Thresholds and Rectangles Metric 2 Threshold1 * Threshold 2 * * ... * 2 Threshold 1 * * 1 Metric 1 0 1 ... 2 Optimization of Metric Sets with Thresholds 19 Contents  Motivation  Software Metrics  Classification with Thresholds  Optim ization of Sets of Metrics  Applications  Case Studies  Summary and Outlook Optimization of Metric Sets with Thresholds 20 10

General Idea  Rectangles = sets of thresholds  Rectangles are computed using machine learning  Data-driven method  Based on previous measurements (or manual classification) of software  Measurements (or classification) partition the software into good and bad software Optimization of Metric Sets with Thresholds 21 Optimization of Metric Sets  Given:  Set of metrics: M = { m 1 , … , m n }  Software system: S = { s 1 , s 2 , … }  s i = classes, methods or functions  Metric values m 1 (s i ), … , m n (s i )  Classification f(s i ) → good ∨ bad  Sought-after:  Subset M* ⊆ M (including thresholds) with  f M* (s i ) ≈ f(s i ) and  | M* | is minimal Optimization of Metric Sets with Thresholds 22 11

Calculation of Thresholds  Calculate thresholds for all subsets  { m 1 } , { m 1 , m 2 } , { m 1 , m 3 } , … , { m 1 , … , m n }  2 n subsets Optimization of Metric Sets with Thresholds 23 Selection of the Best Subset  Determine classification error ε  deviation of metrics subset from input set  probability of f M* (s i ) ≠ f(s i ) (i.e., wrong classification)  Select smallest subset with sufficient ε  ε ≤ δ for a selected error limit δ  δ = 1%  increase δ by 0,5% until a subset is found Optimization of Metric Sets with Thresholds 24 12

Contents  Motivation  Software Metrics  Classification with Thresholds  Optimization of Sets of Metrics  Applications  Case Studies  Summary and Outlook Optimization of Metric Sets with Thresholds 25 Application Overview  Size reduction of sets of metrics  Higher efficiency  Simplification of classification  Better interpretation of classification  Calculation of domain specific threshold  Automated quality assessment in organizations Optimization of Metric Sets with Thresholds 26 13

Size Reduction of Sets of Metrics  Given  Set of metrics M with corresponding thresholds  Classify software by means of M  Calculate optimal subset M* ⊆ M M* is more efficient than M  Optimization of Metric Sets with Thresholds 27 Simplification of the Classification  Goal:  Using thresholds instead of a more complex classifier f complex such as  allowing certain violations of thresholds  decision trees  Classify software S with classifier f complex  Select appropriate set of metrics M  Calculation of an optimal subset M* ⊆ M Optimization of Metric Sets with Thresholds 28 14

Classification with one Violation Metric 2 Threshold 1 * Threshold 2 * * ... * 2 * * 1 Metric 1 0 ... 1 2 Optimization of Metric Sets with Thresholds 29 Approximation with Thresholds Metric 2 Threshold1 * Threshold 2 * * ... * 2 * * 1 Metric 1 0 1 ... 2 Optimization of Metric Sets with Thresholds 30 15