LECTURE 28: METRICS! CSE 442 – Software Engineering
Putting the E in SE ¨ Have mostly focused on creating software
Putting the E in SE ¨ Have mostly focused on creating software ¤ Team Dynamics
Putting the E in SE ¨ Have mostly focused on creating software ¤ Team Dynamics ¤ Management
Putting the E in SE ¨ Have mostly focused on creating software ¤ Team Dynamics ¤ Management ¤ Software Lifecycle
Putting the E in SE ¨ Have mostly focused on creating software ¤ Team Dynamics You want what? ¤ Management ¤ Software Lifecycle ¤ Requirements
Putting the E in SE ¨ Have mostly focused on creating software ¤ Team Dynamics You want what? ¤ Management ¤ Software Lifecycle ¤ Requirements ¤ Design
Putting the E in SE ¨ Have mostly focused on creating software ¤ Team Dynamics ¤ Management ¤ Software Lifecycle ¤ Requirements ¤ Design ¤ Testing
Putting the E in SE ¨ Have mostly focused on creating software ¤ Team Dynamics ¤ Management ¤ Software Lifecycle ¤ Requirements ¤ Design ¤ Testing ¤ Implementation
Engineering Requirements ¨ Demands attention to details, rules, precision…
Engineering Requirements ¨ Demands attention to details, rules, precision… … and numbers
Engineering Requirements ¨ Demands attention to details, rules, precision… … and numbers ¨ Results used to answer many types of questions: ¤ How much … ? ¤ How often … ? ¤ How many… ? ¤ Why?
Metrics ¨ Provide data which drives process of engineering ¤ Allow us to make changes and evaluate results ¤ Apply scientific process to improve process & results
Why We Use Metrics ¨ Stop making all our decisions using intuition
Why We Use Metrics ¨ Stop making all our decisions using intuition to process relying on facts & evidence
Numeric ≠ Useful ¨ Need solid basis by which number generated ¨ Number should accurately reflect reality
Metric Problems There are 3 kinds of lies: 1. Lies 2. Damned Lies 3. Statistics
Metric Problems ¨ Misused metrics no better than random guess ¤ Much, much worse in most situations ¤ People’s faith & trust replaced by doubt ¨ Correlation is not causation – cannot forget this
Statistics Abuse ¨ Fact: People with larger shoe sizes better spellers
Statistics Abuse ¨ Fact: People with larger shoe sizes better spellers
Statistics Abuse ¨ Fact: People with larger shoe sizes better spellers Adults good at spelling; Also wear grown-up shoes
Statistics Abuse ¨ Fact: People with larger shoe sizes better spellers Adults good at spelling; Kids bad at spelling; Also wear grown-up shoes Small feet use kids' sizes
Why He Deserves a Raise I wrote more code last year than any other employee
And The Boss Replies…
And The Boss Replies… Measuring programming progress by lines of code is like measuring aircraft building progress by weight
Lines Of Code ¨ Of these lines, which each equal to 1 LoC? /* Some of these lines are really ugly */ int countMatches(E[] arr, E ignore) { int sum = (2 * 4) – (int)Math.sqrt(64); for (int i = 0; i < arr.length; i++) { E val = arr[i]; if (val.equals(ignore)) ; else { sum++; } } return sum; }
What Are You Measuring?
What Are You Measuring? ¨ Are you sure?
Good Metric Example ¨ Cyclomatic complexity quick & easy to measure ¤ Equal to number of simple decisions + 1 ¤ || and && count also simple decisions 1 ¨ Complexity is 4 for this example ¤ Written as V(G) = 4 2 ¨ Provides deeper insight 3 ¤ Counts different ways to screw up
Good Metric Example ¨ Cyclomatic complexity quick & easy to measure ¤ Equal to number of simple decisions + 1 ¤ || and && count also simple decisions 1 ¨ Complexity is 4 for this example ¤ Written as V(G) = 4 2 ¨ Provides deeper insight 3 ¤ Counts different ways to screw up
Good Metric Example ¨ Cyclomatic complexity quick & easy to measure ¤ Equal to number of simple decisions + 1 ¤ || and && count also simple decisions 1 ¨ Complexity is 4 for this example ¤ Written as V(G) = 4 2 ¨ Provides deeper insight 3 ¤ Counts different ways to screw up
Good Metric Example ¨ Cyclomatic complexity quick & easy to measure ¤ Equal to number of simple decisions + 1 ¤ || and && count also simple decisions 1 ¨ Complexity is 4 for this example ¤ Written as V(G) = 4 2 ¨ Provides deeper insight 3 ¤ Counts different ways to screw up
What Makes it Good? ¨ Studies found V(G) correlated to error probability ¤ Opportunities for bugs much, much larger ¨ Inside complex methods find far more errors 20% of modules, but … # of 80% of bugs Modules V(G)
Is It Enough? ¨ Probability of bugs jumps once V(G) gets too large ¤ Often suggest rewrite methods w/ complexity > 7 ¤ V(G) > 9 demand rewrite or split into pieces ¨ Possible strategy: test complex methods only?
Ego Check ¨ Who is the best programmer in the class?
Ego Check ¨ Who is the best programmer in the class? ¤ How can you prove it?
Real-World Questions ¨ Have 11 employees; but can only promote 3
Real-World Questions ¨ Have 11 employees; but can only promote 3 ¤ Whom do you promote?
Real-World Questions ¨ Have 11 employees; but can only promote 3 ¤ Whom do you promote? ¤ How will you explain your choice in court?
Real-World Questions ¨ Changed the application lifecycle used
Real-World Questions ¨ Changed the application lifecycle used ¤ 1 st project sucked: client was an
Real-World Questions ¨ Changed the application lifecycle used ¤ 1 st project sucked: client was an ¤ How do you justify your decision to your boss?
Singular of Data: Anecdote ¨ Single project cannot provide usable metrics ¤ Too many variables: must isolate cause-and-effect ¨ Results should converge given enough data ¤ If no convergence, results may be random ¤ May take time before convergence occurs
Singular of Data: Anecdote ¨ Important to track metrics for large data set ¤ Many coders on single project ¤ Track group over long period of time ¨ Be careful evaluating metrics ¤ Limit evaluation to trends ¤ Smooth results by using moving averages
Measurement Principles ¨ Define objectives first, collect data second ¤ Will always find patterns in data ¤ Simple coincidences should not be relied upon
Measurement Principles ¨ Select metrics using theory valid for domain ¤ Analyze designs with design concepts not coding ¤ Verify positive results would be found
Measurement Principles ¨ Tailor uses & analysis to answer your question ¤ If metric serves no purpose, do not collect that data ¤ Generate results you care about & not any others ¤ Check & double-check this does not bias results
Once Analysis Is Complete ¨ Interpretation ¤ Look for insights about process ¤ Check how well/if metric represents the results ¤ Secondary effects of the process should be evaluated ¨ Answer the question: What do these results mean?
Once Analysis Is Complete ¨ Interpretation ¤ Look for insights about process ¤ Check how well/if metric represents the results ¤ Secondary effects of the process should be evaluated ¨ Answer the question: What do these results mean?
Most Important Step ¨ Feedback ¤ Recommend changes based upon the results ¤ Clarify confusing results by proposing additional experiments ¤ Verify work as good as possible and collect the big money
Most Important Step ¨ Feedback ¤ Recommend changes based upon the results ¤ Clarify confusing results by proposing additional experiments ¤ Verify work as good as possible and collect the big money
Most Important Step ¨ Feedback ¤ Recommend changes based upon the results ¤ Clarify confusing results by proposing additional experiments ¤ Verify work as good as possible and collect the big money
How Much To Collect? X
Class-Oriented Metrics ¨ Size of each class ¨ Average method size ¨ Method complexity ¨ Weighted methods per class ¨ Average number of parameters per method
Class-Oriented Metrics ¨ Number of subclasses ¨ Depth of the inheritance tree ¨ Number of overridden operations ¨ Number of added operations in subclass ¨ Specialization index
Recommend
More recommend
