SLIDE 1
The Metrics Trap
Michael Feathers
The Metrics Trap Michael Feathers We all know fold foldl1 (+) - - PowerPoint PPT Presentation
The Metrics Trap Michael Feathers We all know fold foldl1 (+) - - PowerPoint PPT Presentation
The Metrics Trap Michael Feathers We all know fold foldl1 (+) [1..5] foldl1 (+) [1..5] 15 We all know Average data Pair = Pair !Int !Double average :: Vector Double -> Double average xs = s / fromIntegral n where Pair n s =
SLIDE 2
SLIDE 3
foldl1 (+) [1..5]
SLIDE 4
foldl1 (+) [1..5] 15
SLIDE 5
We all know Average
SLIDE 6
data Pair = Pair !Int !Double average :: Vector Double -> Double average xs = s / fromIntegral n where Pair n s = foldl' k (Pair 0 0) xs k (Pair n s) x = Pair (n+1) (s+x)
SLIDE 7
data Pair = Pair !Int !Double average :: Vector Double -> Double average xs = s / fromIntegral n where Pair n s = foldl' k (Pair 0 0) xs k (Pair n s) x = Pair (n+1) (s+x)
SLIDE 8
Standard ways of reducing information
SLIDE 9
(average [1, 1, 1, 1, 1, 1]) == (average [11, -9, 12, -10])
SLIDE 10
It turns out that we do this all the time
SLIDE 11
SLIDE 12
SLIDE 13
SLIDE 14
Quantitative Bias In every use of units, there’s an assumption
- f uniformity
SLIDE 15
Quantitative Bias In every use of units, there’s an assumption
- f uniformity
1 can be exchanged for 1
SLIDE 16
“We want to establish key metrics and indicators to measure our progress toward the goal.”
SLIDE 17
“We want to establish key metrics and indicators to measure our progress toward the goal.”
SLIDE 18
But..
SLIDE 19
SLIDE 20
We don’t have (m)any uniform distributions in software
SLIDE 21
SLIDE 22
SLIDE 23
Distortion via Metrics
SLIDE 24
There is no “right” number for method size, class size, amount of complexity, etc
SLIDE 25
Why do we persist in our reductionism?
SLIDE 26
Maybe it is because we can’t think of anything better
SLIDE 27
“We want to establish key metrics and indicators to measure our progress toward the goal.”
SLIDE 28
SLIDE 29
“Laws” of Metrics
SLIDE 30
“Laws” of Metrics
- 1. Distance Causes Misunderstanding
SLIDE 31
“Laws” of Metrics
- 1. Distance Causes Misunderstanding
- 2. Highlighting Leads to Focus
SLIDE 32
“Laws” of Metrics
- 1. Distance Causes Misunderstanding
- 2. Highlighting Leads to Focus
- 3. Focus Leads to Action
SLIDE 33
“Laws” of Metrics
- 1. Distance Causes Misunderstanding
- 2. Highlighting Leads to Focus
- 3. Focus Leads to Action
- 4. Focus Leads to Side-Effects
SLIDE 34
Death of Locality
SLIDE 35
Practice Use Qualitative “Measures” when Possible
SLIDE 36
SLIDE 37
Practice Silent Alarms
SLIDE 38
Silent Alarms
Don’t have check-in gates. Let people make mistakes. Investigate the mistakes off-line and see why they
- happened. Then, intervene
SLIDE 39
Practice
Disposable Metrics
SLIDE 40
Disposable Metrics
Secondary effects are less likely when metrics come and go. Use them to highlight concerns
SLIDE 41
Practice Targeted Metrics
SLIDE 42
Targeted Metric - Feature Trend Cards
Hypothesize a couple of features that you will never add to your code. Task them and estimate them
- periodically. See the debt trend for areas they touch.
SLIDE 43
Practice Deluge with Metrics
SLIDE 44
Deluge
The more metrics you have, the harder it is to take any one of them too seriously. They become “vital signs” and indicators as we have in medicine.
SLIDE 45
Temporal Correlation of Class Changes
SLIDE 46
SLIDE 47
Active Set of Classes
SLIDE 48
Active Set of Classes
SLIDE 49
Active Set of Classes
SLIDE 50
Vital Signs
SLIDE 51