Replication and Robust Results Jim Herbsleb School of Computer - - PowerPoint PPT Presentation
Replication and Robust Results Jim Herbsleb School of Computer Science Carnegie Mellon University jdh@cs.cmu.edu http://conway.isri.cmu.edu/~jdh/ Science Is Based on a Peculiar Logic Experimental method Relationship => hypothesis
Jim Herbsleb
School of Computer Science Carnegie Mellon University jdh@cs.cmu.edu http://conway.isri.cmu.edu/~jdh/
10 20 30 multi site single site 12.7 4.9
Network Element A
Work Days
Last Modification - First Modification All changes for 2-year period
Modification Request (MR) interval
Herbsleb, J.D. & Mockus, A. (2003). An Empirical Study of Speed and Communication in Globally- Distributed Software Development. IEEE Transactions on Software Engineering, 29, 3, pp. 1-14.
7
Variable Measure used in models MR interval Log of number of days, first delta to last delta Number of people Log of number of people Diffusion Log of number of modules touched by change Size Log of number of delta Time Date Severity Is high severity Fix Is fix Multi-site Set of sites of all actors has more than one element
Multi-site Size Number of People Work Interval Diffusion
Graphical model of work interval for Network Element A
148.9 0.25 Multi-site Size 199.7 0.27 154.1 0.24 35.9 0.12 Number of People Work Interval Diffusion
148.9 0.25 Multi-site Size 199.7 0.27 154.1 0.24 35.9 0.12 Number of People Work Interval Diffusion
148.9 0.25 Multi-site Size 199.7 0.27 154.1 0.24 35.9 0.12 Number of People Work Interval Diffusion
148.9 0.25 Multi-site Size 199.7 0.27 154.1 0.24 35.9 0.12 Number of People Work Interval Diffusion
10 20 30 multi site single site 12.7 4.9 18.1 6.9
Network Element A Network Element B
Work Days
Last Modification - First Modification All changes for 2-year period
Modification Request (MR) interval
Herbsleb, J.D. & Mockus, A. (2003). An Empirical Study of Speed and Communication in Globally- Distributed Software Development. IEEE Transactions on Software Engineering, 29, 3, pp. 1-14.
Graphical model of work interval for Network Element A (left) and B (right)
148.9 0.25 Multi-site Size 199.7 0.27 154.1 0.24 35.9 0.12 Number of People Work Interval Diffusion
96.2
2009.7 0.55 566.8 0.25 701.7 0.34 Number of People Work Interval Diffusion Multi-site Size 148.9 0.25 Multi-site Size 199.7 0.27 154.1 0.24 35.9 0.12 Number of People Work Interval Diffusion
Graphical model of work interval for Network Element A (left) and B (right)
Graphical model of work interval for Network Element A (left) and B (right)
Closer More differentiated Amount Of Learning Same result: Robust effect Different result: Many possible causes Original result Was anomalous Effect unlikely To be anomalous