Empirical analysis of the relationship between CC and SLOC in a - PowerPoint PPT Presentation

Empirical analysis of the relationship between CC and SLOC in a large corpus of Java methods Davy Landman Alexander Serebrenik Jurgen Vinju

Metrics • Lines of Code (SLOC) • Cyclomatic Complexity (CC) • Popular in practice and research

Metrics • Lines of Code (SLOC) = 7 • Cyclomatic Complexity (CC) = 2 public ¡ double ¡sqrt( int ¡n){ ¡ 1 ¡ ¡ ¡ ¡ ¡// ¡Newton-­‑Raphson ¡method ¡ ¡ ¡ ¡ ¡ ¡ double ¡r ¡= ¡n ¡/ ¡2.0; ¡ 2 ¡ ¡ ¡ ¡ ¡ while ¡(abs(r ¡– ¡(n ¡/ ¡r)) ¡> ¡0.00001) ¡{ ¡ 3 ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡r ¡= ¡0.5 ¡* ¡(r ¡+ ¡(n ¡/ ¡r)); ¡ 4 ¡ ¡ ¡ ¡ ¡} ¡ 5 ¡ ¡ ¡ ¡ ¡ return ¡r; ¡ 6 ¡ } ¡ 7 ¡

M. Shepperd. "A critique of cyclomatic complexity as a software metric." Software Engineering Journal 3.2 (1988)

Citations Total 218 Last 5 years 90

CC redundant? • Shepperd’s was based on 8 papers (1979-1987) • 7 papers followed (1991-2013) • Fortran, PL/1, Pascal, COBOL, C, C++, and Java • SLOC & CC correlate linearly R 2 = 0.65 - 0.95

Our research • Identify di ff erences in 15 papers • Get data • Reproduce!

we do not conclude that CC is redundant with SLOC • Our result: R 2 = 0.43 • Di ff erence related work: • Aggregation • Power transform • Larger methods correlate even less • Di ff ering variance

Corpus • 13K Open Source Java Projects (14GB of Java) • 17M methods in 362M SLOC 1e+07 1e+07 1e+05 1e+05 Frequency Frequency 1e+03 1e+03 1e+01 1e+01 1 10 100 100 1000 10000 1 10 100 100 1000 SLOC of a Method CC of a Method E. Linstead, S. K. Bajracharya, T. C. Ngo, P. Rigor, C. V. Lopes, and P. Baldi, “Sourcerer: mining and searching internet-scale software repositories,” Data Mining and Knowledge Discovery, 18.2 (2009).

First result • Correlation ( R 2 ) : 0.43 • Lower than other papers: 0.65 - 0.95 • Why?

Other explanations • Correlation ( R 2 ) : 0.43 • Lower than other papers: 0.65 - 0.95 Yes No Power transform 4 12 File level (sum) 9 6

Power transform 8e+06 1e+07 6e+06 1e+05 Frequency Frequency 4e+06 1e+03 2e+06 1e+01 0e+00 0 1 50 10 100 100 100 150 1000 200 10000 250 SLOC of a Method SLOC of a Method

Method level R 2 = 0.43 R 2 = 0.70

File level • Example: 1 fj le, 30 “small” methods. • File SLOC = 30 * avg(SLOC m ) = 30 * 2.5 • File CC = 30 * avg(CC m ) = 30 * 2 • Volume factor causes high correlation [1] [1] K. El Emam, S. Benlarbi, N. Goel, S.N. Rai. "The confounding e ff ect of class size on the validity of object-oriented metrics." IEEE Transactions on Software Engineering 27.7 (2001)

File level R 2 = 0.87 R 2 = 0.65 Aggrega&on ¡causing ¡it? ¡

we do not conclude that CC is redundant with SLOC • Our result: R 2 = 0.43 • Di ff erence related work: • Aggregation • Power transform • Larger methods correlate even less • Di ff ering variance

1e+07 50% 25% 10% 1% 0.1% 1e+05 Frequency 1e+03 1e+01 1 10 100 100 1000 10000 SLOC of a Method Israel Herraiz and Ahmed E. Hassan, “Beyond lines of code: Do we need more complexity metrics?” Making Software What Really Works, and Why We Believe It. (2010)

Statistics R 2 “power” R 2 Tail min. SLOC # Methods 100% 1 17.8M 0.43 0.70 50% 3 8.9M 0.45 0.62 25% 9 4.5M 0.42 0.44 10% 20 1.8M 0.38 0.27 1% 77 179K 0.29 0.05 0.1% 230 18K 0.21 0.00

Large Methods

we do not conclude that CC is redundant with SLOC • Our result: R 2 = 0.43 • Di ff erence related work: • Aggregation • Power transform • Larger methods correlate even less • Di ff ering variance

Variance • R 2 = 0.43 means 57% variance not explained • Variance = actual CC – predicted CC

Method level

log 10 (Method level) Method level

log 10 (Method level) Method level File level

log 10 (Method level) log 10 (File level) Method level File level

Di ff ering variance complicate interpretation of linear models

we do not conclude that CC is redundant with SLOC • Our result: R 2 = 0.43 • Di ff erence related work: • Aggregation • Power transform • Larger methods correlate even less • Di ff ering variance

Method Level File Level Summary Large Methods Di ff ering variance ( data, scripts & preprint: http://is.gd/icsme_cc )