The Correspondence between Software Quality Models and Technical - - PowerPoint PPT Presentation

SLIDE 1

The Correspondence between Software Quality Models and Technical Debt Estimation Approaches

MTD 2014 - Victoria

1

Isaac Griffith, Derek Reimanis, Clemente Izurieta

Software Engineering Laboratory (SEL) Department of Computer Science Montana State University

Zadia Codabux, Ajay Deo, Byron Williams

Department of Computer Science and Engineering Mississippi State University

SLIDE 2

Technical Debt Estimation Approaches

SonarQube TDPlugin Not currently implemented

Marinescu’s Method

CAST’s Method Currently Implemented In Tools

Schmid’s Method

Nugroho et al’s Method

Schmid’s Method

Chin et al’s Method

SQALE Method

MTD 2014 - Victoria 2

How do each of these methods represent the relationship between Quality and Technical Debt?

SLIDE 3

Quality Models

QUAMOCO COLUMBUS SQUALE SIG Maintainability Model QMOOD

MTD 2014 - Victoria 3

SLIDE 4

Technical Debt and Quality

• We selected those methods we considered most widely

available or accessible

• and a quality model which we were sure none of the

methods employed

MTD 2014 - Victoria 4

SonarQube TDPlugin

Marinescu’s Method

CAST’s Method (3 variants)

QMOOD

?

SLIDE 5

Research Questions

1.1 What is the strength of the relationship between technical debt estimates and quality attributes? 1.2 How does the strength of each relationship compare?

H1,1: There is a relationship between each technical debt estimate and each quality attribute

MTD 2014 - Victoria 5

SLIDE 6

Research Questions

2.1 What is the estimated effect of a change in the technical debt estimate on each of the quality attributes?

H2,1: There is a relationship between each technical debt estimate and each quality attribute when accounting for differences between systems

MTD 2014 - Victoria 6

SLIDE 7

Method

MTD 2014 - Victoria 7

JEdit Apache Ant PMD Apache Collections Jruby FindBugs Apache Cayenne JFreeChart JHotDraw Proguard Infusion (Marinescu’s) SonarQube (TD Plugin) PMD/FindBugs (Cast Method) Understand

Reusability Flexibility Understandability Functionality Extendibility Effectiveness TDE1: TD Plugin TDE2a: Cast 1 TDE2b: Cast 2 TDE2c: Cast 3 TDE3: Marinescu

Analysis 10 Systems with 10 Versions each between 25 and 250 KLOC

SLIDE 8

Kendall’s τ Correlation Analysis

Quality Technical Debt Estimates

Sonar CAST 1 CAST 2 CAST 3 Marinescu

Reusability

0.7146 0.6483 0.6636 0.5059 0.2206

Flexibility

0.2538 0.1908 0.1481 0.0955 0.012

Understandability

• 0.715
• 0.658
• 0.673
• 0.506
• 0.219

Effectiveness

0.2522 0.1964 0.1554 0.0988 0.0156

Functionality

0.6846 0.6175 0.6363 0.4748 0.1948

Extendibility

0.2805 0.211 0.1805 0.1105

• 0.03

MTD 2014 - Victoria 8

SLIDE 9

Correlation Matrix

MTD 2014 - Victoria 9

TDE1

0e+00 6e+05

0.80 0.74

100000

0.62 0.27

2000

0.71

• 0.72

1000

0.68 0.25

5 15

0.28

0e+00

0.25

0e+00

TDE2a

0.87 0.55 0.22 0.65

• 0.66

0.62 0.20 0.21 0.19

TDE2b

0.58 0.17 0.66

• 0.67

0.64 0.16 0.18

0e+00 6e+05

0.15

150000

TDE2c

0.25 0.51

• 0.51

0.47 0.10 0.11 0.10

TDE3

0.22

• 0.22

0.19 0.02

• 0.03

20

0.01

3000

Reusability

• 0.99

0.96 0.27 0.33 0.26

Understan

• 0.95
• 0.27
• 0.33
• 2500
• 0.26

1500

Functiona

0.24 0.32 0.24

Effectiven

0.81

4 8

0.91

5 15

Extendibili

0.86

0e+00 8e+05 0e+00 4e+05 20 40

• 2500
• 500

4 8 12 5 10 20 5 15

Flexibility

SLIDE 10

MLR Analysis

Quality Technical Debt Estimate

Sonar CAST 1 CAST 2 CAST 3 Marinescu

Reusability Flexibility

• 1.701e-05

Understandability Effectiveness

1.508546e-13

Functionality Extendibility

MTD 2014 - Victoria 10

SLIDE 11

Threats to Validity

• Content -> QMOOD and TD Estimation

Methods

• External -> Case Study
• Construct -> QMOOD

MTD 2014 - Victoria 11

SLIDE 12

Conclusions

• Results suggest that these techniques do not

represent the relationship between quality (as measured by QMOOD) and TD

• Suggests further analysis of methods should

be conducted

• Suggests practitioners should consider

evaluating technical debt estimation techniques against their quality measurement approach

MTD 2014 - Victoria 12

SLIDE 13

Practical Implications

• Although many tools are based on the same

theoretical models, their formulaic calculations and aggregation rules are different

• Although many tools are based on ISO 9126,

many are not.

• Reporting of remediation is done in different

scales

• It is dangerous to use closed tools without

understanding the underlying quality model they implement

MTD 2014 - Victoria 13

SLIDE 14

Aspirational Goal for our Community

Can we come together and agree on a common standard for these practical models and their respective implementations?

MTD 2014 - Victoria 14

SLIDE 15

Future Work

MTD 2014 - Victoria 15 SonarQube TDPlugin

Marinescu’s Method

CAST’s Method

Schmid’s Method

SQALE Method

Nugroho et al’s Method Chin et al’s Method

COLUMBUS SQALE SIG Maintainability Model

QMOOD

?

SLIDE 16

Thank you.

Are there any questions?

MTD 2014 - Victoria 16