Identifying and Quantifying Architectural Debt Lu Xiao * , Yuanfang - - PowerPoint PPT Presentation

Lu Xiao*, Yuanfang Cai*, Rick Kazmanⱡ, Ran Mo* and Qiong Feng* *Drexel University

ⱡ University of Hawaii & SEI/CMU

Identifying and Quantifying Architectural Debt

Architectural Flaws vs. Real Debts

2

 Architectural Flaws  Architectural Debts

Architectural Debts vs. Real Debts

Debt in Life Architectural Debt

3

 Where are the debts?  How much extra

maintenance cost?

 How fast does penalty

accumulate over time (interest rate)?

Cost: 1,500$ Interest: 14.24% Cost: 20,000$ Interest: 4.46% Cost: 1000,000$ Interest: 4.75%

Problems to Solve

4

 Architectural Debt Identification  Penalty Quantification  Interest Rate Calculation

Definition

5

Architectural debt (ArchDebt): a group of architecturally connected files that incur high maintenance costs over time due to their flawed connections.

ArchDebt = < FileSetSequence, DebtModel> FileSetSequence = (FileSet1,FileSet2,…FileSet3)

#b br 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 1 ObjectHelper 60 1% (1) dp, 2 FileUtil 24 3% dp, (2) 3 DefaultExchange 17 7% dp,18 (3) dp, 4 ExchangeHelper 20 5% dp, dp,32 (4) 5 IntrospectionSupport 15 10% dp,22 (5) 6 URISupport 14 10% dp, (6) 7 EndpointHelper 12 15% dp, dp, dp, (7) 8 ExpressionBuilder 13 12% dp,26 ,18 (8) 9 OnExceptionDefinition 14 10% dp, dp, (9) ih, dp, 10 OnCompletionDefinition 11 18% (10) ih, dp, 11 RouteDefinitionHelper 10 10% dp, dp, dp, (11) dp, dp, 12 ProcessorDefinition 48 1% dp, dp, dp, dp,24 dp,24 (12) dp, 13 DefaultRouteContext 13 12% dp, (13) ag, 14 ErrorHandlerBuilderRef 12 15% dp, (14) dp, 15 RouteDefinition 17 7% dp, dp,16 ih,26 dp, dp, (15) 16 GenericFileEndpoint 20 5% dp, dp, (16) dp, dp, 17 GenericFileConsumer 23 4% dp, ag, (17) dp, 18 GenericFile 20 5% dp, dp,16 (18) 19 GenericFileProducer 15 10% dp, dp, dp, ag, ,60 (19) 20 FileOperations 19 6% dp, dp, ,36 (20) ag, 21 FileConsumer 33 2% dp, dp, dp, ih, dp,74 (21) dp, 22 FileEndpoint 14 6% dp, dp, ih, dp,30 dp, ag, dp,56 (22) 23 MulticastProcessor 58 1% dp, dp, (23) 24 Splitter 13 12% dp, dp,18 ,20 ih,58 (24) 25 XmlConverter 19 6% dp, (25) 26 XsltBuilder 17 7% dp, dp, ag, (26) #b br 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 1 ObjectHelper 60 1% (1) dp, 2 FileUtil 24 3% dp, (2) 3 DefaultExchange 17 7% dp,18 (3) dp, 4 ExchangeHelper 20 5% dp, dp,32 (4) 5 IntrospectionSupport 15 10% dp,22 (5) 6 URISupport 14 10% dp, (6) 7 EndpointHelper 12 15% dp, dp, dp, (7) 8 ExpressionBuilder 13 12% dp,26 ,18 (8) 9 OnExceptionDefinition 14 10% dp, dp, (9) ih, dp, 10 OnCompletionDefinition 11 18% (10) ih, dp, 11 RouteDefinitionHelper 10 10% dp, dp, dp, (11) dp, dp, 12 ProcessorDefinition 48 1% dp, dp, dp, dp,24 dp,24 (12) dp, 13 DefaultRouteContext 13 12% dp, (13) ag, 14 ErrorHandlerBuilderRef 12 15% dp, (14) dp, 15 RouteDefinition 17 7% dp, dp,16 ih,26 dp, dp, (15) 16 GenericFileEndpoint 20 5% dp, dp, (16) dp, dp, 17 GenericFileConsumer 23 4% dp, ag, (17) dp, 18 GenericFile 20 5% dp, dp,16 (18) 19 GenericFileProducer 15 10% dp, dp, dp, ag, ,60 (19) 20 FileOperations 19 6% dp, dp, ,36 (20) ag, 21 FileConsumer 33 2% dp, dp, dp, ih, dp,74 (21) dp, 22 FileEndpoint 14 6% dp, dp, ih, dp,30 dp, ag, dp,56 (22) 23 MulticastProcessor 58 1% dp, dp, (23) 24 Splitter 13 12% dp, dp,18 ,20 ih,58 (24) 25 XmlConverter 19 6% dp, (25) 26 XsltBuilder 17 7% dp, dp, ag, (26) #b br 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 1 ObjectHelper 60 1% (1) dp, 2 FileUtil 24 3% dp, (2) 3 DefaultExchange 17 7% dp, (3) dp, 4 ExchangeHelper 20 5% dp, dp, (4) 5 IntrospectionSupport 15 10% dp, (5) 6 URISupport 14 10% dp, (6) 7 EndpointHelper 12 15% dp, dp, dp, (7) 8 ExpressionBuilder 13 12% dp, , (8) 9 OnExceptionDefinition 14 10% dp, dp, (9) ih, dp, 10 OnCompletionDefinition 11 18% (10) ih, dp, 11 RouteDefinitionHelper 10 10% dp, dp, dp, (11) dp, dp, 12 ProcessorDefinition 48 1% dp, dp, dp, dp, dp, (12) dp, 13 DefaultRouteContext 13 12% dp, (13) ag, 14 ErrorHandlerBuilderRef 12 15% dp, (14) dp, 15 RouteDefinition 17 7% dp, dp, ih, dp, dp, (15) 16 GenericFileEndpoint 20 5% dp, dp, (16) dp, dp, 17 GenericFileConsumer 23 4% dp, ag, (17) dp, 18 GenericFile 20 5% dp, dp, (18) 19 GenericFileProducer 15 10% dp, dp, dp, ag, (19) 20 FileOperations 19 6% dp, dp, (20) ag, 21 FileConsumer 33 2% dp, dp, dp, ih, dp, (21) dp, 22 FileEndpoint 14 6% dp, dp, ih, dp, dp, ag, dp, (22) 23 MulticastProcessor 58 1% dp, dp, (23) 24 Splitter 13 12% dp, dp, ih, (24) 25 XmlConverter 19 6% dp, (25) 26 XsltBuilder 17 7% dp, dp, ag, (26)

ArchDebt Modeling

6

 Design Rule Space (DRSpace) Modeling[1]

 Model structural, historical, probability dependencies simultaneously

[1] L. Xiao, Y . Cai, and R. Kazman. Design rule spaces: A new form of architecture insight. In

• Proc. 36th International Conference on Software Engineering, 2014.

Files Files

#b br 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 1 ObjectHelper 60 1% (1) dp, 2 FileUtil 24 3% dp, (2) 3 DefaultExchange 17 7% dp, (3) dp, 4 ExchangeHelper 20 5% dp, dp, (4) 5 IntrospectionSupport 15 10% dp, (5) 6 URISupport 14 10% dp, (6) 7 EndpointHelper 12 15% dp, dp, dp, (7) 8 ExpressionBuilder 13 12% dp, , (8) 9 OnExceptionDefinition 14 10% dp, dp, (9) ih, dp, 10 OnCompletionDefinition 11 18% (10) ih, dp, 11 RouteDefinitionHelper 10 10% dp, dp, dp, (11) dp, dp, 12 ProcessorDefinition 48 1% dp, dp, dp, dp, dp, (12) dp, 13 DefaultRouteContext 13 12% dp, (13) ag, 14 ErrorHandlerBuilderRef 12 15% dp, (14) dp, 15 RouteDefinition 17 7% dp, dp, ih, dp, dp, (15) 16 GenericFileEndpoint 20 5% dp, dp, (16) dp, dp, 17 GenericFileConsumer 23 4% dp, ag, (17) dp, 18 GenericFile 20 5% dp, dp, (18) 19 GenericFileProducer 15 10% dp, dp, dp, ag, (19) 20 FileOperations 19 6% dp, dp, (20) ag, 21 FileConsumer 33 2% dp, dp, dp, ih, dp, (21) dp, 22 FileEndpoint 14 6% dp, dp, ih, dp, dp, ag, dp, (22) 23 MulticastProcessor 58 1% dp, dp, (23) 24 Splitter 13 12% dp, dp, ih, (24) 25 XmlConverter 19 6% dp, (25) 26 XsltBuilder 17 7% dp, dp, ag, (26)

The largest 5 spaces usually capture 50% to 95%

• f all the error-prone files

7

(1) Architecturally connected file group with high maintenance cost.

R-2.0.0, 11Files, 392 lines

A Sample ArchDebt (Camel)

1 2 3 4 5 6 7 8 9 10 11 1 ProcessorDefinition (1) dp, dp, dp, dp, dp, dp, dp,dp, dp, dp, dp, 2 LoadBalanceDefinition Extend,dp,100% (2) dp, 3 ChoiceDefinition Extend,dp,100% (3) dp, ,100% 4 RollbackDefinition Extend,dp,100% (4) dp, 5 RouteContext dp,67% (5) ,33% ,67% ,33% ,33% dp,33% 6 MarshalDefinition dp,100% dp,67% (6) ,100% ,100% ,50% ,100% 7 PolicyDefinition dp,67% dp,44% ,33% (7) ,33% ,33% ,33% 8 TryDefinition dp,100% ,100% dp, (8) 9 UnmarshalDefinition dp,100% dp,67% ,100% ,100% (9) ,50% ,100% 10 ErrorHandlerBuilderRef dp,50% dp, ,33% (10) 11 InterceptStrategy dp,50% ,33% ,50% ,50% ,50% (11) 1 2 3 4 5 6 7 8 9 10 11 1 ProcessorDefinition (1) dp, dp, dp, dp, dp, dp, dp,dp, dp, dp, dp, 2 LoadBalanceDefinition Extend,dp,100% (2) dp, 3 ChoiceDefinition Extend,dp,100% (3) dp, ,100% 4 RollbackDefinition Extend,dp,100% (4) dp, 5 RouteContext dp,67% (5) ,33% ,67% ,33% ,33% dp,33% 6 MarshalDefinition dp,100% dp,67% (6) ,100% ,100% ,50% ,100% 7 PolicyDefinition dp,67% dp,44% ,33% (7) ,33% ,33% ,33% 8 TryDefinition dp,100% ,100% dp, (8) 9 UnmarshalDefinition dp,100% dp,67% ,100% ,100% (9) ,50% ,100% 10 ErrorHandlerBuilderRef dp,50% dp, ,33% (10) 11 InterceptStrategy dp,50% ,33% ,50% ,50% ,50% (11) 1 2 3 4 5 6 7 8 9 10 11 1 ProcessorDefinition (1) dp, dp, dp, dp, dp, dp, dp,dp, dp, dp, dp, 2 LoadBalanceDefinition Extend,dp,100% (2) dp, 3 ChoiceDefinition Extend,dp,100% (3) dp, ,100% 4 RollbackDefinition Extend,dp,100% (4) dp, 5 RouteContext dp,67% (5) ,33% ,67% ,33% ,33% dp,33% 6 MarshalDefinition dp,100% dp,67% (6) ,100% ,100% ,50% ,100% 7 PolicyDefinition dp,67% dp,44% ,33% (7) ,33% ,33% ,33% 8 TryDefinition dp,100% ,100% dp, (8) 9 UnmarshalDefinition dp,100% dp,67% ,100% ,100% (9) ,50% ,100% 10 ErrorHandlerBuilderRef dp,50% dp, ,33% (10) 11 InterceptStrategy dp,50% ,33% ,50% ,50% ,50% (11)

A Sample ArchDebt (Camel)

8

(2) Accumulated penalty over time.

R-2.12.4, 28 Files, +1742 lines R-2.2.0, 20 Files, +379 lines R-2.0.0, 11 Files, 392 lines

1 2 3 4 5 6 7 8 9 10 11 1 ProcessorDefinition (1) dp, dp, dp, dp, dp, dp, dp,dp, dp, dp, dp, 2 LoadBalanceDefinition Extend,dp,100% (2) dp, 3 ChoiceDefinition Extend,dp,100% (3) dp, ,100% 4 RollbackDefinition Extend,dp,100% (4) dp, 5 RouteContext dp,67% (5) ,33% ,67% ,33% ,33% dp,33% 6 MarshalDefinition dp,100% dp,67% (6) ,100% ,100% ,50% ,100% 7 PolicyDefinition dp,67% dp,44% ,33% (7) ,33% ,33% ,33% 8 TryDefinition dp,100% ,100% dp, (8) 9 UnmarshalDefinition dp,100% dp,67% ,100% ,100% (9) ,50% ,100% 10 ErrorHandlerBuilderRef dp,50% dp, ,33% (10) 11 InterceptStrategy dp,50% ,33% ,50% ,50% ,50% (11)

Architectural Debt Identification

9

ArchDebt Localization

10

 Four architectural flaw patterns

Comprehensively capture all possibilities of how errors propagate.

ArchDebt Localization (1)

11

 Four architectural flaw patterns

(1) Anchor Hub Pattern

“Spaghetti code", or “big ball of mud".

ArchDebt Localization (2)

12

 Four architectural flaw patterns

(2) Anchor Submissive Pattern

An unstable interface changes with its dependents.

 Four architectural flaw patterns

(3) Anchor Dominant Pattern

An unstable interface propagates changes to its dependents.

ArchDebt Localization (3)

13

 Four architectural flaw patterns

(4) Modularity Violation[2] Pattern

Undocumented “secrets” among files.

ArchDebt Localization (4)

14 [2] S. Wong, Y . Cai, M. Kim, and M. Dalton, “Detecting software modularity violations,” proceedings of the 33rd International Conference on Software Engineering, pp. 411-420, 2011.

Penalty Quantification

15

Quantify Penalty

16

 The actual cost:  Penalty approximation

The lines of code modified on the files to fix bugs involved in an ArchDebt.

Subjects

17

 7 Apache open source projects  311 to 9866 java files  4 to 8 years of revision history  2005 to 14673 commits  1857 to 6280 bug tickets  9 to 15 stable releases

ArchDebts Incur Significant Penalty

18

 51% to 85% effort is consumed by ArchDebts.  Top 5 ArchDebts (11% to 32% of files) consume 27% to 67% of effort.  Undocumented “secrets” are the most expensive debts.

Interest Rate Calculation

19

Interest Rate of ArchDebts

20

 Four types of regression models

T: ordinal number of releases Cost(T): bug fixing lines of code at T (1) Linear Regression Model - Stable Interest Rate

Interest Rate of ArchDebts

21

 Four types of regression models

T: ordinal number of releases Cost(T): bug fixing lines of code at T (2) Logarithmic Regression Model – Decreasing Interest Rate

Interest Rate of ArchDebts

22

 Four types of regression models

T: ordinal number of releases Cost(T): bug fixing lines of code at T (3) Exponential Regression Model – Increasing Interest Rate

Interest Rate of ArchDebts

23

 Four types of regression models

T: ordinal number of releases Cost(T): bug fixing lines of code at T (4) Polynomial Regression Model – Fluctuating Interest Rate

Sample ArchDebt Interest Rate

24

R-2.0.0, Age 1, 11Files, 392 lines R-2.2.0, Age 2, 20 Files, 771 lines R-2.12.4, Age 11, 28 Files, 2134 lines

 DebtModel(T) =

* T + 509.35 158.75

Cost Regression Model

25

 Distribution of interest rates for ArchDebts

Support decision making: Whether, when, and where to refactor?

Linear 53% Log 14% Exp 8% Poly 25%

Summary

26

 Architectural Debt Identification

 Locate ArchDebts using 4 patterns

 Penalty Quantification

 Approximate penalty using LOC for bug fixing

 Interest Rate Calculation

 Model interest rate using regression models

Thank you! Q/A

1 2 3 4 5 6 7 8 9 10 1 ColumnFamilyStore (1) dp dp dp dp dp dp dp dp 2 SSTableReader (2) dp 3 SecondaryIndex dp dp (3) 4 AbstractCompactionStrategy dp (4) 5 WrappingCompactionStrategy dp dp Extend,dp (5) 6 Memtable dp (6) 7 DataTracker dp dp (7) 8 ColumnFamilyMetrics dp (8) 9 CacheService dp dp (9) 10 ExtendedFilter dp (10)

ArchDebt Modeling

28

 Design Rule Space (DRSpace) Modeling[1]

 Model structural, historical, probability dependencies simultaneously

[1] L. Xiao, Y . Cai, and R. Kazman. Design rule spaces: A new form of architecture insight. In

• Proc. 36th International Conference on Software Engineering, 2014.

Files Files

1 2 3 4 5 6 7 8 9 10 1 ColumnFamilyStore (1) dp dp dp dp dp dp dp dp 2 SSTableReader (2) dp 3 SecondaryIndex dp dp (3) 4 AbstractCompactionStrategy dp (4) 5 WrappingCompactionStrategy dp dp Extend,dp (5) 6 Memtable dp (6) 7 DataTracker dp dp (7) 8 ColumnFamilyMetrics dp (8) 9 CacheService dp dp (9) 10 ExtendedFilter dp (10) 1 2 3 4 5 6 7 8 9 10 1 ColumnFamilyStore (1) dp,21 dp,4 dp,7 dp, 1 dp,13 dp, 9 dp,2 dp,3 ,2 2 SSTableReader ,21 (2) dp,1 ,5 ,1 ,5 ,6 ,2 3 SecondaryIndex dp, 4 dp,1 (3) ,1 ,1 ,1 4 AbstractCompactionStrategy ,7 dp,5 (4) ,1 ,3 5 WrappingCompactionStrategy dp,1 dp,1 Extend,dp,1 (5) 6 Memtable dp,13 ,5 ,1 (6) ,4 7 DataTracker 9 dp,6 ,1 ,3 dp,4 (7) ,1 8 ColumnFamilyMetrics dp,2 ,1 (8) 9 CacheService dp,3 dp,2 (9) 10 ExtendedFilter dp,2 ,1 (10)

Cyclic Dependency

1 2 3 4 5 6 7 8 9 10 1 ColumnFamilyStore (1) dp dp dp dp dp dp dp dp 2 SSTableReader ,32% (2) dp, 3 SecondaryIndex dp, 40% dp (3) 4 AbstractCompactionStrategy ,37% dp (4) 5 WrappingCompactionStrategy dp,100% dp,100% Extend,dp,100% (5) 6 Memtable dp,30% (6) 7 DataTracker 31% dp dp (7) 8 ColumnFamilyMetrics dp,57% (8) 9 CacheService dp,38% dp (9) 10 ExtendedFilter dp,33% (10)