Mining Version Histories to Guide Software Changes by T. Zimmermann, - - PowerPoint PPT Presentation

Mining Version Histories to Guide Software Changes

by T. Zimmermann, P. Weißgerber, S. Diehl, A. Zeller

in IEEE Transaction on Software Engineering,

• Vol. 31, No. 6., June 2005

The Idea

Can we make similar suggestions for software changes?

Extending Eclipse Preferences

 Extend Eclipse IDE with a new preference  Preferences are stored in a field fKeys[]

Extending Eclipse Preferences

 What else do you need to change?

 Which of the 27,000 files?  Which of the 20,000 classes?  Which of the 200,000 methods?

 Program analysis

 fKeys[] and initDefaults() use the same variables  Usage does not induce change  Usage can be detected only within the source code

 Eclipse has 12,000 non-Java files

Learning from History

 Programmer who changed fKeys[] also changed …

From CVS to Transactions

 The CVS archive for Eclipse has more than 47,000

transactions

ROSE in a Nutshell

Changes -> Transactions -> Rules



Entity – a triple (c, i, p),

 where c – syntactic category; i – identifier; p – parent entity



Example: (method, initDefaults(), (class, Comp, (file, Comp.java, …))



Operations on entities: add_to, del_from, alter



Transaction – the set of changes simultaneously submitted by a developer to a version archive

Getting Syntactic Entities

Light-Weight Analysis with ROSE

Light-Weight Analysis with ROSE

Rose analyzes C/C++, JAVA, PYTHON, TEX and TEXINFO files We get modified methods, variables and subsections

Changes -> Transactions -> Rules

 ROSE retrieves changes and transactions from CVS

[Berliner’90]

 CVS provides only file versioning  Per-file changes are grouped into transactions

 Files -> Transactions -> Sliding window approach

[Fogel’02]

 Two subsequent changes, the same author, 200 second apart

 Branches and Merges in CVS

 Rose ignores changes that affect more than 30 entities

Changes -> Transactions -> Rules

 Rules are mined from transactions  Rules are mined with Apriori Algorithm [Agrawal’94]  The generated rules have the form:



antecedent(s) => consequent (s)

 The rules have a probabilistic interpretation



Evidence: support count (# of transactions) and confidence (the strength of the correspondence)

Evolutionary Coupling

Evolutionary Coupling

Evolutionary Coupling

Support: How much evidence (= simultaneous changes)? Confidence: How much relevant is coupling for participants?

Evolutionary Coupling

Support: How much evidence (= simultaneous changes)? Confidence: How much relevant is coupling for participants?

Applying Rules

 The programmer performs a change – “a situation”:  ROSE suggests further changes by applying matching

rules

 Matching rule = situation = antecedent

 The suggestion = union of the consequents of all the

matching rules

 The # of rules depends on support count and

confidence

Multi-Dimensional Rules

 If something is added to software, there is no way to

predict the change based on history

 E.g., the developer adds “Foo” constant to Comp.java  ROSE can do that in “operation” dimension

Examples of Rules

 GCC arrays that define the cost of different assembler

• perations for INTEL CPUs

 The arrays have been altered 9 times; 9 out of 11 times,

the change is triggered by a change in the type:

Examples of Rules

 Python and C files – detecting evolutionary couplings

in different programming languages

 It would require cross-language program analysis to

detect this coupling

Examples of Rules

 POSTGRES documentation

ROSE Server and Client

 The ROSE server determines coupling and rules  The ROSE client guides the programmer along related

changes

Evaluation

 How good are rules at predicting changes?  Training period: ROSE infers rules from the past  Evaluation period: ROSE applies the mined rules  In evaluation period, every transaction T is checked:

 Navigation: given one change in T, does ROSE point to

further changes in T?

 Error Prevention: given all but one change from T, does

ROSE point to the missing change?

 Closure: given all changes of T, does ROSE stay silent?

Evaluating Additional Questions

 Granularity

 Files and functions

 Maintenance

 No addition or deletions

 Multiple Dimensions

 What is the benefit of add_to and del_from?

 History

 How much history? Usefulness over time? Quality or

recommendations depending of the development cycle and releases?

 Recent Changes

 Relevance of old changes

Projects Used for Evaluation

Precision vs. Recall

 Recall: How many relevant entities are returned?  Precision: How many of the returned entities are

relevant?

Precision vs. Feedback / Support Count vs. Confidence

Results: Navigation, Prevention, Closure

Navigation, Prevention, Closure

 The programmer has changed one single entity. Can

ROSE suggest other entities that should be changed?

 The programmer has changed several entities but one.

Does ROSE find the missing one?

 The programmer made all necessary changes. How often

does ROSE still suggest a missing change?

Results for Fine Granularity

Results: Navigation

 Given one initial item, ROSE makes predictions in 66

percent of all queries

 On average, the predictions contain 33 percent of all

items changed

 For those queries for which ROSE makes

recommendations, in 7 percent of the cases, a correct location is within ROSE’s topmost three suggestions

Results: Prevention and Closure

 In 3 percent of the queries where one item is missing,

ROSE issues a correct warning

 A warning predicts 75 percent of the items that need to

be changed

 ROSE’s warning about missing items should be taken

seriously …

 Only 2 percent of all transactions cause a false alarm (!)

Results for Coarse Granularity

Results for Maintenance

 Rose shows the best predictive power for

changes to existing entities

Threads to Validity

 Kinds of version histories and software projects

 8 projects; 100,000 transactions

 Transactions do not record the order

 CVS limitation

 Quality of transactions?  User studies?

Summary



For stable systems like GCC, ROSE gives precise suggestions (recommendation in 63% of transactions, precision – 30%, in 90%

• f all recommendations – 3 topmost suggestions contain correct

entity)



For rapidly changing systems like KOFFICE, most useful suggestions are at the file level (because prediction new functions –

• ut of reach for any approach)



Predictive power of ROSE is best during maintenance phases



In about 2-7% of all erroneous transactions, ROSE correctly detects the missing change (only 2% of all transactions cause false alarm)



ROSE detects coupling between non-program entities (e.g. docs, manuals, mappings)

Future Work

 Taxonomies: identify patterns of changes  Sequence rules: detect rules across multiple

transactions

 Further data sources: log messages, bug databases  Refactoring: ROSE does not recognize renamings of

methods or files

 Program analysis: can improve the overall approach  Rule presentation: visualization of rules can help

Downloads

ROSE is publicly available as a plug-in for ECLIPSE For details and downloads visit http://www.st.cs.uni-sb.de/softevo