DEFECT DETECTION IN A DEFECT DETECTION IN A DISTRIBUTED SOFTWARE - - PowerPoint PPT Presentation

Defect Detection in a Distributed Software Maintenance Project1 DIB

DEFECT DETECTION IN A DEFECT DETECTION IN A DISTRIBUTED SOFTWARE DISTRIBUTED SOFTWARE MAINTENANCE PROJECT MAINTENANCE PROJECT

Alessandro Bianchi, Danilo Caivano, Filippo Lanubile, Giuseppe Visaggio

SER_Lab - Department of Informatics - University of Bari {bianchi, caivano, lanubile, visaggio}@di.uniba.it

Defect Detection in a Distributed Software Maintenance Project2 DIB

Case Study Case Study

Post mortem analysis on a maintenance project

carried out in EDS Italia

Massive maintenance

• f a large banking software system

to solve the Y2K problem

Defect Detection in a Distributed Software Maintenance Project3 DIB

The System The System

Banking Software System FA1 FA2 FA4 FA3 WP1, 2 WP1, 1 WP1, n … Item1, 1, 1 Item1, 1, 2 … Item1, 1, m

Defect Detection in a Distributed Software Maintenance Project4 DIB

The Maintenance Process … The Maintenance Process …

Project Management Configuration Management Review Test SQA Change Verification & Validation

Defect Detection in a Distributed Software Maintenance Project5 DIB

… The Maintenance Process … The Maintenance Process

Process execution started on Site 1 for all WPs Depending on rework needs and currently

available resources, Change and V&V phases were switched for some WPs to Site 2

Both sites were settled in Italy The Collocated project includes WPs entirely

executed at Site 1

The Distributed project includes WPs

executed at both Site 1 and Site 2

Defect Detection in a Distributed Software Maintenance Project6 DIB

Previous Results* Previous Results*

There are not statistically significant differences

between collocated and distributed projects for

Duration Effort Staff Reworking cycles

There are statistically significant differences

between collocated and distributed projects for

Number of reports Number of meetings

* A. Bianchi, D. Caivano, F. Lanubile, F. Rago, G. Visaggio, “An Empirical Study of Distributed Software Maintenance”, Proc. of the IEEE Intl. Conf on Sw Maint., 2002

Defect Detection in a Distributed Software Maintenance Project7 DIB

Further Analysis: Defect Metrics Further Analysis: Defect Metrics

Research Question: Does the distribution among

sites affect defect metrics?

Therefore, for each defect metric Mi the

following are posed

Hi0: There is no difference between the values

• f defect metric Mi for collocated WPs and for

distributed WPs

Hia: There is a difference between the values of

defect metric Mi for collocated WPs and for distributed WPs

Defect Detection in a Distributed Software Maintenance Project8 DIB

Observed metrics Observed metrics

# executed test cases & # of faults that caused

failures (faults from testing)

# reviews & # of found defects (faults from

review)

# audits & # of found issues (non conformities

from audits)

WPs Size (# items)

Defect Detection in a Distributed Software Maintenance Project9 DIB

Results … Results …

Audits: p-level=0.453

V&V activities are comparable

Test Cases p-level=0.633 Reviews: p-level=0.359

Box Plot (defects2.sta 27v*52c) Median 25%-75% Non-Outlier Range Outliers Extremes Distributed Collocated Project

• 0,1

0,0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9

• N. Audit per Item

Box Plot (defects2.sta 27v*52c) Median 25%-75% Non-Outlier Range Outliers Distributed Collocated Project 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0 Test Cases per Item Box Plot (defects2.sta 27v*52c) Median 25%-75% Non-Outlier Range Outliers Extremes Distributed Collocated Project

• 0,2

0,0 0,2 0,4 0,6 0,8 1,0 1,2

• N. Review per Item

Defect Detection in a Distributed Software Maintenance Project10 DIB

…Results …Results

Non-conformities: p-level=0.633

A significant difference DOES NOT exist

Faults from Testing: p-level=0.489 Faults from Review : p-level=0.212

Box Plot (defects2.sta 27v*52c) Median 25%-75% Non-Outlier Range Outliers Extremes Distributed Collocated Project

• 0,01

0,00 0,01 0,02 0,03 0,04 0,05 0,06 0,07 0,08 Faults from Testing per Item Box Plot (defects2.sta 27v*52c) Median 25%-75% Non-Outlier Range Outliers Extremes Distributed Collocated Project

• 0,05

0,00 0,05 0,10 0,15 0,20 0,25 0,30 0,35 0,40 0,45 Faults from Review per Item Box Plot (defects2.sta 27v*52c) Median 25%-75% Non-Outlier Range Outliers Extremes Distributed Collocated Project

• 0,02

0,00 0,02 0,04 0,06 0,08 0,10 0,12

• N. NCN per Item

Defect Detection in a Distributed Software Maintenance Project11 DIB

Hypotheses for Lack of Differences Hypotheses for Lack of Differences

The specific project management The tasks are independent of each other

They can be executed concurrently

The application domain is well-known by

both sites

Homogeneity of behavior of sites

because both belonging to the same company,

certified CMM 3

Defect Detection in a Distributed Software Maintenance Project12 DIB

Lessons Learned Lessons Learned

Need of an adequate management of:

strategic issues cultural issues technical issues

to make effective distribution of software process

This allows to

execute independent tasks exploit proper skills wherever they are