The Influence of Organizational Structure on Software Quality: An - - PowerPoint PPT Presentation

▶
the influence of organizational
SMART_READER_LITE
LIVE PREVIEW

The Influence of Organizational Structure on Software Quality: An - - PowerPoint PPT Presentation

Jul 05, 2023 18 likes •351 views

The Influence of Organizational Structure on Software Quality: An Empirical Case Study Nachiappan Nagappan Brendan Murphy Victor R. Basili Microsoft Research Microsoft Research University of Maryland Redmond, WA, USA Cambridge, UK College

slide-1
SLIDE 1

The Influence of Organizational Structure on Software Quality: An Empirical Case Study

Nachiappan Nagappan

Microsoft Research

Redmond, WA, USA nachin@microsoft.com

Brendan Murphy

Microsoft Research

Cambridge, UK bmurphy@microsoft.com

Victor R. Basili

University of Maryland

College Park, MD, USA basili@cs.umd.edu

Presentation by Etan Kissling 18.05.2010

slide-2
SLIDE 2

SOFTWARE METRICS

slide-3
SLIDE 3

Code Churn

  • Software change history
  • Large / recent changes
  • Total added, modified and deleted LOC
  • Number of times that a binary was edited
  • Number of consecutive edits
slide-4
SLIDE 4

Code Complexity

  • Gathered from code itself
  • Multiple complexity values
  • Cyclomatic complexity
  • Fan-In / Fan-Out of functions
  • Lines of Code
  • Weighted methods per class
  • Depth of Inheritance
  • Coupling between objects
  • Number of subclasses
  • Total global variables
slide-5
SLIDE 5

Dependencies

  • Components that a class uses
  • Both data and call dependencies
  • Incoming / outgoing direct / indirect

dependencies to a binary

  • Layer information: Distance of a binary from

the system kernel

slide-6
SLIDE 6

Code coverage

  • Degree to which the source code is tested

foo (x: INTEGER; y: INTEGER): INTEGER local c: INTEGER do c := y if x > 5 and y > 0 then c := x end Result := x * c end

slide-7
SLIDE 7

Statement 1 Statement 2 Statement 3 Statement 4

Statement coverage

  • Has each node in the program been executed?

foo (x: INTEGER; y: INTEGER): INTEGER local c: INTEGER do c := y if x > 5 and y > 0 then c := x end Result := x * c end A testing suite which includes foo(7, 1) would cover all statements of this code.

slide-8
SLIDE 8

Branch coverage

  • Has each control structure been evaluated both to true and false?

foo (x: INTEGER; y: INTEGER): INTEGER local c: INTEGER do c := y if x > 5 and y > 0 then c := x end Result := x * c end foo(7, 1) and foo(7, 0) together would cover this branch completely

slide-9
SLIDE 9

Pre-release defects

  • Number of pre-release bugs found in a binary
  • Strong relationship between development

defects per module and field defects per module

slide-10
SLIDE 10

ORGANIZATIONAL METRICS

slide-11
SLIDE 11

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 40 30 200 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 5 engineers edit 190 edits done

slide-12
SLIDE 12

Number of Engineers

  • Touched a binary
  • Still employed by the company

› The more people who touch the code the lower the quality

slide-13
SLIDE 13

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 40 30 200 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 5 engineers edit 190 edits done

slide-14
SLIDE 14

Number of Ex-Engineers

  • Touched a binary
  • Left the company

› A large loss of team members affects the knowledge retention and thus quality

slide-15
SLIDE 15

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 40 30 200 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 5 engineers edit 190 edits done

slide-16
SLIDE 16

Edit Frequency

  • Number of edits

› The more edits to components the higher the instability and lower the quality

slide-17
SLIDE 17

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 40 30 200 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 5 engineers edit 190 edits done

slide-18
SLIDE 18

Depth of Master Ownership (DMO)

  • Level of ownership
  • More than 75% of the edits done by engineers

which report to the owner › The lower level is the ownership the better is the quality

slide-19
SLIDE 19

Level 0 Level 1 Level 2

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 40 30 200 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 5 engineers edit

slide-20
SLIDE 20

Percentage of Org contributing to development

  • 𝑂𝑣𝑛𝑐𝑓𝑠 𝑝𝑔 𝑞𝑓𝑝𝑞𝑚𝑓 𝑠𝑓𝑞𝑝𝑠𝑢𝑗𝑜𝑕 𝑏𝑢 𝑢𝑖𝑓 𝐸𝑁𝑃 𝑚𝑓𝑤𝑓𝑚

𝑁𝑏𝑡𝑢𝑓𝑠 𝑝𝑥𝑜𝑓𝑠 𝑝𝑠𝑕 𝑡𝑗𝑨𝑓

› The more cohesive are the contributors (organizationally) the higher is the quality

slide-21
SLIDE 21

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 40 200 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 5 engineers edit 190 edits done

7 30

slide-22
SLIDE 22

Level of Organizational Code Ownership

  • If there is an owner:

Percent of edits from the owner’s organization

  • If there is no owner:

Percent of edits from the organization which made the majority of edits › The more cohesive are the contributions (edits) the higher is the quality

slide-23
SLIDE 23

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 40 30 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 5 engineers edit 190 edits done

200 250

slide-24
SLIDE 24

Overall Organization Ownership

  • 𝑂𝑣𝑛𝑐𝑓𝑠 𝑝𝑔 𝑞𝑓𝑝𝑞𝑚𝑓 𝑏𝑢 𝑢𝑖𝑓 𝐸𝑁𝑃 𝑚𝑓𝑤𝑓𝑚 𝑛𝑏𝑙𝑗𝑜𝑕 𝑓𝑒𝑗𝑢𝑡

𝑈𝑝𝑢𝑏𝑚 𝐹𝑜𝑕𝑗𝑜𝑓𝑓𝑠𝑡 𝐹𝑒𝑗𝑢𝑗𝑜𝑕

› The more the diffused contribution to a binary the lower is the quality

slide-25
SLIDE 25

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 40 30 200 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 190 edits done

5 32

slide-26
SLIDE 26

Organization Intersection Factor

  • Number of different organizations that

contribute greater than 10% of edits › The more diffused the different organizations contributing code, the lower is the quality

slide-27
SLIDE 27

Organizational Structure

A AB AC AD ABA ABB ABC ACA ADA ABCA E1 E2 E3 E4 E5 E6 E7

30 10 40 30 10 30 45 60 25 30 Org Size

  • No. of Edits

Total edits = 250 Total Engineers Editing = 32 Total Ex-Engineers (edited) = 0 5 engineers edit 190 edits done

slide-28
SLIDE 28

CASE STUDY

slide-29
SLIDE 29

Case study

  • Windows Vista:

3404 binaries 50+ Million LOC

  • Access to people management software to

build tree maps for organizational metrics

  • 50 random splits:

2/3 to build prediction model 1/3 to verify prediction accuracy

slide-30
SLIDE 30

Precision and recall

Predicted Not Failure-prone Failure-prone Actual Not failure-prone A B Failure-prone C D 𝑄𝑠𝑓𝑑𝑗𝑡𝑗𝑝𝑜 = 𝑒 𝑐 + 𝑒 Percentage of correct failure-prone predictions 𝑆𝑓𝑑𝑏𝑚𝑚 = 𝑒 𝑑 + 𝑒 Percentage of correctly identified failure-prone binaries

slide-31
SLIDE 31

Comparization

Model Precision Recall Organizational Structure 86.2% 84.0% Code Churn 78.6% 79.9% Code Complexity 79.3% 66.0% Dependencies 74.4% 69.9% Code Coverage 83.8% 54.4% Pre-Release Bugs 73.8% 62.9%

slide-32
SLIDE 32

Threats to validity

  • Internal validity:

Influence of study to Windows

  • Construct validity:

Errors in measurement

  • External validity:

All data from one software system

slide-33
SLIDE 33