CMMI Level 5: Return on Investment for Raytheon N TX Donna Freed - - PowerPoint PPT Presentation

▶
cmmi level 5 return on investment for raytheon n tx
SMART_READER_LITE
LIVE PREVIEW

CMMI Level 5: Return on Investment for Raytheon N TX Donna Freed - - PowerPoint PPT Presentation

Nov 22, 2023 287 likes •488 views

CMMI Level 5: Return on Investment for Raytheon N TX Donna Freed Network Centric Systems, McKinney, TX Achieving CMMI Level 5 We did it! How Did We Do It? Achieve Engineering Goals. This presentation describes the benefits This

slide-1
SLIDE 1

CMMI Level 5: Return on Investment for Raytheon N TX

Donna Freed Network Centric Systems, McKinney, TX

slide-2
SLIDE 2

12/1/2004 Page 2

Achieving CMMI Level 5

  • We did it!
  • How Did We Do It?

– Achieve Engineering Goals.

This presentation describes the benefits

  • f achieving CMM Level 4 in 2001, and

then to achieve CMMI Level 5 in 2003. This presentation describes the benefits

  • f achieving CMM Level 4 in 2001, and

then to achieve CMMI Level 5 in 2003.

slide-3
SLIDE 3

12/1/2004 Page 3

We Did IT!

–Raytheon North Texas is the first site in Raytheon and fifth

company in the world to achieve CMMI Level 5.

–Measurable results are achieved before achieving Level 5. –This Presentation shows the actual ROI of going to each

level, as well as our ROI projection.

slide-4
SLIDE 4

12/1/2004 Page 4

How Did We Use CMMI to Achieve?

  • Why is Raytheon North Texas pro-active about achieving

CMMI Level 5?

–Because we want to achieve the performance excellence

goals required by our business. We are focused on achieving performance excellence and recognition as the preferred supplier for new business.

slide-5
SLIDE 5

12/1/2004 Page 5

Envision Improvement

Product teams use common tools and processes in an environment of continuous improvement guided by industry “Best Practices”

Integrated Product Teams: Cross-functional resources to implement our processes

IPTs IPTs IPTs

Capability Maturity Model Integration: The yardstick for judging the maturity of

  • ur processes

Integrated Product Development System: Where we define our product development processes Raytheon Six Sigma: How we improve our processes

Programs Integrate R6σ, IPDS and CMMI into their Pland

slide-6
SLIDE 6

12/1/2004 Page 6

How Did We Achieve Performance Goals? How did we use CMMI to achieve our performance excellence goals?

–We picked performance goals that were important

to us.

–The metrics data we collect characterizes the

  • rganizational performance in terms of our
  • rganizational goals and identifies opportunities of

improvement.

slide-7
SLIDE 7

12/1/2004 Page 7

SWEC SWIP Objectives

  • Meet Commitments (to Customer)

– Intent: Meet the cost and schedule objectives of the programs we

support.

– Quantification: CPI and SPI

  • SW Price

– Intent: Price software engineering products competitively – Quantification: $ / DLOC

  • Deliver Quality

– Intent: Deliver quality software engineering products – Quantification: In-phase Defects and Defect Density

We have been executing statistical process control on the overall process using these measures for years.

slide-8
SLIDE 8

Organization Process Analysis

Use R6σ Tools for Metrics Analysis

Cp = 0.09 Cpk = -0.03 Cpk (upper) = 0.2 Cpk (lower) = -0. Cr = 11.61 Cpm = 0.09 K = -0.61

Process Capability for CPI

LSL = 0.975, Nominal = 1.0, USL = 1.15

CPI frequency

0.5 1 1.5 2 2.5 3 4 8 12 16 20 24

M ultiple data sources Process Tooling People Training Program s Defect Containm ent M etric has Excessive Variation: s > 28% Peer review process W hat counts (definition of defects) Procedure Tools not user friendly Not a priority
  • n a program
Type & reason codes not useful How to count defects in reused code Too program focused, need product line &
  • rg view
Round trip eng. (i.e. integrated tooling) No com m on repository for shared code Tools & process to autom ate defect capture Tim e to do prework Using correct people in reviews Subjectivity (differing points
  • f view)
People concerned about defect counts being used against them Clear guidelines for phases of
  • rig
Peer review training People don't know how to use the defect data Effective m eeting training needed for peer reviews New people do not know how to classify defects People need to know the concept
  • f operations
Mix of program s Always start at 100% Custom er care abouts Pjt to pjt differences in counting Inconsistent Varying types Criteria variation JIT No Org Training Purpose not clear Needs to include: Severity Type & reason Phases Roles Conducting Meeting & follow-up Corrective action Old/New Domain Variation Schedule Constraints SW Experts Systems Limited Domain Experts Overloaded Attrition Phase of origin Defect Existence Reason code Types Source Doublecounts?? Defect Logger Continuus Doors Continuus learning curve Defect

SW Program Duration (M onths)

5 10 15 20 <= 6 > 6 and <= 12 > 12 and <=18 >18 and <=24 >24 and <=30 >30 and <=36 >36 and <=42 >42 and <=48 > 48 Mean = 23 Median = 18 Min = 5 Max =120 STD EV = 22 n = 55

$/DLOC Regression Model ELOC/DLOC Dollars per DLOC

0.2 0.4 0.6 0.8 1 100 200 300 400

slide-9
SLIDE 9

12/1/2004 Page 9

Improvement Results

  • Demonstrated the linkage between R6σ and CMMI Levels 4 & 5.
  • Characterization included over 300 applications of R6σ tools such

as ANOVA, cause and effect, regression analysis, histograms, Cpk, hypothesis testing, logical process mapping, and others.

  • Identified five projects to reduce variation in organizational

performance and support the CMMI Level 5 timeline.

  • Enabled CMMI Level 5 certification.

– Improvement of Business Performance was recognized by Assessment

Team as global strength in the CMMI Level 5 Assessment.

  • Contributed ROI of 3:1 through significant cost avoidance realized

by organization improvements

slide-10
SLIDE 10

12/1/2004 Page 10

Operational Results

  • Achieving CMMI Level 5 Certification for Raytheon image

and competitive advantage is one thing, but look at the

  • perational results.
  • “Meeting Commitments” all improved concurrent with SEI

CMMI Level 5 certification Across the organization, we improved:

–CPI by 5 percentage points, and reduced variation by 34%. –SPI by 8 percentage points, and reduced variation by 50% –Defect Density by 44 percentage points, and reduced

variation by 31%

slide-11
SLIDE 11

12/1/2004 Page 11

Process Capability for CPI CPI frequency

10 20 30 40

Process Capability for CPI CPI

frequency

2 4 6 8 10

Cost Performance Index

We no longer have CPI special cause variation on the low end!

January 2004 February 2001

slide-12
SLIDE 12

12/1/2004 Page 12

Cost Performance Index

SWEC CPI Trend

F e b 1 M a r 1 A p r 1 M a y 1 J u l 1 S e p 1 D e c 1 1 Q 2 2 Q 2 3 Q 2 4 Q 2 1 Q 3 2 Q 3 3 Q 3 4 Q 3

Upper DROV Nominal value Lower DROV Mean Performance

Improved CPI by 5 percentage points, and reduced variation by 34%.

CMM Level 4 June 2001 CMMI Level 5 September 2003

slide-13
SLIDE 13

12/1/2004 Page 13

Process Capability for SPI SPI frequency

5 10 15 20 25 30

Process Capability for SPI SPI frequency

3 6 9 12 15

Schedule Performance Index

February 2001

We are still moving in the right direction!

January 2004

slide-14
SLIDE 14

12/1/2004 Page 14

Schedule Performance Index

SWEC SPI Trend

Feb 01 Mar 01 Apr 01 May 01 Jul 01 Sep 01 Dec 01 1Q02 2Q02 3Q02 4Q02 1Q03 2Q03 3Q03 4Q03

Upper DROV Nominal value Lower DROV Mean Performance

Improved SPI by 8 percentage points, and reduced variation by 50% CMM Level 4 June 2001 CMMI Level 5 September 2003

slide-15
SLIDE 15

12/1/2004 Page 15

Process Capability for Defect Density Defect Density frequency

10 20 30 40 50 60

Process Capability for Defect Density Defect Density frequency

1 2 3 4

Defect Density

February 2001

Continuing improvement in mean and variation.

January 2004

slide-16
SLIDE 16

12/1/2004 Page 16

Defect Density

Improved Defect Density by 44 percentage points, and reduced variation by 31%

SWEC Defect Density Trend

F e b 1 M a r 1 A p r 1 M a y 1 J u l 1 S e p 1 D e c 1 1 Q 2 2 Q 2 3 Q 2 4 Q 2 1 Q 3 2 Q 3 3 Q 3 4 Q 3

Upper DROV Nominal value Lower DROV Mean Performance

CMM Level 4 June 2001 CMMI Level 5 September 2003

slide-17
SLIDE 17

12/1/2004 Page 17

Plot of Regression Model Percentage Organization Process Adherence CPI

Characterize – CPI Analysis

  • Regression Analysis included a sample of various process

characteristics.

  • Projects that follow the standard process tend to have a better

and more predictable CPI performance.

  • Process adherence is not a guarantee of CPI success. It improves

the probability of CPI success.

slide-18
SLIDE 18

12/1/2004 Page 18

Characterize – SPI Analysis

Predicted SPI based on Process Adherence vs. Observed SPI Multiple Regression Analsysis Prediction of SPI Observed SPI

  • Multiple Regression Analysis included Level 2, Level 3, and Level 4 process

characteristics.

  • Projects that follow the standard process tend to have a better and more

predictable SPI performance.

  • Process adherence is not a guarantee of SPI success. It improves the

probability of SPI success.

  • Organizational process adherence is the only identified factor affecting SPI.
slide-19
SLIDE 19

12/1/2004 Page 19

Results

  • Our improvements were recognized as organizational

strengths in the appraisal. “This accomplishment leads the way for Raytheon to distinguish ourselves from the competition and achieve customer satisfaction through superior program execution. There is no higher illustration of customer focus than this level of excellence.” Colin Schottlaender, Raytheon NCS President These improvements contributed to ROI of 3:1