Vector-Based Metrics for Assessing Technology Maturity Gerard E. - - PowerPoint PPT Presentation

Systems Engineering Conference October 2011

Vector-Based Metrics for Assessing Technology Maturity

Gerard E. Sleefe, Ph.D.

Senior Technical Deputy to the Chief Engineer Sandia National Laboratories Albuquerque, NM, USA Contact: 505-844-2195; gesleef@sandia.gov

Sandia is a multi-program laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. SAND Number: 2010-0238 P

Presentation Outline

• Background and Motivation
• Scalar Metrics for Technology Maturity
• Introduction to Vector-Based Metrics
• Systems Engineering Example
• Technology Maturation Example
• Conclusion and Recommendations

Quiz Question

• A car is traveling at 50 mph, and a truck is

travelling at 60 mph.

• When and where will they meet?

…

Quiz Question #2

• A Next-Generation

Microprocessor is currently being prototyped (TRL=4, MRL=3).

• When will the new microprocessor

hit the market (TRL=9, MRL=9)?

Scalar Technology Metrics

• Scalar Metrics play an important role in technology

management, acquisition, systems engineering

• But: they measure only the magnitude of the current state
• And: they usually do not have a mathematical basis for

performing systems engineering calculations

*For more technology metrics, see for example E. Geisler, 1999

Technology Readiness Scale

Vector-Based Metrics

• Measure the Magnitude AND Direction
• Enables Vector Mathematics between Metrics

* after Marsden et.al., Vector Calculus, 2003

Vector-based Technology Metrics

Some proposed vector metrics

• Technology Maturation Rate (TMR):

) ( ) ( t TRL dt d t TMR

• Technology Profit Margin (TPM):

) ( ) ( ) ( t I t MV t TPM

MV = Market Value of the technology I = Investment in the technology TRL = Technology Readiness Levels

Systems Engineering Example

System

Sub-System #1

Component A Component B Component C Component D Component E

Sub-System #2

Systems Aggregation of TRL’s

TRL=4 TRL=3 TRL=5 TRL=8 TRL=6

• TRL of Sub-Sys #1 = min (TRL4, TRL3, TRL5) = TRL3
• TRL of Sub-Sys #2 = min (TRL8, TRL6) = TRL6

TRL of the System = min (TRL3, TRL6) = TRL3

TRL’s alone do not give full insight into system-level maturity

System

Sub-System #1

Component A Component B Component C Component D Component E

Sub-System #2

TRL=3? TRL=6? TRL=3?

Vector Analysis of Systems

Time (t) TRL(t) 2 3 4 5 Sub-System #1

Component A Component B Component C

TRL=4 TRL=3 TRL=5

A B

C

Yr 1 Yr 2 Yr 3

) ( ) ( ) ( ) (

1

t C t B t A t TMR

1

TMR

Technology Maturation Study

• Monitor an actual product development effort
• ver the course of 18 months

– Measure technology metrics throughout, and make informed decisions using technology vector analysis

Initial State (Switch Open) Actuated State (Switch Closed)

Acceleration g (m/s2) TO-18 Package with single gold pin (other pin-out arrangements possible) Metal Standoff

COTS Acceleration Switch MEMS Acceleration Switch

Acknowledgement: Polosky and Garcia, 2006

Experimental Observables

• Traditional project management metrics

– Cost, schedule, and technical requirements

• Quantitative technology metrics

– Technology Readiness Metrics (TRL, MRL, TMR, etc. ) – Product development cycle time (months) – Prototype production yield (%)

Experimental Results:

MEMS Technology Development Progression

Month Month 5 Month 7 Month 9 Month 12 Month 17

Develop Model-Based Designs Develop Low Contact Resistance Meets all Normal Environmental Specifications Feasibility Demonstrated Final Design and Process Adjustments Meets Most Reliability Requirements Go, No-Go Decisions Develop Robust Spring Design Meets Severe Environmental Specifications Design and Process Enhancements Fully-Functional Devices Yielded

Product Development Cycle Time (months) Time (months)

1 2 3 4 5 6 5 6 7 8 9 10 11 12 13 14 15 16 17

Experimental Results:

MEMS Development and Production Metrics

Prototype Production Yield (%) Time (months)

10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

MEMS reached TRL=7 after 18 months

COTS Challenges

• Acceleration Sensitivity deviates from manufacturer’s spec
• Part Failed due to Metal Shard

G-level ByCurve F S R 25 20 15 10 5

COTS

Technology Maturation Vector Analysis

Time (t) TRL(t) 2 3 4 5 Month 0 Month 5 Month 11 6 7 8 Month 17

COTS MEMS

Vector-based Metrics Complement Traditional Technology Management Tools

Technology Management Framework

Innovation Cost Schedule Performance

(Technical Requirements)

Summary

• Vector-based metrics can provide additional technology

management insight:

– Enable the assessment of both magnitude and direction – Provide a mathematical framework for system analytics

• Recommend that Maturation Rates (vector quantity) be

used to complement the TRL and MRL scales

• Follow-on studies recommended:

– To evaluate effectiveness of vector-based metrics – To establish a technology maturation database

• TRL, MRL, Vectors, etc. versus technology categories
• would support predictive modeling of technology maturation

Backups

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

MEMS COTS

Technology Readiness Level (TRL) Time (months)

TRL History: MEMS vs COTS