Vector-Based Metrics for Assessing Technology Maturity Gerard E. - PowerPoint PPT Presentation

Vector-Based Metrics for Assessing Technology Maturity Gerard E. Sleefe, Ph.D. Senior Technical Deputy to the Chief Engineer Sandia National Laboratories Albuquerque, NM, USA Contact: 505-844-2195; gesleef@sandia.gov Sandia is a multi-program laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE -AC04-94AL85000. SAND Number: 2010-0238 P Systems Engineering Conference October 2011

Presentation Outline • Background and Motivation • Scalar Metrics for Technology Maturity • Introduction to Vector-Based Metrics • Systems Engineering Example • Technology Maturation Example • Conclusion and Recommendations

Quiz Question • A car is traveling at 50 mph, and a truck is travelling at 60 mph. • When and where will they meet? …

Quiz Question #2 • A Next-Generation Microprocessor is currently being prototyped (TRL=4, MRL=3). • When will the new microprocessor hit the market (TRL=9, MRL=9)?

Scalar Technology Metrics Technology Readiness Scale *For more technology metrics, see for example E. Geisler, 1999 • Scalar Metrics play an important role in technology management, acquisition, systems engineering • But: they measure only the magnitude of the current state • And: they usually do not have a mathematical basis for performing systems engineering calculations

Vector-Based Metrics • Measure the Magnitude AND Direction • Enables Vector Mathematics between Metrics * after Marsden et.al., Vector Calculus , 2003

Vector-based Technology Metrics Some proposed vector metrics • Technology Maturation Rate (TMR): d TMR ( t ) TRL ( t ) dt TRL = Technology Readiness Levels • Technology Profit Margin (TPM): TPM ( t ) MV ( t ) I ( t ) MV = Market Value of the technology I = Investment in the technology

Systems Engineering Example System Sub-System Sub-System #1 #2 Component Component Component Component Component A B C D E

Systems Aggregation of TRL’s System TRL=3? Sub-System Sub-System TRL=6? TRL=3? #1 #2 Component Component Component Component Component A B C D E TRL=4 TRL=3 TRL=5 TRL=8 TRL=6 • TRL of Sub-Sys #1 = min (TRL4, TRL3, TRL5) = TRL3 • TRL of Sub-Sys #2 = min (TRL8, TRL6) = TRL6 TRL of the System = min (TRL3, TRL6) = TRL3 TRL’s alone do not give full insight into system -level maturity

Vector Analysis of Systems TRL(t) TMR 1 C 5 Sub-System A #1 4 B 3 Component Component Component A B C 2 Time (t) TRL=4 TRL=3 TRL=5 Yr 1 Yr 2 Yr 3 TMR ( t ) A ( t ) B ( t ) C ( t ) 1

Technology Maturation Study • Monitor an actual product development effort over the course of 18 months – Measure technology metrics throughout, and make informed decisions using technology vector analysis MEMS Acceleration Switch COTS Acceleration Switch Actuated State Initial State (Switch Closed) (Switch Open) Metal Standoff Acceleration g (m/s 2 ) TO-18 Package with single gold pin (other pin-out arrangements possible) Acknowledgement: Polosky and Garcia, 2006

Experimental Observables • Traditional project management metrics – Cost, schedule, and technical requirements • Quantitative technology metrics – Technology Readiness Metrics (TRL, MRL, TMR, etc. ) – Product development cycle time (months) – Prototype production yield (%)

Experimental Results: MEMS Technology Development Progression Month Month Month Month Month Month 0 5 7 9 12 17 Develop Low Design and Final Design Develop Robust Go, No-Go Develop Model-Based Contact Process and Process Spring Design Decisions Designs Resistance Enhancements Adjustments Meets all Meets Severe Meets Most Feasibility Fully-Functional Normal Environmental Reliability Demonstrated Devices Yielded Environmental Specifications Requirements Specifications

Experimental Results: MEMS Development and Production Metrics Product Development Cycle Time (months) 6 5 4 3 2 1 0 5 6 7 8 9 10 11 12 13 14 15 16 17 Time (months) Prototype Production Yield (%) 100 MEMS reached TRL=7 after 18 months 90 80 70 60 50 40 30 20 10 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Time (months)

COTS Challenges • Acceleration Sensitivity deviates from manufacturer’s spec 25 20 15 G-level • Part Failed due to Metal Shard 10 5 COTS 0 ByCurve R S F

Technology Maturation Vector Analysis TRL(t) COTS 8 7 6 5 4 3 MEMS 2 Time (t) Month 0 Month 5 Month 11 Month 17

Vector-based Metrics Complement Traditional Technology Management Tools Innovation Technology Cost Schedule Management Framework Performance (Technical Requirements)

Summary • Vector-based metrics can provide additional technology management insight: – Enable the assessment of both magnitude and direction – Provide a mathematical framework for system analytics • Recommend that Maturation Rates (vector quantity) be used to complement the TRL and MRL scales • Follow-on studies recommended: – To evaluate effectiveness of vector-based metrics – To establish a technology maturation database • TRL, MRL, Vectors, etc. versus technology categories • would support predictive modeling of technology maturation

Backups

TRL History: MEMS vs COTS 9 8 MEMS 7 Technology Readiness Level (TRL) 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 9 COTS 8 7 6 5 4 3 2 Time (months) 1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17