Overwrapped Pressure Vessel Life Prediction Anne Ryan Driscoll - - PowerPoint PPT Presentation

Experimental Design for Composite Overwrapped Pressure Vessel Life Prediction

Anne Ryan Driscoll

Department of Statistics Virginia Tech Blacksburg, VA 24061-0439 USA

adriscoll@vt.edu

1

Outline

• Overview of NESC Project
• Statistical Based Testing
• Experimental Design for NESC Project
• Lessons Learned

 Communication  Assumption Checking 2

NASA Strand and Vessel Testing

• NASA’s Engineering Safety Center (NESC)

project to assess safety of Composite Overwrapped Pressure Vessels (COPVs)

• COPVs

 Transport gasses under high pressure  Metal Liner  Wrapped by a Series of Carbon Strands

• Research Question: Reliability of COPVs at

Use Conditions for the Expected Mission Life

 Primary Focus on Strands  Secondary Focus on Relationship to Vessels  Strands Less Expensive to Test

3

NASA Strand and Vessel Testing

• Analyses Use Classic Weibull Model

R 𝑢𝑗 = 𝑓

− 𝑢𝑗 𝑢𝑠𝑓𝑔𝑇𝑆𝜍

𝛾

• Observed Life Time: 𝑢𝑗
• 𝑇𝑆: Stress Ratio, ratio of stress level to

strength scale parameter

• Critical Parameters:

 𝜍: Sensitivity to Stress Ratio  𝛾: Shape parameter for time to Failure  𝑢𝑠𝑓𝑔: Reference time to Failure

4

NASA Strand Study

• Previous Strand Test

 Relevant strand study conducted at a national

lab

 57 strands at high loads for 10 years  Net information learned:

 Strands either fail very early or  Last more than 10 years

 Limited information based on 10 years of study!

• Estimates of Critical Parameters for Planning

5

NASA Strand Study

• The Core NASA Analytics Team:

 Reliability Engineers:

• JPL
• Langley Research Center
• Glenn Research Center

 Statisticians:

• Marshall Space Flight Center
• Virginia Tech

 Project Engineers

• White Sands Testing Facility

6

What is Statistically Based Testing?

• Begins with Scientific Method
• Sequential Learning Means

Sequential Experimentation

• Steps in Planning Experiments

7

Scientific Method

• The heart of sound engineering practice is

the scientific method.

 systematic approach for problem solving  constant interplay between the concrete and

abstract

• Concrete: Actual engineering process
• Abstract: Mathematical models
• An iterative induction – deduction process
• Conduct an investigation!

Scientific Method

• The proper application of the scientific

method requires

 model building  data collection  data analysis  data interpretation.

• In essence, the scientific method requires

experimentation to support investigation!

9

Scientific Method

10

Scientific Method and Sequential Experimentation

• The scientific method is a discovery

process that involves sequential learning.

• Consequently, the experimentation

supporting the scientific method is sequential!

11

Sequential versus “One-Shot”

• Discovery versus Confirmation
• Discovery:

 Limited initial information  Must determine

• Important factors
• Proper experimental levels (experimental region)
• Appropriate region/model to address questions
• Discovery Requires a Sequential Approach!
• Final stage: Confirmation

12

Benefits of Sequential Approach

• Formal Sequential Strategy Allows Great

Flexibility

 Different Factors and/or Levels  Move to New Experimental Region

• Ability to Fit More Complex Models over

Time

• Running the Experiment in Blocks Can

Mitigate Disasters!

13

Steps in Planning Experiments

• 1. Define the Problem
• 2. Select Response Variables
• 3. Identify Sources of Variation
• 4. Choose the Experimental Design
• 5. Train the Experimenters/Conduct the

Experiment

• 6. Analyze the Data
• 7. Reach Conclusions

14

Experiments Do Fail!

• Management Did Not Build Proper Team
• Team Lacked Essential Skills

 Proper soft skills  Proper project management

• Team Did Not Put Enough Thought Prior

to Collecting Data.

15

NASA Strand Study

• Team’s Initial Concept

 Much larger study that the original 10 year study  Censor very early  Reduces time  Allows the larger study in a practical

amount of time

• Proceed in phases
• Have detailed data records to track any

problems

16

NASA Strand Study

Experimental Phases

• Phase A – During “shake-out” of tests rigs
• Phase B – “Gold Standard” Experiment for

Strands

• Phase C – “Proof” Study
• In Parallel: Vessel Studies (Opportunistic)

17

NASA Strand Study

• Phase A: Conducted During Shake-Out of

Equipment

 Small study (although bigger than the national

lab study!)

 Statistical goal: Determine if the parameters

from the national lab study are valid as the basis for planning the larger study!

 Note: Phase A gave the team an opportunity

to re-plan the larger experiment, if necessary!

18

NASA Strand Study

• Phase B: “Gold Standard” Experiment

 Planned time required: 1 year  Used 4 “blocks” of equal numbers of strands

• Allowed the team to correct for time effects
• Allowed the team to mitigate problems, especially

early

 Study assumed the “classic” Weibull model  Size of the experiment assured ability to

assess model

19

NASA Strand Study

• Phase A: Surprisingly Similar to Initial Study
• Phase B:

 Serious problem occurred with the gripping in the

first block

 Serious conversations with possibility of

replacing!

 Other three blocks well behaved and by

themselves produced better than the planned precision for the estimates

• Final Decision: Drop the First Block

20

NASA Strand Study: Benefits

• Phase A:

 Opportunity to Confirm Initial Study

Parameter Estimates

 Allowed opportunity to revise the

experimental protocol if the estimates were significantly different

21

NASA Strand Study: Benefits

• Phase B:

 Allowed opportunity to model changes in

time over the year.

 Mitigated the problem with the first

block!

 Provided simple mechanism for

replacing the first block if needed!

22

Lessons Learned

• Communication Language Is Critical

 Engineers and Statisticians Speak very Different

Languages

 Must Use the Simplest Language that Can

Address the Question of Interest

• Checking Model Assumptions Is Critical

 Under-appreciated by Engineers  Fundamental to Statistical Standards of Practice  Engineering Language Often Inadequate

23

Details on Lessons Learned

24

First Lesson Learned

Common Language Is Important

• Classic Engineering Weibull Model:

R 𝑢𝑗 = 𝑓

− 𝑢𝑗 𝑢𝑠𝑓𝑔𝑇𝑆𝜍

𝛾

• Standard Statistical Model

 Re-parameterization of the Engineering Model  Uses Relationship between Weibull and the

Smallest Extreme Value (SEV) Distributions

 If 𝑢𝑗 is Weibull, then ln(𝑢𝑗) is SEV 25

First Lesson Learned

• Common Definition of a Weibull Model:

𝑆 𝑢𝑗 = 𝑓

− 𝑢𝑗 𝜃𝑗

𝛾

• Let 𝜃𝑗 =

𝑢𝑠𝑓𝑔 𝑇𝑆𝜍

• Note:

R 𝑢𝑗 = 𝑓

− 𝑢𝑗 𝑢𝑠𝑓𝑔𝑇𝑆𝜍

𝛾

= 𝑓

− 𝑢𝑗 𝜃𝑗

𝛾

26

First Lesson Learned

• The smallest extreme value reliability function:

𝑆 𝑧𝑗 = 𝑓−𝑓− 𝑧−𝜈

𝜏

• Assume that ln(𝑢𝑗) follows SEV distribution; thus,

𝑧𝑗 = ln(𝑢𝑗)

• Let 𝜈 = ln(𝜃) and 𝜏 =

1 𝛾

• Therefore:

𝑧 − 𝜈 𝜏 = ln 𝑢𝑗 − ln(𝜃) 1/𝛾 = 𝛾 ln 𝑢𝑗 − ln 𝜃

27

First Lesson Learned

𝑆 𝑧𝑗 = 𝑆 ln 𝑢𝑗 = 𝑓−𝑓−𝛾 ln 𝑢𝑗 −ln 𝜃 = 𝑓

− 𝑢𝑗 𝜃

𝛾

= 𝑓

− 𝑢𝑗 𝑢𝑠𝑓𝑔𝑇𝑆𝜍

𝛾

• Key Point: This smallest extreme value

model is a re-parameterization of the classic model!

• More importantly: It Generalizes.

28

First Lesson Learned

• Note:

𝜃𝑗 = 𝑢𝑠𝑓𝑔 𝑇𝑆𝜍 𝜈𝑗 = ln 𝜃𝑗 = ln 𝑢𝑠𝑓𝑔 − 𝜍ln(𝑇𝑆) = 𝛿0 + 𝛿1ln(𝑇𝑆)

• where

𝛿0 = ln 𝑢𝑠𝑓𝑔 𝛿1 = −𝜍

29

First Lesson Learned

• Common Vessel/Strand Model:

𝜈𝑗 = 𝛿0 + 𝛿1 ln 𝑇𝑆𝑗 + 𝛿2𝑨𝑗

 𝑨𝑗 = 1 if a vessel  𝑨𝑗 = 0 if a strand

• Adjusts 𝑢𝑠𝑓𝑔 for vessels.
• Communication Issue: This

Generalization:

 Very natural for the statistician  Very un-natural for the engineer 30

First Lesson Learned

• Critical Lesson: Communicate in the

Simplest Language for the Question under Study

• Engineering Model:

 Planning  All Phase A analyses  Initial Phase B analyses

• Statistical Model

 More complicated situations (Combined

Model)

31

Second Lesson Learned

• Checking Assumptions Is Critical to Proper

Data Analysis

• Engineers and Statisticians View

Assumptions Differently!

• Basic Assumption Checking Should Be a

Requirement for the Final analysis

• Those Who Do Check Assumptions, Often

Used Ad Hoc Techniques

32

Second Lesson Learned

• Applied Statisticians Prefer Standardized

Residuals to Assess Assumptions

 “Raw” Residuals

𝑓𝑗 = 𝑧𝑗 − 𝑧𝑗 = 𝑝𝑐𝑡𝑓𝑠𝑤𝑓𝑒 − 𝑞𝑠𝑓𝑒𝑗𝑑𝑢𝑓𝑒 Note: The units for these residuals are the same as the units of 𝑧

33

Second Lesson Learned

• “Standardized” Residuals

𝑠𝑗 = 𝑓𝑗 𝑊𝑏𝑠 𝑓𝑗 = 𝑧𝑗 − 𝑧𝑗 𝑊𝑏𝑠 𝑓𝑗 Note: Dimensionless and interpreted as 𝑎 or 𝑢 in normal theory. For normal theory

• Symmetric Distribution around

𝑧𝑗

• Relatively Small Variance

34

Second Lesson Learned

• SEV Distribution Can Be Very Non-

Normal!

 Highly Skewed  Large Variability around Predicted Value  Censoring Compounds Problem  Normal Theory Interpretation Problematic

• Issue: What Do We Mean by Predicted

Value?

• Residual is Basis for Predicted Reliability.

35

Second Lesson Learned

• Standard Statistics Texts for Reliability

Analysis Plot Based on the Standardized Residuals

• Let

𝑆𝑓𝑛𝑞 𝑢𝑗 be the empirical reliability function based on ranks of the failures.

 X axis: Standardized Residuals  Y axis: ln −ln

𝑆𝑓𝑛𝑞 𝑢𝑗

• If Weibull Is Reasonable, Forms Straight

Line.

• Very Natural for Statisticians

36

Second Lesson Learned

37

Note: Just example data, not the data from NESC project

Second Lesson Learned

• Engineers Do Not Use the SEV Model
• Standardized Residual is a Foreign

Concept

 Engineers See No Proper Rationale  Engineers Reject as “Ivory Tower”

• Engineers Reject this Approach.
• However, They Like Plots of the “Fits” to

the Data.

38

Second Lesson Learned

• Issue: What Is Proper “Predicted Value”
• The standardized residual based on the

SEV model is 𝑠𝑗 = 𝛾 ln 𝑢𝑗 − ln(𝜃𝑗) = 𝛾 ln(𝑢𝑗) − ln 𝑢𝑠𝑓𝑔 + 𝜍 ln 𝑇𝑆𝑗

• The engineering interpretation is

𝑠𝑗 = ln −ln 𝑆 𝑢𝑗

39

Second Lesson Learned

• Connection Between SEV Residual and

Engineering Interpretation

• Classic Engineering Model:

𝑆 𝑢𝑗 = 𝑓

− 𝑢𝑗 𝑢𝑠𝑓𝑔𝑇𝑆𝜍

𝛾

ln 𝑆 𝑢𝑗 = − 𝑢𝑗 𝑢𝑠𝑓𝑔 𝑇𝑆𝜍

𝛾

ln −ln 𝑆 𝑢𝑗 = 𝛾 ln 𝑢𝑗 − ln 𝑢𝑠𝑓𝑔 + 𝜍 ln 𝑇𝑆𝑗

• The engineering interpretation is

𝑠𝑗 = ln −ln 𝑆 𝑢𝑗

40

Second Lesson Learned

• SEV Residual is Basis for Predicted

Reliability!

• Proper Plot Linearizes Predicted Reliability
• Compare to Linearized Empirical

Reliability

• SEV Residual Gives the “Fit”
• Still a Residual Plot!
• Better: Has Engineering Interpretation

41

Second Lesson Learned

• Challenge for the Statistician to Convey

Standard Statistical Practice to Engineers

• Required a Compromise Approach

 Statistician Needed to Understand Engineer’s

Perspective

 Statistician Needed to Teach the Engineer

Some of the Statistician’s Language

 Required Huge Trust on Both Sides! 42

Team

43

Lorie Grimes-Ledesma JPL Geoff Vining Virginia Tech Anne Driscoll Virginia Tech Ken Johnson MSFC Pappu Murthy GRC James Reeder LARC Dan Wentzel WSTF Luis Hernandez WSTF