The Illusion of Method: Challenges of Model-Based Safety Assessment - - PowerPoint PPT Presentation
The Illusion of Method: Challenges of Model-Based Safety Assessment Oleg Lisagor Linling Sun Tim Kelly Overview Why do we trust safety assessment? Why do we trust FTA? Can we trust current MBSA techniques on the same basis? What do we need
ISSC 2010: The Illusion of Method - 2
Why do we trust FTA? Can we trust current MBSA techniques on the same basis? What do we need to do to better justify adequacy of the models and assessment?
ISSC 2010: The Illusion of Method - 3
Regardless of the methods used How we think the system will behave under conditions of failure
Can check for consistency with design models Can check for consistency with equipment test data (FMESs) Can check for consistency with experience with similar systems Can review and, sometimes, “test”
Why?
ISSC 2010: The Illusion of Method - 4
Ground Rules and Key Principles for Fault Tree construction
“Primary-Secondary-Command” “Immediate and Necessary cause”
Provide guidance and keywords
Facilitate completeness of assessment
Professional scrutiny Training and expertise
Strengths and limitations are known Standard errors and misconceptions are well-publicised
ISSC 2010: The Illusion of Method - 5
ISSC 2010: The Illusion of Method - 6
Often focus on notation rather than safety engineering concepts
Proliferation of idiosyncratic languages
Many techniques – little information on how are they related
There are currently three “purist” approaches to MBSA
Focus on what to model rather than how to assess the system
No public guidance
What systems are inappropriate for the application of the technique? What are the “error inducing features”? Does it actually work?
ISSC 2010: The Illusion of Method - 7
Adequacy of component models is context dependent Context of component models is not obvious
Unlike the context of intermediate events in FTs
“Emergent behaviour”
Not attributable to individual components
Exhaustive simulation is infeasible Need strategies for selecting “simulation cases”
ISSC 2010: The Illusion of Method - 8
Too little experience to trust the “gut feeling” New challenges Too many risks
Model adequacy argument
Side-by-side with the system safety argument
Incorporated into the overall safety case
ISSC 2010: The Illusion of Method - 9
ISSC 2010: The Illusion of Method - 10
ConstructionArg Argument over adequacy of construction process and methodology MethodologyAdequacy The modelling methodology is adequate ProcessAdequacy Construction process for model {M} was adequate MethodologyDefinition The methodology {T} is adequately defined MethodologyAppropriate Methodology is robust and appropriate for the type of system and the type of safety analysis performed MethodologyImplementation Construction process for Model {M} has adhered to the methodology {T} ConceptsDefinition Key concepts of the methodology and their relationship are adequately defined ArchitectureElicitation The methodology for determining model architecture is adequately defined ArchitectureVerification The methodology for verification
the model architecture is adequately defined ComponentModelling The methodology for definition of (detailed) component models is adequately defined HistoricalArg Argument over previous application of the methodology and similarity
Competency Safety engineers responsible for model constructon have received adequate training in modelling methodology Methodology Model-Based Safety Assessment Methodology {T}, that is [claimed to be] followed in construction of the Model {M}
Top Level Argument
Top Level Argument
ISSC 2010: The Illusion of Method - 11
ISSC 2010: The Illusion of Method - 12
Only weak evidence for some goals
A challenge even for the traditional approaches More critical for MBSA
Assumptions log Part of the adequacy argument in the safety case
ISSC 2010: The Illusion of Method - 13
ConstructionArg Argument over adequacy of construction process and methodology MethodologyAdequacy The modelling methodology is adequate ProcessAdequacy Construction process for model {M} was adequate MethodologyDefinition The methodology {T} is adequately defined MethodologyAppropriate Methodology is robust and appropriate for the type of system and the type of safety analysis performed MethodologyImplementation Construction process for Model {M} has adhered to the methodology {T} ConceptsDefinition Key concepts of the methodology and their relationship are adequately defined ArchitectureElicitation The methodology for determining model architecture is adequately defined ArchitectureVerification The methodology for verification
the model architecture is adequately defined ComponentModelling The methodology for definition of (detailed) component models is adequately defined HistoricalArg Argument over previous application of the methodology and similarity
Competency Safety engineers responsible for model constructon have received adequate training in modelling methodology Methodology Model-Based Safety Assessment Methodology {T}, that is [claimed to be] followed in construction of the Model {M}
Top Level Argument
Top Level Argument
ISSC 2010: The Illusion of Method - 14
ISSC 2010: The Illusion of Method - 15
Only weak evidence for some goals
A challenge even for the traditional approaches More critical for MBSA
Assumptions log Part of the adequacy argument in the safety case
ISSC 2010: The Illusion of Method - 16
ISSC 2010: The Illusion of Method - 17
Use of traditional formats hides these challenges
What is being modelled: Clear conceptual methods definition How to model: Guidance
Comprehensive and public
Strategies for model review and selective simulation Industrial Application (alongside traditional methods)
In the system safety case
Identify, manage and justify in the model adequacy argument
ISSC 2010: The Illusion of Method - 18