Safety Rapporteur Report George Donohue Summary Observations 7 - PDF document

Safety Rapporteur Report George Donohue

Summary Observations • 7 papers: 1 European, 1 Joint, 5 US • Need for EARLY Safety Analysis Historical Analysis Required for Hypothesis Confirmation • • Analysis Prioritization Methodology is Required to Proceed from System CONOPS to Qualitative Fault Tree/Hazard Analysis to Quantitative Analysis Quantitative Analysis is Required to Provide System Specifications for • New Technology and Procedures Quantitative Analysis will Develop Validated Models that provide both • Normalization and Quantitative Safety Metrics for System Monitoring We have a Shortage of Trained Safety Analysts to deal with large • number of Issues to be Addressed Example Analysis Presented for New En-Route System Concept, • UAV’s in the NAS, Wake Vortex Encounter and Runway Incursion Severity

Barry Kirwan, EEC Safety Needs • Recent accidents involving ATM • Increasing traffic (capacity) • Advanced systems (2012; 2017; 2025) • Keep ATM safe • Anticipate & Resolve Problems • Learn before accidents occur

Barry Kirwan, EEC Guiding Principles • ATM must become a • ATM must be sure that learning organization the systems it is developing will deliver • ATM must have suitable the required safety methods with which to levels anticipate and protect itself against risks • ATM must retain its ‘High Reliability’ status • Safety must be built in and its ‘safe culture’ at the early stages of ATM system design, • The above collaboration right through to should be achieved implementation effectively and cost- efficiently • ATM must improve safety in key risk areas

Barry Kirwan, EEC Safety Methods • Toolbox of 30+ methods (FAA + Eurocontrol + ANSPs): – Hazard and human error identification – Representation in fault and event trees – Quantification of events & human errors; evidence from incidents/simulations – Analysis of dependence and common mode failures – Evaluation of uncertainty, sensitivity, and risk impact – Determination of safety requirements – Documentation for re-usability • Need more Analysts Trained to use these tools 1.6 Issue delegation instruction Plan: Do 1. Then do as appropriate. 1.6.1 1.6.2 1.6.3 1.6.4 1.6.5 Decide on Instruct pilot to Instruct pilot 'Heading Instruct pilot to 'Merge Instruct pilot 'Heading appropriate 'Remain behind' then remain behind' behind' then merge behind' instruction Plan: Do 1. Then do 2 if Plan: Do 1 throughout. Plan: Do 1 throughout. Plan: Do 1 throughout. required. Then do 3 to 5 Then do 2 if required. Then do 2 if required. Then do 2 if required. in order. Then do 3 to 6 in order. Then do 3 to 5 in order. Then do 3 to 6 in order. 1.6.2.1 1.6.3.1 1.6.4.1 1.6.5.1 Ensure Ensure Ensure Ensure applicability applicability applicability applicability conditions are conditions are conditions are conditions are met/maintained|| met/maintained|| met/maintained|| met/maintained|| 1.6.2.2 1.6.3.2 1.6.4.2 1.6.5.2 Issue Issue Issue Issue instruction(s) to instruction(s) to instruction(s) to instruction(s) to ensure ensure ensure ensure � S elect H F Issu e (e.g . “R eco ve ry fro m F ailu re”) applicability applicability applicability applicability conditions are conditions are conditions are conditions are met|| met|| met|| met|| W h at if? L ikely Im pact S afegu ards A ction 1.6.2.3 1.6.3.3 1.6.4.3 1.6.5.3 Issue 'remain Issue 'heading Issue 'merge Issue 'heading behind' then remain behind' then merge instruction|| behind' instruction|| behind' instruction|| instruction|| � � A n alys e L ikely Im p act & 1.6.2.4 1.6.4.4 Brainstorm What Ifs Receive pilot 1.6.3.4 Receive pilot 1.6.5.4 readback|| Receive pilot readback|| Receive pilot S afeg u ards fo r each W h at if readback|| readback|| 1.6.2.5 1.6.4.5 Click mouse 1.6.3.5 Click mouse 1.6.5.5 � A n alyse all o th e r co lum n s fo r e ach W h at if button A over Click mouse button A over Click mouse delegated a/c|| button A over delegated a/c|| button A over delegated a/c|| delegated a/c|| 1.6.3.6 1.6.5.6 Receive 'pilot Receive pilot's resuming' merging report|| distance report|| � S elect n ext H F Is su e (e.g . “S taffin g an d O rg an isatio n ”)

Barry Kirwan, EEC Safety in Design • EEC therefore carries out concept exploration and preliminary design • EEC research suggests that 50% 50% of accidents have their roots in the of accidents have their roots in the design phase design phase • EEC has a safety policy, and safety plans for sector tools, traffic flow, and airport research areas • Safety activities are ongoing for each project in these areas • Integrative project is ongoing to determine safety levels for the integrated vision for 2012

Mark Hansen, UC/B Historical Analysis for Hypothesis/Model Validation Analysis of Daily OE Count: Tower OE ∑ ∑ λ = α + β + β + β + γ ln( ) ln( OP ) ln( WI ) ln( Factor ) D i 0 i 1 i j ij k k j k Paramete Standard Description Estimate r Error α Intercept -17.00 0.57 β 0 1.97 0.09 Logarithm of operations β 1 -0.68 0.08 Logarithm of weather index β 2 0.33 0.03 Log of Airfield/Airspace Delay β 3 Log of Arrival Delay 0.04 0.03 Log of Downstream β 4 -0.15 0.04 Congestion Yearly dummy variable for γ 2001 0.22 0.03 2001 yearly dummy variable for γ 2002 -0.08 0.03 2002 0.264 Scale

John Shortle, GMU Process for New WV CONOPS Analysis Prioritization Concepts of Operations Conops Team Hazard Analysis Qualitative Analysis Initial n n -1 . . . . . . Scenario Safety 2 1 3 Ranking Analysis Team Quantitative Modeling Model A Model B Not Modeled Quantitative Analysis Safety Metrics

Safety Assessment for New En-Route System Concept (AAC) Concept Definition HARDWARE HARDWARE FAULT MASKING FAULT DEFINED NOMINAL NON-NOMINAL RECONFIGURATION STATE STATE Fault Identification MULTI-AC SINGLE AC SINGLE AC FAULT FAULT (TYPE 2) MODE TRANSITION (TYPE 1) unresolved fault resolved fault Fault Management Design EVENT TYPE k P[fault correction P[TSAFE fails] fails] α k β k Rate of Class k Non-conformance events λ f λ k Mission T k Failure γ k Avg. duration Rate of non-conformance λ P[TCAS fails|TSAFE fails] crit Rate of Critical Fault Analysis Miss Distance (N event types) Conflicts ACCIDENT RATE Safety Calculation (PER FLIGHT HR) k=1 1.0E-00 λ crit = 1.0E-01 0.00166 1.0E-02 1.0E-03 λ = 1.0E-04 0.005 1.0E-05 τ crit= 0.0111 1.0E-06 1.0E-07 α= 0.10 1.0E-08 β =0.01 ∗ 1.0E-09 1.0E-10 γ =0.1* 1.0E-11 1.0E-12 1.0E-14 John Andrews, MIT/LL

John Andrews, MIT/LL Safety Assessment 1.0E-08 1.0E-07 Rate of collisions due to specified 1.0E-06 fault type 0.00001 visual 0.0001 TCAS 0.001 TSAFE Rate of collisions to be resolved goal 0.01 AAC 0.1 1.0 10. Rate of Type k 100. faults 1000. Pcoll 10000 P[critical miss] 100,000 Rate of 1,000,000 fault 10,000,000 100,000,000 (AAC_B_03e-1)

UAV Safety Analysis Simulation Components Manned Aircraft Visual acquisition Pilot Aircraft response dynamic TCAS model model UAV Pilot UAV Comm TCAS Comm response dynamic model See-and-avoid model system = areas of planned growth James Kuchar, MIT/LL

WV encounter Probability Can Be Computed Using Aircraft and WV Stochastic Models • Two Phase WV Decay and Propagation Model (2P2) combined with Aircraft Arrival Flight Track Deviation Model (3 DOF) • Effects of Cross Wind and Cross Wind Variance can be evaluated • Encounter Probability and Severity can be Computed for any mixture of Aircraft Types Yue Xie, GMU

Severity of Runway Incursions • Safety Analysis is moving from just Event Counting to Modeling Events and Assessing Severity of Events Kim Cardosi, Volpe

Congratulations “S/V Esprit” The Best Boat Won From the “S/V Ana G”