Assignm ent of Assignm ent of Design Assurance Levels Design - - PowerPoint PPT Presentation

August 20, 2008 FAA Software & AEH Conference 1

SAE S1 8 & EUROCAE W G-6 3

Presented by Steve Beland Associate Technical Fellow Boeing Com m ercial Airplanes

Assignm ent of Assignm ent of Design Assurance Levels Design Assurance Levels

August 20, 2008 FAA Software & AEH Conference 2

Status of New Draft Guidance

SAE S-18 Airplane Safety Assessment Committee updating SAE ARP 4754

• Subcommittee: Fellowship of the DAL (FotDAL)

Regular coordination with EUROCAE WG-63 Mature text inserted as Sec 5.2 in Draft G2, S18 meeting Jan ’08

• Delicate consensus, WG-63 participated remotely
• Established baseline

Draft H2 contains April joint meeting effort

• Flowchart added
• Consensus strengthened, FotDAL done

August 20, 2008 FAA Software & AEH Conference 3

FotDAL Problem Statement

Initial DAL has not always been based on rigorous safety analysis Delineation of the architectural containment boundaries were not always properly defined Inconsistency between guidance sources Difficult to delineate the subtleties between “independence” and “dissimilarity” Probabilities have often been improperly linked to development assurance levels Present ARP 4754 Section 5.4 does not clearly distinguish between loss vs. erroneous output of a function (but it is there “between the lines” of Table 4)

August 20, 2008 FAA Software & AEH Conference 4

Situation

Use of ARP4754 is becoming more common

• ARP4754 was published in 1996
• Referenced in AMC 25.1309, AC 23.1309-1C and

draft 25.1309 (Arsenal) published in 2002 FAA Policy Memo allows ESF for Part 25

• Section 5.4 permits reduced level of process rigor

for development assurance activities provided proper safety analysis shows architectural containment

• Some perceived inconsistencies between ARP4754

and DO-178B and DO-254 exist (Ref. FAA Policy Statement ANM-10-117-09, Jan 2004)

August 20, 2008 FAA Software & AEH Conference 5

Related Changes to ARP4754

Revision A restructured to include both the aircraft and system development life cycle process models Document outline has a smoother flow Sections 4 through 10 combined into 2 sections

• Section 4 outlines the aircraft/system development

process model

• Section 5 addresses the integral processes used

during aircraft/system development New Section 5.2 is influenced by existing ARP 4754 Table 4 and the Functional Failure Path Analysis concept in RTCA DO-254 Appendix B

• Includes constraints to preclude extreme FFPA

interpretations

August 20, 2008 FAA Software & AEH Conference 6

Development vs. Design Assurance

Development Assurance:

• All of those planned and systematic actions used

to substantiate, at an adequate level of confidence, that errors in requirements, design and implementation have been identified and corrected such that the system satisfies the applicable certification basis. Note: The development assurance of an item has sometimes been called design assurance, such as in RTCA/EUROCAE DO-254/ED-80.

Development Assurance Level (DvAL)

• Applies to aircraft and systems
• V&V Process defined in SAE ARP4754A
• Includes Safety Analysis
• Influences Architectures

August 20, 2008 FAA Software & AEH Conference 7

Development vs. Design Assurance

Design Assurance:

• All of those planned and systematic actions used to

substantiate, at an adequate level of confidence, that design errors have been identified and corrected such that the items (hardware, software) satisfy the applicable certification basis.

Design Assurance Level (DsAL)

• Applies to items (software / hardware)
• V&V Process defined in RTCA DO-178 & DO-254

August 20, 2008 FAA Software & AEH Conference 8

n Systems in Airplane m Items in n Systems Airplane Systems

Layered Development Life Cycle

Aircraft Function DvAL System Function DvAL I tem DsAL

August 20, 2008 FAA Software & AEH Conference 9

Assurance Activities

Assurance Activity At system level ARP 4754/ED-79 At Hardware Level DO-254/ED-80 At Software Level DO-178B/ED-12B FHA Yes None None PSSA/SSA Yes None None FMEA Yes Implied via Functional Failure Path Analysis. None Common Cause Analysis (including SW and complex HW common mode failures) Yes None None Requirement Capture Yes Yes Yes Requirement Validation Yes Yes, for HW specific requirements Yes, for SW specific requirements Implementation Verification Yes Yes Yes Configuration management Yes Yes Yes Process assurance Yes Yes Yes

August 20, 2008 FAA Software & AEH Conference 10

FotDAL Approach to Assigning Levels

Existing ARP explicitly addresses system level (with

• nly some mention of airplane level)
• Level assignment/reduction applied to items

defined from the system architecture New ARP addresses airplane & system levels explicitly

• DvAL is effectively new for the new ARP and should

be assigned to the systems from the aircraft architecture using the PASA/PSSA

• DsAL assignment should be similar to existing ARP

Discusses “independence” rather than “dissimilarity” Emphasize assigning levels rather than reducing levels

• “Reduction” is a misnomer, but arises when a function

has a level lower than its parent function

August 20, 2008 FAA Software & AEH Conference 11

ARP4754A Draft Outline

4 AIRCRAFT & SYSTEM DEVELOPMENT PROCESS

• 4.1 Conceptual Aircraft/System Development Process
• 4.2 Aircraft Function Development
• 4.3 Allocation of Aircraft Functions to Systems
• 4.4 Development of System Architecture
• 4.5 System Implementation

5 INTEGRAL PROCESSES

• 5.1 Safety Assessment
• 5 .2 Developm ent Assurance Level ( DvAL) Assignm ent
• 5.3 Requirements Capture
• 5.4 Requirements Validation
• 5.5 Implementation Verification
• 5.6 Configuration Management
• 5.7 Process Assurance
• 5.8 Certification and Regulatory Authority Coordination

August 20, 2008 FAA Software & AEH Conference 12

System Development Process Model

CONCEPT DESIGN FUNCTION DEVELOPMENT FUNCTION ALLOCATION ARCHITECTURE DEVELOPMENT REQUIREMENT ALLOCATION SYSTEM IMPLEMENTATION PLANNING

• 5.1 SAFETY ASSESSMENT
• 5.2 DEVELOPMENT ASSURANCE
• 5.3 REQUIREMENTS CAPTURE
• 5.4 REQUIREMENTS VALIDATION
• 5.6 CONFIGURATION MANAGEMENT
• 5.7 PROCESS ASSURANCE
• 5.8 CERTIFICATION & REGULATORY GUIDANCE COORDINATION

AIRCRAFT/SYSTEM DEVELOPMENT PROCESS 4.1 4.2 4.3 4.4 4.5 4.6 3.0

DATA & DOCUMENTATION

6.0

5.5 IMPLEMENTATION VERIFICATION

INTEGRAL PROCESSES

Figure 4 -2

August 20, 2008 FAA Software & AEH Conference 13

5.2 Assurance Level Assignment 5.2.1 Introduction 5.2.2 General Principles 5.2.3 Independence Attributes 5.2.4 DvAL & DsAL Assignment Guidelines

August 20, 2008 FAA Software & AEH Conference 14

DvAL/DsAL Assignment Process

Figure 5 -2

August 20, 2008 FAA Software & AEH Conference 15

Terminology

Functional Failure Set:

A single member, or a specific group of members (not necessarily limited to one system) whose anomalous behavior (random failure or systematic error) leads to a top level Failure Condition. An FFS member can be related to a function, a sub-function, or an item.

August 20, 2008 FAA Software & AEH Conference 16

DvAL Assignment General Principles

Catastrophic Failure Condition:

• At least 1 development process is Level A, or at

least 2 independent development processes are Level B, but none any lower than their individual hazard and no lower than Level C

• Overall integration process is Level A

Hazardous Failure Condition:

• At least 1 development process is Level B, or at

least 2 independent development processes are Level C, but none any lower than their individual hazard and no lower than Level D

• Overall integration process is Level B

August 20, 2008 FAA Software & AEH Conference 17

Independence Attributes

Functional: members with different fcns & reqts

• Common requirements errors
• Requirements interpretation errors

Design: members with different designs

• Hardware or software design errors
• Software language or HDL errors
• Design tool errors

Others: do not influence DvAL/ DsAL assignment

• Physical

Redundancy, installation

• Process

Between independent designs or functions Between development/design vs. verif/validation

August 20, 2008 FAA Software & AEH Conference 18

Independence Attributes

DvAL considers the functional independence of the aircraft (or system) functions DsAL considers the design independence of items Once the DsALs are assigned to items, they should be fed back to the system and aircraft processes to ensure that no common mode is inadvertently introduced that violates any claimed functional independence. The assertion of independence needs to be substantiated & address potential common-modes One type of independence does not necessarily imply the other

August 20, 2008 FAA Software & AEH Conference 19

DvAL/DsAL Assignment Process

Development Assurance Level (DvAL)

• Aircraft and system levels assigned within Aircraft

Level FHA & PASA

• Validated with Aircraft & System level Safety Analysis

Design Assurance Level (DsAL)

• Assigned from System Level FHAs & PSSAs
• Validated per System level Safety Analysis and

Component Functional Failure Analysis

• Must trace up to upper level functions’ DvAL so that it

is not decomposed/assigned more than once (e.g. 4 Level D items assuring a Level A aircraft function).

• Non-complex items that are fully and deterministically

tested and analyzed may be considered Level A

August 20, 2008 FAA Software & AEH Conference 20

Table 5-2: DvAL Assignments (paraphrased) Two or more Independent FFS Members One

FFS Members

Option 1 Option 2 CAT A

A for 1 member, more members assigned per its failure cond (but ≥C). B for 2 members, more members assigned per its failure cond (but ≥C).

HAZ B

B for one member, more members assigned per its failure condition (≥D) C for 2 members, more members assigned per its failure condition (≥D)

MAJ C

C for one member, more members assigned per its failure condition D for 2 members, more members more per its failure condition

MIN D

D for one member, more members assigned per its failure condition members assigned per its failure condition

None E

E E

Top Event

August 20, 2008 FAA Software & AEH Conference 21

DEVELOPMENT ASSURANCE LEVEL functional failure sets with multiple independent members (note 2) functional failure sets with a single member

• r with non-

independent members Option 1 (NOTE 3) Option 2 Catastrophic DvAL A (NOTE 1) DvAL A for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level C for the additional members). DvAL B for two of the members leading to top level failure condition. The other member(s) at the level associated with the most severe individual effects

• f an error in their development process for all

applicable top level Failure Conditions (but no lower than level C for the additional member(s)). Hazardous DvAL B DvAL B for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions (but no lower than level D for the additional members). DvAL C for two of the members leading to top level failure condition. The other members at the level associated with the most severe individual effects

• f an error in their development process for all

applicable top level Failure Conditions (but no lower than level D for the additional members). Major DvAL C DvAL C for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions. DvAL D for two of the members leading to top level failure condition. The other members at the level associated with the most severe individual effects

• f an error in their development process for all

applicable top level Failure Conditions. Minor DvAL D DvAL D for one member, additional member(s) contributing to the top level failure condition at the level associated with the most severe individual effects of an error in their development process for all applicable top level Failure Conditions. DvAL of each member at the level associated with the most severe individual effects of an error in their development process No Safety Effect DvAL E DvAL E DvAL E Top Event / Failure Condition Classification

August 20, 2008 FAA Software & AEH Conference 22

Notes after DvAL/DsAL Table

1. When a FFS has a single member and the mitigation strategy for systematic errors is to be DvAL A alone, then the applicant may be required to substantiate to the Certification Authorities that the applicant’s development assurance activities have the rigor to provide for an acceptable level of safety.

• A single Level A member is OK, but may get more scrutiny

2. When decomposing an aircraft function it is necessary to stay in the same row no matter the number of functional decompositions performed (e.g. any degree of decomposition from a level A function should include at least one level A or 2 level B functions 3. If there is a large disparity on the numerical availability of the members in the functional failure set, the higher level DvAL should generally be assigned the higher availability 4. Some classes of 14CFR Part 23 /CS-23 aircraft will allow DvALs lower than derived in this process. See the current FAA AC23.1309 and EASA policy for guidance.

August 20, 2008 FAA Software & AEH Conference 23

DsAL Assignment Cases:

No independence:

• Assign DvAL & DsAL for top FC (DvAL = DsAL)

1 Both functional & design independence:

• Assign DvAL using Table 2, then DsAL using Table 2

staying in the same row as the top failure condition

• Watch cross-products of one member’s DvAL and

another member’s DsAL; re-assess assignment for FFS not meeting general principles

2 Functional but no design independence:

• DsAL of common items per top-level failure condition
• Ensure partitioning ensure functional independence?

May lead to re-evaluation of DvAL if not

3 Design but no functional independence:

• Assign DsAL using Table 2, staying in the same row

as the top-level failure condition

August 20, 2008 FAA Software & AEH Conference 24

Level Assignment Flow Fig 5-5

(1 of 2, DvAL)

CAN FUNCTIONAL INDEPENDENCE BE CLAIMED? (5.2.4.1) FAILURE CONDITIONS FROM THE FHA SELECT ONE (OR NEXT) FC AND ITS FFS CAN DESIGN ASSIGN DvAL = TOP DvAL PER COL 2 OF TABLE 2 ASSIGN DvAL PER OPTION 1 OR 2 OF TABLE 2 ASSIGN DsAL PER OPTION 1 OR 2 OF TABLE 2 USING SAME ROW AS USED TO HAVE ALL FCs FOR THIS FUNCTION BEEN ASSESSED? FUNCTION DvAL ITEM DsAL COMPILE A LIST OF ALL FUNCTIONS AND THEIR DvALs FINAL DvAL MUST SATISFY ALL APPLICABLE FCs. NO NO YES YES ASSIGN DvAL TO AIRCRAFT-LEVEL FUNCTION (TOP DvAL) PER TABLE 1

Function DvAL Item DsAL

August 20, 2008 FAA Software & AEH Conference 25

Level Assignment Flow Fig 5-5

(2 of 2, DsAL)

CAN DESIGN INDEPENDENCE BE CLAIMED? (5.2.4.2) ASSIGN ITEM DsAL = TOP DvAL (5.2.4.2.2) ASSIGN DsAL PER OPTION 1 OR 2 OF TABLE 2 USING SAME ROW AS USED TO DETERMINE DvAL; CHECK CROSS-PRODUCTS OF DvAL & DsAL IF DsAL DIFFERS FROM ITS DvAL (5.2.4.2.1, .3) THIS FUNCTION BEEN ASSESSED? HAVE ALL FCs FOR THIS ITEM BEEN ASSESSED? FUNCTION DvAL ITEM DsAL COMPILE A LIST OF ALL FUNCTIONS AND THEIR DvALs FINAL DvAL MUST SATISFY ALL APPLICABLE FCs. FINAL DsAL MUST SATISFY ALL FCs (AND MAY NEED TO BE REASSIGNED FOR COMBINATION OF ALL APPLICABLE FCs) NO NO NO YES YES YES

Function DvAL Item DsAL

August 20, 2008 FAA Software & AEH Conference 26

CAN FUNCTIONAL INDEPENDENCE BE CLAIMED? (5.2.4.1) FAILURE CONDITIONS FROM THE FHA SELECT ONE (OR NEXT) FC AND ITS FFS CAN DESIGN INDEPENDENCE BE CLAIMED? (5.2.4.2) ASSIGN DvAL = TOP DvAL PER COL 2 OF TABLE 2 ASSIGN DvAL PER OPTION 1 OR 2 OF TABLE 2 ASSIGN ITEM DsAL = TOP DvAL (5.2.4.2.2) ASSIGN DsAL PER OPTION 1 OR 2 OF TABLE 2 USING SAME ROW AS USED TO DETERMINE DvAL; CHECK CROSS-PRODUCTS OF DvAL & DsAL IF DsAL DIFFERS FROM ITS DvAL (5.2.4.2.1, .3) HAVE ALL FCs FOR THIS FUNCTION BEEN ASSESSED? HAVE ALL FCs FOR THIS ITEM BEEN ASSESSED? FUNCTION DvAL ITEM DsAL COMPILE A LIST OF ALL FUNCTIONS AND THEIR DvALs FINAL DvAL MUST SATISFY ALL APPLICABLE FCs. FINAL DsAL MUST SATISFY ALL FCs (AND MAY NEED TO BE REASSIGNED FOR COMBINATION OF ALL APPLICABLE FCs) NO NO NO NO YES YES YES YES ASSIGN DvAL TO AIRCRAFT-LEVEL FUNCTION (TOP DvAL) PER TABLE 1

Whole Figure 5-5 For Your Handouts

August 20, 2008 FAA Software & AEH Conference 27

Special Cases:

• Non-complex items (e.g. mechanical parts, relays,

electro-mechanical devices, electro valves, servo valves, simple logic devices, etc.) when the requirements that are applicable to the design have been validated and the analysis and testing of the design for verification with those requirements are considered to provide a level of confidence equivalent to DsAL A. However, this is only when the non complex item is fully tested or fully analyzed relative to the identified failure conditions.

• Independent functions using common resources based
• n the same COTS designs (e.g. computers,

networks, interfaces) are likely to require careful consideration if assigned a DsAL of A. A practical case of functional and design independence is whereby independent functions are implemented in designs that are independent from one another.

August 20, 2008 FAA Software & AEH Conference 28

DvAL Considerations for Conditional Events

Arises for protection systems for environmental or intrinsic threats

• e.g. windshear, fire

DvAL should be consistent with reduction in safety margins Does not apply to operations intentionally planned (e.g. autolands, ETOPS missions)

August 20, 2008 FAA Software & AEH Conference 29

DvAL Assignm ent as a Function

• f an External Event

A B C

10-9 10-7 10-5 10-3

DvAL

Probability of the external event Legend: CAT Top Level Failure Condition HAZ Top Level Failure Condition 10-4 10-6

Figure 5 -5

August 20, 2008 FAA Software & AEH Conference 30

Outputs of DvAL/DsAL Assignment

Describe FFS members and explain their DvALs in Cert Plan and PASA/PSSA Describe items and explain their DsALs in applicable plans and PSSA Describe interactions between FFS members in PASA/PSSA

• Functional interactions
• Substantiate claimed functional & design independence
• Capture assumptions made in the supporting analyses
• Derive requirements needed to support assumptions and

analyses.

Substantiate that all planned activities are successfully accomplished in the ASA/SSA and Cert Summary.

August 20, 2008 FAA Software & AEH Conference 31

CAST Comments on Draft H

Comments received are in-work by S18 & WG-63 Some clarifications are needed A couple significant misunderstandings arising: 2 Level Bs don’t easily make a Level A function Even if the item (DsAL) can be Level B, the system & aircraft functions still need to be developed & integrated at DvAL A Not intended to be endorsement of n-version programming Concern of entering Table 5-2 more than once may lead to 4 Cs to meet Level A, etc. Text disallows moving from row to row, each use must stay in the same row. S18/WG-63 will disposition comments & reply

August 20, 2008 FAA Software & AEH Conference 32

More Difficult Than Algebra

First we had Algebra in math class Then Boolean Algebra in college Introducing……..

August 20, 2008 FAA Software & AEH Conference 33

DALgebra

A = A*C = B*B B = B = B*D = C*C Substituting……. B*D*C*C = A?

August 20, 2008 FAA Software & AEH Conference 34

Closing

DvAL and DsAL are based on safety analysis; do it early (and revisit it often as the system evolves)! Emphasize assigning levels rather than reducing levels PASA and PSSA should be used to derive requirements including DvAL/DsAL Development/Design Assurance can be an enabler to focus resources on aspects that matter most. Aim is to have consistent guidance in one place for use across a system The DsAL assigned to software or hardware should be about the same as determined when using existing ARP 4754.