[PPT] - Risk Structures: Concepts, Purpose, and the Causality Problem Mario PowerPoint Presentation

SLIDE 1

Risk Structures: Concepts, Purpose, and the Causality Problem

Mario Gleirscher University of York, UK June 26, 2019

Shonan, JP

SLIDE 2

Part I Risk-aware Systems: Abstraction by Example

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

52

SLIDE 3

Example: Air-traffjc Collision Avoidance System (TCAS) Example: Safe Autonomous Vehicle (SAV)

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

53

SLIDE 4

Risk Mitigation / Intervention / Enforcement

Approach: Active safety monitors, enforcement monitors

System/ Process Model MDP, LTS, HA Risk Structure Active Monitor Test model Active Monitor Implemen tation Monitored Process a b s t r a c t s f r

m

abstracts from synthesised from conforms to r e c

g

n i s e s / e n f

r

c e s p r

p

e r t y

RQ: How to build a mitigation monitor? / Which model to use? / Which abstraction? / What do we need to verify?

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

56

SLIDE 5

Example: Air-traffjc Collision Avoidance System (TCAS)

SLIDE 6

Example: Traffjc Collision Avoidance System (TCAS) Ego

Risk Model R Process Model P abstracts from p1 p4 e.g. red ”

1`severity 1`benefit ą c

p6 rl, uq mov Λ : coll 1 ´ Λ : fj n

1 Risk Factor

0coll tp1, p6u start coll tp4u

collision

t✘✘ m

v

{ ecoll u t ? u Σztmov{ecollu Σztmov{finu

2 Risk Factors before

0ncoll tp1, p3, p4, p6u start ncoll tp2, p5u

near-collision

t✘✘

✘

tmov{encollu t✘✘

✘

tmov{finu Σzttmov{encollu Σzttmov{finu

p1 start p2 p3 p4 rl4, u4q p5 rl5, u5q p6 mov Λ2 : encoll 1 ´ Λ2 : fin a1 a2 (TCAS move) tmov Λ5 : encoll Λ4 : ecoll a3 Λ6 : fin a5 a4

0ncoll, 0coll ncoll, 0coll 0ncoll, coll ncoll, coll e

n c

l

l

✚ ✚

e

c

l

l

ecoll encoll mncoll “ t m

v

{ f i n

✚ ✚

m “ mov{fin ěm

Risk Space

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

57

SLIDE 7

Example: Traffjc Collision Avoidance System (TCAS) Ego

Risk Model R Process Model P abstracts from p1 p4 e.g. red ”

1`severity 1`benefit ą c

p6 rl, uq mov Λ : coll 1 ´ Λ : fj n

1 Risk Factor

0coll tp1, p6u start coll tp4u

collision

t✘✘ m

v

{ ecoll u t ? u Σztmov{ecollu Σztmov{finu

2 Risk Factors before

0ncoll tp1, p3, p4, p6u start ncoll tp2, p5u

near-collision

t✘✘

✘

tmov{encollu t✘✘

✘

tmov{finu Σzttmov{encollu Σzttmov{finu

p1 start p2 p3 p4 rl4, u4q p5 rl5, u5q p6 mov Λ2 : encoll 1 ´ Λ2 : fin a1 a2 (TCAS move) tmov Λ5 : encoll Λ4 : ecoll a3 Λ6 : fin a5 a4

0ncoll, 0coll ncoll, 0coll 0ncoll, coll ncoll, coll e

n c

l

l

✚ ✚

e

c

l

l

ecoll encoll mncoll “ t m

v

{ f i n

✚ ✚

m “ mov{fin ěm

Risk Space

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

58

SLIDE 8

Example: Traffjc Collision Avoidance System (TCAS) Ego

Risk Model R Process Model P abstracts from p1 p4 e.g. red ”

1`severity 1`benefit ą c

p6 rl, uq mov Λ : coll 1 ´ Λ : fj n

1 Risk Factor

0coll tp1, p6u start coll tp4u

collision

t✘✘ m

v

{ ecoll u t ? u Σztmov{ecollu Σztmov{finu

2 Risk Factors before

0ncoll tp1, p3, p4, p6u start ncoll tp2, p5u

near-collision

t✘✘

✘

tmov{encollu t✘✘

✘

tmov{finu Σzttmov{encollu Σzttmov{finu

p1 start p2 p3 p4 rl4, u4q p5 rl5, u5q p6 mov Λ2 : encoll 1 ´ Λ2 : fin a1 a2 (TCAS move) tmov Λ5 : encoll Λ4 : ecoll a3 Λ6 : fin a5 a4

0ncoll, 0coll ncoll, 0coll 0ncoll, coll ncoll, coll e

n c

l

l

✚ ✚

e

c

l

l

ecoll encoll mncoll “ t m

v

{ f i n

✚ ✚

m “ mov{fin ěm

Risk Space

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

58

SLIDE 9

Example: Safe Autonomous Vehicle (SAV)

SLIDE 10

Risk Mitigation / Intervention / Enforcement

Approach: Active safety monitors, enforcement monitors

System/ Process Model MDP, LTS, HA Risk Structure Active Monitor Test model Active Monitor Implemen tation Monitored Process a b s t r a c t s f r

m

abstracts from synthesised from conforms to r e c

g

n i s e s / e n f

r

c e s p r

p

e r t y

RQ: How to build a mitigation monitor? / Which model to use? / Which abstraction? / What do we need to verify?

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

62

SLIDE 11

SAV: Low Level Vehicle Dynamics

r9 p “ 0, 9 v “ 0s v “ 0 ^ a “ 0

start

halt r9 p “ v, 9 v “ as 0 ă v ă l ^ a ą 0 accel r9 p “ v, 9 v “ 0s v “ l ^ a “ 0 drive at max speed r9 p “ v, 9 v “ as 0 ă v ă l ^ a ď 0 decel a ą 0 v = l a ă 0 a ą 0 a ď 0 v = 0

r 9

α “ 0s α “ 0

start

neutral r 9 α “ fs 0 ă y ă 5 turn left r 9 α “ 0s α ‰ 0 move straight r 9 α “ fs ´5 ă y ă 0 turn right y ą 0 ^ |v| ą 0 9 α “ 0 ^ α ‰ 0 y ą 0 ^ |v| ą 0 9 α “ 0 ^ α “ 0 y ă 0 ^ |v| ą 0 9 α “ 0 ^ α ‰ 0 y ă ^ | v | ą α “ 0 ^ 9 α “ 0

Longitudinal dynamics LoD

drive halt accel decel drive at max speed

Lateral dynamics LaD (relative to route segment)

neutral turn left turn right move straight

Overall low-level dynamics: drive LaD

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

63

SLIDE 12

SAV: Situational Perspective of Urban Driving

Mode model of the driving activity:

basic || supplyPower drive driveAtL4 || requestTakeOverByDr driveAtL1 || operateVehicle driveAtLowSpeed driveAtL4Generic exitTunnel driveThroughCrossing parkWithRemote autoOvertake driveAtL1Generic manuallyOvertake autoLeaveParkingLot manuallyPark leaveParkingLot steerThroughTrafficJam halt start

Integration with low level dynamics: In each mode, verify contract: inv ^ pre ñ wppdrive LaD, inv ^ postq

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

64

SLIDE 13

SAV: Risk Identifjcation and Assessment

Knowledge sources for risk/hazard identifjcation, e.g.

accident reports
domain experts
situation/activity model
local dynamics model
control system architecture
control software

Analysis techniques, e.g.

hazard identifjcation: FHA, PHL, …
process/scenario analysis: HazOp, LOPA, BA, STPA, …
causal reasoning: ETA, FMEA, FTA, Bowties, …

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

65

SLIDE 14

SAV: Situational Perspective of Urban Driving

Mode model of the driving activity:

basic || supplyPower drive driveAtL4 || requestTakeOverByDr driveAtL1 || operateVehicle driveAtLowSpeed driveAtL4Generic exitTunnel driveThroughCrossing parkWithRemote autoOvertake driveAtL1Generic manuallyOvertake autoLeaveParkingLot manuallyPark leaveParkingLot steerThroughTrafficJam halt start

Risk factors (Yap script):

1 HazardModel for "drive" { 3 OC alias "on occupied course" ; 5 CR alias "increased collision risk" ; 7 CC alias "on collision course" ; 9 ICS alias "inevitable collision state" ; 11 Coll alias "actual collision" ; 13 ES alias "perception system fault" ; 15 } 4.0 / Gleirscher / Shonan, JP/ June 26, 2019

66

SLIDE 15

Risk Structures: Tool Support and Recent Publications

nColl Coll prvnColl

c

enColl eColl

1 OperationalSituation "generic" {} 3 ControlLoop "Robot" for "generic" { emgBr alias "Emergency Brake"; 5 } 7 HazardModel for "generic" { nColl alias "near-collision" 9 mitigatedBy (PREVENT_CRASH.emgBr) direct; 11 Coll alias "collision" requires (nColl) 13 mishap; }

From Hazard Analysis to Hazard Mitigation Planning: The Automated Driving Case Mario Gleirscher(B ) and Stefan Kugele Technische Universit¨ at M¨ unchen, Munich, Germany {mario.gleirscher,stefan.kugele}@tum.de

Abstract. Vehicle safety depends on (a) the range of identified hazards

and (b) the operational situations for which mitigations of these hazards are acceptably decreasing risk. Moreover, with an increasing degree of autonomy, risk ownership is likely to increase for vendors towards regula- tory certification. Hence, highly automated vehicles have to be equipped with verified controllers capable of reliably identifying and mitigating hazards in all possible operational situations. To this end, available meth-

ds for the design and verification of automated vehicle controllers have

to be supported by models for hazard analysis and mitigation. In this paper, we describe (1) a framework for the analysis and design

f planners (i.e., high-level controllers) capable of run-time hazard iden-

tification and mitigation, (2) an incremental algorithm for constructing planning models from hazard analysis, and (3) an exemplary application to the design of a fail-operational controller based on a given control system architecture. Our approach equips the safety engineer with concepts and steps to (2a) elaborate scenarios of endangerment and (2b) design

perational strategies for mitigating such scenarios.

Keywords: Risk analysis · Hazard mitigation · Safe state · Controller design · Autonomous vehicle · Automotive system · Modeling · Planning 1 Challenges, Background, and Contribution Automated and autonomous vehicles (AV) are responsible for avoiding mishaps and even for mitigating hazardous situations in as many operational situations as possible. Hence, AVs are examples of systems where the identification (2a) and mitigation (2b) of hazards have to be highly automated. This circumstance makes these systems even more complex and difficult to design. Thus, safety engineers require specific models and methods for risk analysis and mitigation. As an example, we consider manned road vehicles in road traffic with an autopilot (AP) feature. Such vehicles are able to automatically conduct a ride

nly given some valid target and minimizing human intervention. The following

AV-level (S)afety (G)oal specifies the problem we want to focus on in this paper: SG: The AV can always reach a safest possible state σ wrt. the hazards identified and present in a specific operational situation os. c Springer International Publishing AG 2017

C. Barrett et al. (Eds.): NFM 2017, LNCS 10227, pp. 310–326, 2017.

DOI: 10.1007/978-3-319-57288-8 23

L. Bulwahn, M. Kamali, S. Linker (Eds.): First Workshop on

Formal Verification of Autonomous Vehicles (FVAV 2017). EPTCS 257, 2017, pp. 75–90, doi:10.4204/EPTCS.257.8 c

M. Gleirscher

This work is licensed under the Creative Commons Attribution License. Run-Time Risk Mitigation in Automated Vehicles: A Model for Studying Preparatory Steps Mario Gleirscher Faculty of Informatics Technical University of Munich Munich, Germany mario.gleirscher@tum.de We assume that autonomous or highly automated driving (AD) will be accompanied by tough assurance obligations exceeding the requirements of even recent revisions of ISO 26262 or SOTIF. Hence, automotive control and safety engineers have to (i) comprehensively analyze the driving process and its control loop, (ii) identify relevant hazards stemming from this loop, (iii) establish feasible automated measures for the effective mitigation of these hazards or the alleviation of their consequences. By studying an example, this article investigates some achievements in the modeling for the steps (i), (ii), and (iii), amenable to formal verification of desired properties derived from potential assurance obligations such as the global existence of an effective mitigation strategy. In addition, the proposed approach is meant for step-wise refinement towards the automated synthesis of AD safety controllers implementing such properties. 1 Introduction For many people, driving a car is a difficult task even after guided training and many years of driving practice: This statement gets more tangible when driving in dense urban traffic, complex road settings, road construction zones, unknown areas, with hard-to-predict traffic co-participants (i.e., cars, trucks, cyclists, pedestrians), bothersome congestion, or driving a defective car. Consequently, hazards such as drivers misjudging situations and making poor decisions have had a long tradition. Hence, road vehicles have been equipped with many sorts of safety mechanisms, most recently, with functions for “safety decision support” [7], driving assistance, and monitor-actuator designs, all aiming at reducing operational risk or making a driver’s role less critical. In AD, more highly automated mechanisms will have to contribute to risk mitigation at run-time and, thus, constitute even stronger assurance obligations [7]. In Sec. 1.1, we introduce basic terms for risk analysis and (run-time) mitigation (RAM) for automated vehicles (AV) as discussed in this work. Sec. 1.2 elaborates some of these assurance obligations. 1.1 Background and Terminology According to control theory, a control loop L comprises a (physical) process P to be controlled and a controller C, i.e., a system in charge of controlling this process according to some laws defined by an application and an operator [20]. The engineering of safety-critical control loops typically involves reducing hazards by making controllers safe in their intended function (SOTIF), resilient (i.e., tolerate disturbances), dependable (i.e., tolerate faults), and secure (i.e., tolerate misuse). In automated driving, the process under consideration is the driving process which we decompose into a set of driving situations S , see Sec. 2. Taxonomies of such situations have been published in, e.g. 154 8.4 Strukturen für die Gefahren- erkennung und -behandlung in autonomen Maschinen

Dr. Mario Gleirscher

Department of Computer Science, University of York und Fakultät für Informatik, Technische Universität München In diesem Abschnitt werden hochautomatisierte – insbesondere autonom handelnde – Maschinen (AM) wie zum Beispiel autonome mobile Roboter betrachtet. Man erwartet von solchen Syste- men, dass deren Regelungen in Gefahrensituationen ebenso nützliche Handlungsalternativen bieten wie im Normalbetrieb. Ausgehend von dieser Problemstellung wird nun eine werkzeug- gestützte Herangehensweise

i. für die Modellierung von Gefahrensituationen sowie
ii. für die Bewertung der Plausibilität und Vollständigkeit

solcher Modelle anhand eines Beispiels aus dem Bereich des automatisierten Fahrens (AF) diskutiert. 1 Motivation Im Rahmen der gefahrenreduzierenden Absicherung von Syste- men ist es eine Herausforderung, möglichst starke Sicherheits- eigenschaften für autonome, hochautomatisierte Maschinen schon zum Entwurfszeitpunkt festzulegen und Regler solcher Maschinen für die Einhaltung dieser Eigenschaften zur Laufzeit zu entwerfen. Im Folgenden werden formale Strukturen, welche für die Modellbildung und später für die detaillierte Entwicklung von Reglern hilfreich sind, besprochen. Die Motivation, solche Strukturen zu nutzen, resultiert aus § Bestrebungen, die Risikobewertung und den Verlässlich- keitsnachweis für allgemeine Systemklassen durch speziali- sierte Modellbildung zu unterstützen (siehe Kapitel 4/ Bertsche et al.), 1 |

Vgl. Kugele et al. 2017.

2 |

Vgl. Alexander et al. 2009.

3 |

Vgl. McDermid 2001 und Kumamoto 2007 diskutieren „As Low As Reasonably Practicable“ (ALARP).

4 |

Vgl. Schnieder/Schnieder 2009, Schnieder/Schnieder 2008, Schnieder/Drewes 2008.

5 |

Vgl. Lund et al. 2011.

§ Erfahrungen in der Architekturabsicherung komplexer einge- betteter Systeme1 und § der Beobachtung, dass einige Empfehlungen für die Gewähr- leistung von AM-Sicherheit nicht eindeutig und vollständig sind.2 2 Hintergrund In diesem Abschnitt werden einige Begriffe aus der Literatur und Vorarbeiten des Autors dargestellt, auf denen die spätere Diskus- sion aufbaut. 2.1 Allgemeine Grundlagen In der Risikoanalyse wird bewertet, inwiefern eine Gefahrenquel- le (Aggressor) ein Risiko für ein Schutzziel (zum Beispiel Safety, Security, körperliche Unversehrtheit) eines Schutzobjekts (im Englischen: asset) darstellt und inwieweit ein Schutzmechanis- mus (auch Beschützer, Sicherheitsfunktion) dieses Risiko wenigs- tens auf ein akzeptables Restrisiko3 reduzieren kann. Häufjg wird dazu eine Menge wahrscheinlicher Szenarien in Form von potenziell unendlichen Ursache-Wirkungs-Ereignisketten betrachtet, wobei die ultimativen und unerwünschten Auswirkungen als Un- glücks-, Unfalls- oder Schadensereignis und alle Ereignisse auf diesem Wege als Zusammensetzung von potenziell verursachen- den Faktoren beschrieben werden. Hierzu wird weiter unten von Kausalfaktoren und -strukturen gesprochen. Ein kompatibler Be- griffsrahmen wird in den Kapiteln 3/Schnieder und Schnieder, 5/Beyerer und Geisler sowie 7.1/Vieweg ausführlicher behandelt. Dieser vielfach diskutierte Begriffsrahmen lässt sich auf ganz un- terschiedliche Bereiche anwenden, zum Beispiel technische An- lagen, IT-Systeme, in der Patientenbehandlung im Krankenhaus, im unternehmerischen Projektmanagement, in Arbeitsprozessen im Hochbau oder an Flughäfen (siehe Kapitel 8.1/Wolf und Lichte sowie 8.2/Deutschmann et al.). Je nach Art des Schutz- ziels und -objekts gibt es spezifjsche Herangehensweisen zur RA sowie verschiedene Bezeichnungen, wie zum Beispiel funktiona- le Sicherheit in der Mechatronik und Automatisierungstechnik4

der Cyber Security für stark vernetzte IT-Systeme.5 Regelmäßig

wird versucht, Erkenntnisse aus verschiedenen Arbeitsfeldern arXiv:submit/2663380 [cs.SE] 23 Apr 2019 RISK STRUCTURES: TOWARDS ENGINEERING RISK-AWARE AUTONOMOUS SYSTEMS A PREPRINT Mario Gleirscher Computer Science Department, University of York, York, UK∗ April 23, 2019 ABSTRACT Inspired by widely-used techniques of causal modelling in risk, failure, and accident analysis, this work discusses a com positional framework for risk modelling. Risk models capture fragments of the space of risky events likely to occur when operating a machine in a given environment. More-

ver, one can build such models into machines such as autonomous robots, to equip them with the

ability of risk-aware perception, monitoring, decision making, and control. With the notion of a risk factor as the modelling primitive, the framework provides several means to construct and shape risk models. Relational and algebraic properties are investigated and proofs support the validity and consistency of these properties over the corresponding models. Several examples throughout the discussion illustrate the applicability of the concepts. Overall, this work focuses on the qualitative treatment of risk with the outlook of transferring these results to probabilistic refinements of the discussed framework. Keywords Causal modelling · risk · analysis · modelling · safety monitoring · risk mitigation · robots · autonomous systems Contents 1 Introduction 2 1.1 Abstractions for Machine Safety and Risk Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Contributions and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Background 7 2.1 Notions of Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 The Risk of Undesired Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Formal Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3 Risk Elements 9 ∗Correspondence and offprint requests to: Mario Gleirscher, Univerity of York, Deramore Lane, Heslington, York YO10 5GH,

UK. e-mail: mario.gleirscher@york.ac.uk

This work is supported by the Deutsche Forschungsgemeinschaft (DFG) under Grants no. GL 915/1- 1 and GL 915/1-2. c

2019.

This manuscript is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/. Reference Format: Gleirscher, M.. Risk Structures: Towards Engineering Risk-aware Autonomous Systems (April 23, 2019). Unpublished working paper. Department of Computer Science, University of York, United Kingdom.

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

67

SLIDE 16

Risk Factors (Basic Phase Model)

0f f f ef …endangerment

f

n

nominal

peration…

mf …mitigation recovery… mf

r

mf

d …direct mitigation

endangerment … ef

f

e

…endangered

peration
f

m …mitigated operation

Purposes:

Modelling primitive for risk space exploration
Semantics of basic events in DFTs or DFRTs
Synthesis of local enforcement monitors

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

68

SLIDE 17

SAV: Situational Risk Space

CCOCICSCR CollCR CCOCCR CR OCICSColl OCCR CCICSCR CCOCColl CCCollCR CCOCICSColl CCICSColl ICS ICSColl OCColl OCICSCR CCOC OCICSCollCR CCICS ICSCollCR CCOCICS ICSCR CC OCCollCR OCICS CCCR CCOCCollCR Coll CCColl CCOCICSCollCR CCICSCollCR OC e Vehicle ICS e Vehicle OC e Vehicle OC e Vehicle CC e Vehicle CC e Vehicle Coll e Vehicle CR e Vehicle ICS e Vehicle CC e Vehicle ICS e Vehicle CR e Vehicle OC e Vehicle Coll e Vehicle CR e Vehicle Coll e Vehicle CR e Vehicle CC e Vehicle ICS e Vehicle Coll e Vehicle Coll e Vehicle CR e Vehicle ICS e Vehicle ICS e Vehicle OC e Vehicle CR e Vehicle OC e Vehicle Coll e Vehicle ICS e Vehicle Coll e Vehicle ICS e Vehicle CR e Vehicle OC e Vehicle CR e Vehicle OC e Vehicle OC e Vehicle ICS e Vehicle CC e Vehicle CC e Vehicle ICS e Vehicle OC e Vehicle ICS e Vehicle CC e Vehicle Coll e Vehicle ICS e Vehicle OC e Vehicle Coll e Vehicle ICS e Vehicle CC e Vehicle Coll e Vehicle CR e Vehicle OC e Vehicle CR e Vehicle Coll e Vehicle CC e Vehicle CC e Vehicle OC e Vehicle ICS e Vehicle CR e Vehicle Coll e Vehicle Coll e Vehicle OC e Vehicle OC e Vehicle CR e Vehicle CC e Vehicle Coll e Vehicle CR e Vehicle CC e Vehicle OC e Vehicle CR e Vehicle CC e Vehicle Coll e Vehicle CR e Vehicle CC e Vehicle CR e Vehicle ICS e Vehicle ICS e Vehicle OC e Vehicle CC e Vehicle CC e Vehicle Coll

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

69

SLIDE 18

SAV: Situational Risk Space (zoomed)

CCOCICSCR CCOCCR CCOCColl ICS ICSColl CCOC CR CCOCCollCR Coll CCOCICSCollCR

CR e Vehicle ICS e Vehicle CR V e Vehicle Coll e Vehicle Coll e ehicle ICS e Vehicle OC e Vehicle Coll e Vehicle CR e Vehicle OC e Vehicle OC e Vehicle ICS e Vehicle ICS e Vehicle CC e Vehicle CR e Vehicle Coll e Vehicle CC e Vehicle OC e Vehicle CR e Vehicle OC e Vehicle OC e Vehicle CC e Vehicle Coll e Vehicle CR e Vehicle e Vehicle ICS e Vehicle CC

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

69

SLIDE 19

Causality

(Lewis 1973)

Defjnition (Counterfactual Conditional) A Ñ C is nonvacuously true iff C holds at all the closest A-worlds. What are the closest A-worlds?

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

70

SLIDE 20

SAV: Risk Identifjcation and Assessment Yap script

OperationalSituation "drive" 2 { include "envPerc"; 4 } 6 ControlLoop "Vehicle" for "drive" { 8 replan alias "Slow-down || re-plan route"; brake alias "Standard brake"; 10 swerve alias "Short-term circumvention

f obstacle";

EB alias "Emergency brake"; 12 accel alias "Accelerate"; airbag alias "Front airbag"; 14 } 16 HazardModel for "drive" { 18 OC alias "on occupied course" mitigatedBy (PREVENT_CRASH.replan) 20 direct ; 22 CR alias "increased collision risk" requires (OC) 24 deniesMit (OC) excludes (OC) 26 mitigatedBy (PREVENT_CRASH.EB) ; 28 CC alias "on collision course" requires (CR) 30 deniesMit (CR,OC) excludes (CR,OC) 32 mitigatedBy (PREVENT_CRASH.swerve) ; 34 ICS alias "inevitable collision state" requires (CC) 36 excludes (CC,CR,OC) causes (Coll) 38 mitigatedBy (PREVENT_CRASH.EB) ; 40 Coll alias "actual collision" requires (ICS) 42 excludes (CC,CR,OC,ICS,ES) mitigatedBy (ALLEVIATE.airbag) 44 mishap ; 46 ES alias "perception system fault" excludes (CC,CR,OC,ICS) 48 deniesMit (OC,CC) ; 50 } 4.0 / Gleirscher / Shonan, JP/ June 26, 2019

71

SLIDE 21

SAV: Situational Risk Space

ES CC OC CR Coll

e Vehicle ES e Vehicle CollICS e Vehicle CR e Vehicle CC e Vehicle OC e Vehicle ES e Vehicle ES e Vehicle ES

Approach: LOPA/BA to create chain of possible interventions

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

72

SLIDE 22

SAV: Situational Risk Space

ES CC CC ES CR Coll OC Coll CR eES eES eES mCC

e

eCR prvCC

c

mColl

e

eES eCC eOC prvOC

c

prvCR

c

attColl

alv

eES rES eCollICS mCR

e

fopES

fb

eES

Approach: LOPA/BA to create chain of possible interventions Layered intervention pattern for SAV obstacle avoidance Nice side-effect: Use pattern as enhanced phase model for similar risk factors

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

72

SLIDE 23

Taxonomy of Mitigations

Overall Safety Safety Constraints Fault Containment Fail- Safe Fault Detection Fault Avoidance Preventive Safety Passive Safety Error- Correcting Codes Redundancy Fail- Operational Limp- Home Fail- Silent Fail- Secure Condition Monitoring Checking Pre-Crash Assistance Obstacle Avoidance Limiter Protection partially realizes Override fully realizes Non- Interference Sanity Check Vigilance Check Warning Masking Voting Degradation Recovery Rollback Repair Simplicity Substitution Replication Diversity Barrier Comparison Shutdown Interlocking Separation Stabilization Reconfiguration Notification Filter

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

73

SLIDE 24

Taxonomy of Mitigations

F Cont Fail- Safe Fault Detection Preventive Safety Fail- Operational Limp- Home Fail- Silent Fail- Secure Checking Pre-Crash Assistance Obstacle Avoidance Vigilance Check Warning Degradation Recovery Repair Comparison Shutdown Reconfiguration Filter

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

73

SLIDE 25

SAV: Situational Risk Space (Monitor Candidate)

ES CC CC ES CR Coll OC Coll CR eES eES eES mCC

e

eCR prvCC

c

mColl

e

eES eCC eOC prvOC

c

prvCR

c

attColl

alv

eES rES eCollICS mCR

e

fopES

fb

eES

Approach: LOPA/BA to create chain of possible interventions = Specifjcation of a valid safe system expected order violated Ñ lack of observability, incomplete monitor, context mismatch?

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

74

SLIDE 26

Risk Structures: Templates for Causal Reasoning

A risk structure R is valid iff for s, s1, s2, s3 P RpFq, s3 P Mishaps, e, m, e1 P Σ˚ @s, s3 P RpFq, t P R Ds1, s2 P RpFq, m P Σ˚ : t “ eme1^ s s’ s” s”’ e m e’

Defjnition (Mitigation from Counterfactual Perspective)

m is mitigation of cause c ‰ s2 of a mishap s3 iff e1 gets unlikely. (s2, e1 form the counterfactual.) Proof obligations for each t: Check that, from s,

1. s2 is actual cause of s3,
2. s1 is recognisable,
3. from s1, m reduces s2.

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

75

SLIDE 27

Risk Mitigation / Intervention / Enforcement

Approach: Active safety monitors, enforcement monitors

System/ Process Model MDP, LTS, HA Risk Structure Active Monitor Test model Active Monitor Implemen tation Monitored Process a b s t r a c t s f r

m

abstracts from synthesised from conforms to r e c

g

n i s e s / e n f

r

c e s p r

p

e r t y

RQ: How to build a mitigation monitor? / Which model to use? / Which abstraction? / What do we need to verify?

4.0 / Gleirscher / Shonan, JP/ June 26, 2019

76