PURSS: Towards Perceptual Uncertainty Aware Responsibility Sensitive - - PowerPoint PPT Presentation

PURSS: Towards Perceptual Uncertainty Aware Responsibility Sensitive Safety with ML

Rick Salay,1 Krzysztof Czarnecki,1 Ignacio Alvarez,2 Maria Soledad Elli,2 Sean Sedwards,1 Jack Weast2

• 1Dept. Electrical and Computer Engineering, Univ. of Waterloo

2Intel Corporation, Automated Driving Group

1

Automated Driving Systems (ADS)

2

Perception Planning & control Sensing Actuation

ADS

World model

Automated Driving Systems (ADS)

3

Perception Planning & control Sensing Actuation

ADS

Traditional Safety Assurance

ADS Specification

verify

World model

Automated Driving Systems (ADS)

4

Perception Planning & control Sensing Actuation

ADS

Traditional Safety Assurance

RSS

Responsibility Sensitive Safety

ADS Specification

verify

World model

Responsible Sensitive Safety (RSS)

Formalizes “common sense safety” e.g., Do not hit the car in front

5

Responsible Sensitive Safety (RSS)

Do not hit the car in front

• Safe actions maintain

distance 𝑒𝑛𝑗𝑜

• If 𝑒𝑛𝑗𝑜 is breached, “proper

response” is safe action

6

Shalev-Shwartz, Shai, Shaked Shammah, and Amnon Shashua. "On a formal model of safe and scalable self-driving cars." arXiv preprint arXiv:1708.06374 (2017).

Responsible Sensitive Safety (RSS)

Do not hit the car in front

7

Problem: Assumes perfect perception Misperception -> wrong action

• > safety risk!

Automated Driving Systems (ADS)

8

Perception Planning & control Sensing Actuation

ADS

Traditional Safety Assurance

RSS

Responsibility Sensitive Safety

ADS Specification

??

verify verify

World model

Automated Driving Systems (ADS)

9

Perception Planning & control Sensing Actuation

ADS

Traditional Safety Assurance

RSS

Responsibility Sensitive Safety

ADS Specification

?? World model

World model

Automated Driving Systems (ADS)

10

Perception Planning & control Sensing Actuation

ADS

Traditional Safety Assurance

RSS

Responsibility Sensitive Safety

ADS Specification

??

Automated Driving Systems (ADS)

11

Perception Planning & control Sensing Actuation

ADS

Traditional Safety Assurance

RSS

Responsibility Sensitive Safety

ADS Specification

??

Is there another approach to perceptual safety?

Perceptual Uncertainty

• Uncertainty of perceptual component is cause
• f misperception

– many factors*: poor labeling, inadequate dataset coverage, etc.

• ML components can report their own

uncertainty!

– as long as they are calibrated…

12

*Czarnecki, Krzysztof, and Rick Salay. "Towards a framework to manage perceptual uncertainty

for safe automated driving." In International Conference on Computer Safety, Reliability, and Security, pp. 439-445. Springer, Cham, 2018.

PURSS

PURSS = perceptual uncertainty (PU) + RSS Safety Idea: PURSS formalizes this idea

13

Use perceptual uncertainty measure to make RSS rules appropriately cautious and limit safety risk

Precise World Model

14

Real-world situation

Pedestrian speed = 0.1 activity = walking

Perception (+ PU) Accuracy

Pedestrian speed = 0 activity = standing

True state (unknowable)

Misperception: precise but inaccurate

Perceptual Uncertainty Handling via Imprecise World Models

15

Real-world situation Perception (+PU) Accuracy

Pedestrian speed = 0 activity = standing

True state (unknowable)

Pedestrian speed = 0.1 activity = walking Pedestrian speed = 0 activity = standing

… PU -> Imprecise World Model (𝜷) Covers a “credible set” of world models with conf. level 𝛽

Probability 𝛽 that true world model is in the set

Perceptual Uncertainty Handling via Imprecise World Models

16

Real-world situation Perception (+PU) Accuracy

Pedestrian speed = 0 activity = standing

True state (unknowable)

Pedestrian speed = 0.1 activity = walking Pedestrian speed = 0 activity = standing

… PU -> Imprecise World Model (𝜷) Covers a “credible set” of world models with conf. level 𝛽

Probability 𝛽 that true world model is in the set

Safety parameter 𝛽 is set to desired level of safety

Perceptual Uncertainty Handling via Imprecise World Models

17

Real-world situation Perception (+PU) Accuracy

Pedestrian speed = 0 activity = standing

True state (unknowable)

Pedestrian speed = 0.1 activity = walking Pedestrian speed = 0 activity = standing

… PU -> Imprecise World Model (𝜷) Covers a “credible set” of world models with conf. level 𝛽

Probability 𝛽 that true world model is in the set

RSS rules are “lifted” to accept imprecise world models Result: exercises caution by limiting actions to those safe for any covered world model

Responsible Sensitive Safety (RSS)

Do not hit the car in front

18

Lifting: replace values with credible intervals corresponding to 𝛽

Responsible Sensitive Safety (RSS)

Do not hit the car in front

19

e.g., precise: 𝑤𝑔 = 30 𝑛/𝑡 PU: 𝜏

𝑔 2 = 1 𝑛/𝑡

lift to imprecise: 𝛽 = 68%: 𝑤𝑔 = 29,31 𝑛/𝑡 𝛽 = 95%: 𝑤𝑔 = 28,32 𝑛/𝑡 Lifting: replace values with credible intervals corresponding to 𝛽

Responsible Sensitive Safety (RSS)

Do not hit the car in front

20

e.g., precise: 𝑤𝑔 = 30 𝑛/𝑡 PU: 𝜏

𝑔 2 = 1 𝑛/𝑡

lift to imprecise: 𝛽 = 68%: 𝑤𝑔 = 29,31 𝑛/𝑡 𝛽 = 95%: 𝑤𝑔 = 28,32 𝑛/𝑡 Lifting: replace values with credible intervals corresponding to 𝛽

Given uncertainty 𝜏

𝑔 2,

increasing confidence 𝛽 ⇒ decreasing precision of 𝑤𝑔 ⇒ larger 𝑒𝑛𝑗𝑜 to be more cautious

Responsible Sensitive Safety (RSS)

Do not hit the car in front

21

e.g., precise: 𝑤𝑔 = 30 𝑛/𝑡 PU: 𝜏

𝑔 2 = 1 𝑛/𝑡

lift to imprecise: 𝛽 = 68%: 𝑤𝑔 = 29,31 𝑛/𝑡 𝛽 = 95%: 𝑤𝑔 = 28,32 𝑛/𝑡 Lifting: replace values with credible intervals corresponding to 𝛽

Given uncertainty 𝜏

𝑔 2,

increasing confidence 𝛽 ⇒ decreasing precision of 𝑤𝑔 ⇒ larger 𝑒𝑛𝑗𝑜 to be more cautious

Benefits and Costs

• Benefit: Safety parameter 𝛽 can be increased

to get as safe as you want

– RSS rules become correspondingly more cautious

• Cost: More cautious behaviour may negatively

impact progress

• Important future work: negotiating the trade-
• ff

22

Summary

• RSS provides a spec on planning & control

– supports traditional safety assurance

• Perception is hard to specify and needs ML

– different safety approach is needed

• PURSS approach to safety

– Set desired level of safety (𝛽) – Perceptual uncertainty →𝛽 imprecise world models – Lift RSS rules to be correspondingly cautious

• Much further work coming!

23