Probabil babilisti istic c Sen ensor or Models els for Virtua - - PowerPoint PPT Presentation

2

Probabil babilisti istic c Sen ensor

• r Models

els for Virtua ual l Val alidat dation ion – Use e Ca Cases es an and Ben enef efits its

• Dr. Robin
• bin Sch

chuber bert Co Co-Founder der & C CEO BASEL SELAB ABS GmbH bH

Apply & Innovate 2016

3

BASE SELABS S enabl bles es data fusi sion

• n results.

lts.

4 Apply & Innovate 2016

 Data fusion enthusiasts  Team of 25 engineers, software developers and managers  Software supplier to OEMs and automotive suppliers  Active contributors to advances in research

Who we are

 Focus on multiple sensor scenarios – with software products and projects  Enable data fusion as a key technology for automated driving

What we do We partner

 Vector is strategic partner with 49% of shares  Partnership with simulation providers for improved virtual validation

5

A A brief ef surve rvey on dat ata fusion ion

Apply & Innovate 2016

6

Exampl mple of

• f a 36

360° data fusion ion sys ystem tem for automa

• mated

ted drivi iving ng

BASELABS‘ miss ssion ion is is to to provi vide de a a unified ied, , ambi biguit guity-fre free, , reliab able le repres resent ntatio ation of

• f the vehicle‘s environm

ironment nt

Apply & Innovate 2016

7

Data fusion

• n in a nutshe

shell

A A brief summ mmary ary of

• f Bayesian

sian filte teri ring ng and multi ti objects ts trac acking king

Apply & Innovate 2016

World Model

Posterior state vector

ො 𝑦𝑙−1

Prior state vector

ො 𝑦𝑙 Sensor Model

Sensor data hypothesis

Ƹ 𝑨𝑙 Sensor Data Evaluation Update

8

Virtual ual valid idatio tion in a nutshe shell ll

A A brief summ mmary ary of

• f model/so

software ftware/hard hardware ware-in in-the he-loo

• op

p (xiL iL) ) vali lida datio tion

Apply & Innovate 2016

World Model

Scenario Simulated entities

Sensor Model

Sensor data

DUT Feedback DUT = Device under Test

9

Data fusion

• n revisi

isited ted

Prope pertie rties s of

• f a

a good sens nsor

• r mode

del in Bayesian ian filtering ring

Apply & Innovate 2016

Sensor Model

Sensor data hypothesis

Ƹ 𝑨𝑙 Sensor Data Evaluation Predict

• High similarity between simulated and real

sensor data (in case of valid hypothesis)

• Similar level of preprocessing
• Mathematical sensor model needs to

capture sensor behavior

• Parameters of the model need to capture

properties of specific sensor device

• Model needs to deal with data variability
• Fast comparison between prediction and data

 Probabilistic sensor models (deterministic & probabilistic part)

10

Exampl mple: : Sensor

• r mode

del l for a m multi ti objects cts track cking ing sys ystem tem

Mode delled led pheno nome mena na

Apply & Innovate 2016

World model

• 3 object hypotheses with position, direction and speed

Sensor model: We expect

• 3 sensor detections
• of which some may not be detected (false negatives)
• whereas we might get some clutter (false positives) and
• the detections may not be precisely where expected (noise) plus
• they might be delayed (latency)

This expectation is evaluated against the actual sensor data DT

11

Use ca cases es an and req equir ireme ements nts for virtua tual val alidation dation

(Focu

• cus on

n simulat mulatio ion of

• f ADAS

AS-rel relat ated ed senso nsor data)

Apply & Innovate 2016

12

Use Case 1: „Lucky-Pa Path th-Testing sting“

Apply & Innovate 2016

• Test of developed system under optimal conditions
• Rationale: „If it does not even work with perfect data yet, I do not need to start testing with real data.“

Potential target audience:

• Developers of virtual proof-of-concept implementations
• Testers at the beginning of the test process

13

Use Case 2: „Virtual Validation“

Apply & Innovate 2016

• Ensure the correct functionality of the system
• Complement or even partial replacement of field testing
• Objective: Simulation should be as realistic as possible
• Limitation: Real-time requirements of test system (in particular for HiL-setups)
• Most realistic sensor models: Simulation on physical level (EM-waves)

Camera images Radar power Would physical simulation models be the solution if they were fast enough? Yes and No (a.k.a. it depends)

14

The required level of simulated sensor models depends on the DUT‘s input

Apply & Innovate 2016

Sensor

Image Example

Function Raw Data Detector/ Classifier Detections

Image Regions of Interest

Tracking Tracks

Object List

15

The required level of simulated sensor models depends on the DUT‘s input

Targe get t group A: Deve velo lope pers rs and test sters rs of

• f image

ge processin sing (dete tecto ctors rs/class lassifi ifiers rs)

Apply & Innovate 2016

Camera Detector/ Classifier DUT Image

• Input of DUT:

Camera images

• Requirement for virtual validation:

Simulated camera images that are as realistic as possible

• Optimal solution:

Sensor models on physical level (image rendering)

16

The required level of simulated sensor models depends on the DUT‘s input

Targe get t group A2: Deve velop

• pers

rs and teste ters rs of

• f complet

lete data processin sing chain includin ding image ge process ssing ng

Apply & Innovate 2016

Camera Detector/ Classifier DUT Image

• Input of DUT:

Camera images

• Requirement for virtual validation:

Simulated camera images that are as realistic as possible

• Optimal solution:

Sensor models on physical level (image rendering)

Tracking Function Detections Tracks DUT DUT

17

The required level of simulated sensor models depends on the DUT‘s input

Targe get t group B: Functio tion deve velop

• pers

rs and test sters rs

Apply & Innovate 2016

• Input of DUT:

Track list (e.g., list of vehicles in front of ego vehicle)

• Requirement for virtual validation:

Track lists that are as realistic as possible

• Optimal solution:

Sensor models on physical level (image rendering) Sensor model that behaves like a realistic tracker Smart sensors (e.g., MobilEye cameras or recent radars) do not even output raw data, just track lists.

Camera Function Image Tracks DUT

?

18

The required level of simulated sensor models depends on the DUT‘s input

Targe get t group B2: Tracki cking/Dat ng/Data a Fusion ion deve velop

• pers

rs and test sters rs

Apply & Innovate 2016

• Input of DUT:

Detections (e.g., list of image regions containing vehicles)

• Requirement for virtual validation:

List of detections that are as realistic as possible

• Optimal solution:

Sensor models on physical level (image rendering) Sensor model that behaves like a realistic detector/classifier

Sensor Tracking Raw Data Detections

?

DUT

19

Inte term rmedia diate te Summar mmary: y: Claims ms and Assumptio umptions ns

Apply & Innovate 2016

Idealized (=error free) sensor models cannot be more than a starting point Sensor models on physical level are a valuable solution for developers and testers of systems based on sensor raw data (e.g., detectors/classifiers) Sensor models on physical level are not suitable for developers and testers of data fusion, tracking, or functions (as well as users of smart sensors) For this target group, sensor models should simulate a realistic behavior on the correct processing level

20

Probabil babilisti istic sen ensor

• r model

dels

BASEL SELABS ABS Models dels for Carmake rmaker

Apply & Innovate 2016

21

There is is no no need to to re re-inv nven ent the wheel

Proba babil bilistic istic sens nsor

• r models

ls from m the tracki cking ng/da data ta fusi sion

• n domain

ain are designe igned for these se requi uirem rement nts

Apply & Innovate 2016

World Model

Posterior State vector

ො 𝑦𝑙−1

Prior state vector

ො 𝑦𝑙 Sensor Model

Sensor data hypothesis

Ƹ 𝑨𝑙 Sensor Data Evaluation Predict Update

22

BASELA LABS BS Model els s for CarMake aker (planne nned)

Apply & Innovate 2016

• Probabilistic sensor

models from BASELABS

• Development based on

extensive experience using real ADAS sensors

• Models can simulate

detector and tracking

• utput
• Tight integration IPG

Carmaker (including visualization, parameterization, simulation, and licensing)

23

Model elled led sensors

• rs

Apply & Innovate 2016

ADAS sens nsors

• rs with typical

al interfaces rfaces

Radar Detector

• Provides an interface typical for automotive radar sensors
• Delivers measurements in Polar coordinates relative to the sensor position
• Models specific errors of radar sensors delivering detections

Smart Camera

• Provides an interface typical for automotive smart camera systems
• Delivers measurements in Cartesian coordinates relative to the sensor position

Errors common to sensors

• Latency

24

True positives tives with measuremen urement t noise se

Simulatio lation n of a more realisti listic c radar ar sens nsor

• r

Apply & Innovate 2016

Phenomena modelled by BASELABS Models:

• False negatives (Objects that are not detected by

the sensor)

• False positives (Detections that do not originate

from real objects)

• True positives with measurement noise: Stochastic

variations of measured signals (range, velocity, azimuth angle)

• Cardinality: One object generates more than one

detection

25

Cardinali nality ty

Simulatio lation n of a more realisti listic c sens nsor

• r

Apply & Innovate 2016

Phenomena modelled by BASELABS Models:

• Cardinality: One object generates more than one detection
• Depends on the size of the object (trucks, cars, …)
• Depends on the distance of the object
• Depends on characteristics of specific sensor
• Enables simulation of data fusion for extended object tracking

26

Laten ency cy

Simulatio lation n of a more realisti listic c sens nsor

• r

Apply & Innovate 2016

Phenomena modelled by BASELABS Models for all sensors:

• Latency: Object detections represent time in the past (due

to delay caused by the signal processing and data transmission in real sensors) DT

27 Apply & Innovate 2016

With BASELABS Models for CarMaker, the developed perception algorithm or ADAS function can be tested in real-time with more realistic sensor data!

28

Par arame amete teriza rization tion of

• f probabi

babilis listic tic sen ensor sor model dels

Simu mula latin ing the statistic istical al beha havior vior of

• f specific

cific senso nsor devices vices

Apply & Innovate 2016

29

Main question: what are the right parameters?

• The parameters describing the quality of

the data of the probabilistic sensor model can be adopted to simulate different sensors

• General option for the selection of model

parameters

• Based on the user‘s experience
• Based on validation of real sensor to

determine its statistical parameters

Main question: what are the right parameters?

Select cting ing the right t model del paramete meters rs

Apply & Innovate 2016

Dete terminat rmination

• n of statist

atistic ical al parame ramete ters rs of the proba babil bilistic istic sens nsor

• r models

ls

Model/ Software/ Hardware in the loop

Simulation environment with ideal model

• f the virtual world

Observation

Parameters

30 Apply & Innovate 2016

Tool-suppor supporte ted d inspec ectio tion n of real sensor

• r data

BASEL ELABS BS Valida date te

• Integrated annotation of

sensor and reference data

• Visual inspection and

comparison of sensor data and reference data

• Automatic calculation of

geo-referenced sensor performance metrics

31 Apply & Innovate 2016

Inte tegrate grated d ca calcul culation ation of statisti istica cal l parameter meters

• „Heat map“ of detections on

the reference objects

• Exemplary error distribution

for azimuth, range, and range rate of a radar sensor

• Derivation of statistical

parameters, e.g.

• Measurement noise
• Detection rate
• Cardinality
• Latency

BASEL ELABS BS Valida date te

Azimuth Range rate Range

32

What at is is yo your opinion nion on the topic ic?