Embedded Perception & Risk Assessment for next Cars Generation - PowerPoint PPT Presentation

Embedded Perception & Risk Assessment for next Cars Generation Christian LAUGIER, Research Director at Inria Chroma Team & IRT Nanolec Christian.laugier@inria.fr Contributions from Mathias Perrollaz, Christopher Tay Meng Keat, Stephanie Lefevre, Javier-Ibanez Guzman, Amaury Negre, Lukas Rummerlhard, Tiana Rakotovao, Nicolas Turro, Julia Chartre, Jean-Alix David Séminaire “ Voiture Autonome: Technologies, Enjeux et Applications” February 10-11 2016, Paris (France) Asprom – UIMM – Cap’Tronic C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 1 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Socio-economic & Scientific Context  Perception for Autonomous Vehicles: New trend of automotive industry !  Perception is a bottleneck for Motion Autonomy  Strong improvements (sensors & algorithms) during the last decade  A Huge ADAS market: $16 billions in 2012 & Expected $261 billions in 2020 (f) CES 2015 & 2016 (Las Vegas) Valeo’s Cruise4U Mercedes F015 Audi A7  But… High Computational requirement & Insufficient Robustness are still an obstacle to the deployment Inria / Toyota Audi A7 Google Car (f) Forecasted US$ 260 Billion Global Market for ADAS Systems by 2020. ABI Research. 2013 . C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 3 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Socio-economic & Scientific Context  Perception Technologies are pushed forward by Automotive industry ? Technologies for Safety & Comfort Ownership & Affective behaviors Driving Assistance v/s Autonomous Driving Driving pleasure  Main Issues : Robustness, Efficiency (real time processing) , Dynamicity constraints … and also Miniaturization (Reducing Size / Cost / Energy consumption) Models & Algorithms for Embedded implementation Pedestrian Dynamic environments Free space Detected Car Embedded Appropriate world model Sw/Hw integration C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 4 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Addressed Problem & Challenges Robust Embedded Perception & Risk Assessment for Safe & Socially Compliant Navigation in Open & Dynamic Human Environments Complex Dynamic Scenes Road Safety campaign, France 2014 ADAS & Autonomous Driving Situation Awareness Anticipation & Prediction & Decision-making Main features  Dynamic & Open Environments (Real-time processing)  Incompleteness & Uncertainty (Model & Perception)  Human in the loop (Social & Interaction Constraints)  Hardware / Software integration (Embedded constraints) C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 5 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Key Technology 1: Bayesian Perception Sensors Fusion => Mapping & Detection Characterization of the Safe navigable space (local) Embedded Multi-Sensors Perception => Continuous monitoring the dynamic environment Scene interpretation => Using Context & Semantics  Main difficulties Noisy data, Incompleteness, Dynamicity, Discrete measurements + Real time !  Approach: Bayesian Perception  Reasoning about Uncertainty & Time window (Past & Future events)  Improving robustness using Bayesian Sensors Fusion  Interpreting the dynamic scene using Contextual & Semantic information C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 6 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Bayesian Perception : Basic idea Multi-Sensors Observations Lidar , Radar, Stereo camera, IMU … Bayesian Multi-Sensors Fusion Probabilistic Environment Model • Sensor Fusion • Occupancy grid integrating uncertainty Pedestrian • Probabilistic representation of Velocities Free space • Prediction models Black car Occupancy probability + Velocity probability + Motion prediction model C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 7 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

A new framework: Dynamic Probabilistic Grids A clear distinction between Static & Dynamic & Free parts [Coué & Laugier IJRR 05] [Laugier et al ITSM 2011] [Laugier, Vasquez, Martinelli Mooc uTOP 2015] Sensing (Observations) Velocity field (particles) 25 Hz Bayesian Filtering (Grid update at each time step) Occupancy & Velocity Probabilities Solving for each cell Sum over the possible antecedents A and  Bayesian Occupancy Filter (BOF) their states (O -1 V -1 ) => Patented by Inria & Probayes => Commercialized by Probayes Joint Probability decomposition: => Robust to sensing errors & occultation P(C A O O -1 V V -1 Z) = P(A) P(O -1 V -1 | A) P(O V | O -1 V -1 ) P(C | A V) P(Z | O C)  Used by: Toyota, Denso, Probayes, IRT Nanoelec / CEA  Academic license available C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 8 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Bayesian Occupancy Filter (BOF) – Outline Sensing Main features: • Estimate Spatial occupancy Grid update => Bayesian Filter • Analyze Motion Field (using Bayesian filtering) Occupancy Probability (P Occ ) + • Reason at the Grid level (i.e. no object segmentation Velocity Probability (P velocity ) at this reasoning level) Occupancy Grid Motion field (static part) (Dynamic part) Sensors data fusion + Bayesian Filtering Pedestrians Moving car Pedestrians Camera view C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 9 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Data fusion: The joint Occupancy Grid • Observations Z i are given by each sensor i (Lidars, cameras, etc) • For each set of observation Z i , Occupancy Grids are computed: P (O | Z i ) • Individual grids are merged into a single one: P (O | Z) Laser scanners (left + right) Joint Occupancy Grids C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 10 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Taking into account dynamicity: Filtered Occupancy Grid (Bayesian filtering) • Filtering is achieved through the prediction/correction loop (Bayesian Filter) => It allows to take into account grid changes over time Observations • Observations are used to update the environment model • Update is performed in each cell in parallel ( using BOF equations) Bayesian Filter • Motion field is constructed from the resulting filtered data (25 Hz) Instantaneous OG Filtered OG (includes motion field) Motion field is represented in orange color C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 11 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Underlying Conservative Prediction Capability => Application to Conservative Collision Anticipation Parked Vehicle Autonomous (occultation) Vehicle (Cycab) Thanks to the prediction capability of the BOF technology, the Autonomous Vehicle “anticipates” the behavior of the pedestrian and brakes (even if the pedestrian is temporarily hidden by the parked vehicle) C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 12 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Implementation & Experiments (Vehicles) CPU+GPU+ROS / Stereo vision + Lidars + GPS + IMU + Odometry Stereo & Mono cameras GPS + IMU + Odometry 2 Lidars IBEO Lux (8 layers) Manycore SThorm GPU Nvidia Jetson Miniaturization Integrated Perception Box Renault Zoé Toyota Lexus Movable & Connected C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 13 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Implementation & Experiments (Infrastructure) IRT Nanoelec experimental platform (connected infrastructure + 2 Twizy) Equipped Renault Zoé Connected Perception Box Towards a connected infrastructure Equipment for pedestrian crash test C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 14 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016

Experimental Results Stereo vision & Lidars Fusion (Inria / Toyota Lexus) [Perrollaz et al 10] [Laugier et al ITSM 11] IROS Harashima Award 2012 Stereo & Mono cameras 2 Lidars IBEO Lux (8 layers) Stereo Vision Bayesian Sensor Fusion (Stereo Vision + Lidars) (U-disparity OG + Road / Obstacles classification) C. LAUGIER – “ Embedded Perception & Risk Assessment for next Cars Generation” 15 Séminaire Asprom-UIMM- Cap’Tronic “ Voiture Autonome ”, Paris, February 10-11 2016