Cooperative Autonomous Driving and Interaction with Vulnerable Road - - PowerPoint PPT Presentation

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

Cooperative Autonomous Driving and Interaction with Vulnerable Road Users

Miguel Ángel Sotelo

miguel.sotelo@uah.es Full Professor University of Alcalá (UAH) SPAIN

9th Workshop on PPNIV – Keynote

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

2

Motivation Autonomous Cooperative Driving GCDC Results Interaction with VRUs Conclusions and Future Work

Content

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

3

• Despite the great development in the past years, there

are still some major limitations in Autonomous Driving: Motivation

Legal framework Legal framework Legal framework Navigation Navigation Navigation Reliability Reliability Reliability Efficiency Efficiency Efficiency Social Acceptance Social Acceptance Social Acceptance

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

4

• Navigation:
• Enriched maps are needed (2 Gb/Km).
• International Consortia: BMW, Daimler, Audi (HERE).
• Online data acquisition and map building.

Limitations of Autonomous Vehicles

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

5

• Reliability:
• Improvement in sensorial capabilities (adverse weather).
• Development of Cooperative Systems.
• Efficiency:
• Human-like decision making and maneuvering.
• Emulation of human driving by means of prediction of

intentions of other traffic agents, such as pedestrians and

• ther vehicles.
• There

is a need for enhanced cooperation and interaction capabilities.

Limitations of Autonomous Vehicles

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

6

Motivation Autonomous Cooperative Driving GCDC Results Interaction with VRUs Conclusions and Future Work

Content

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

7

• Techniques:
• Cooperation with other vehicles (autonomously or manually

driven) and with the infrastructure.

• Cooperation

with VRUs (Vulnerable Road Users) by prediction their intentions and trajectories.

Autonomous Cooperative Driving

• Limitations:
• Strong dependency on penetration rate.
• Goal:
• Increase reliability and efficiency of autonomous vehicles.

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

8

• GCDC 2016 (three tests):
• Platooning + Merging.
• Management of T-intersections.
• Management of emergency vehicles.
• Initiatives:
• European Commission: funding of research projects on

Cooperative Systems and Autonomous Driving (FP7 and H2020).

• Grand

Cooperative Driving Challenge (GCDC): International Competition

• n

Autonomous Cooperative Driving in Helmond (The Netherlands) in 2011 and 2016.

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

9

• GCDC 2016: Platooning + Merging

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

10

• GCDC 2016: Management of T intersections

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

11

• GCDC 2016: Management of emergency vehicles

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

12

• DRIVERTIVE –

General Architecture Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

13

• Data Fusion - Localization Example

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

14

• Communications System (ITS G5 V2V standard)

Messages types

• CAM (Cooperative Awareness Message): position, geometry and

vehicles dynamics.

• DENM

(Decentralized Environmental Notification Message): asynchronous messages from infrastructure o from other vehicles (e.g. emergency vehicles approaching).

• iCLCM (iGame Cooperative Lane Change Message): messages for

interaction protocol in different scenarios.

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

15

• Scenario 1: Platooning + Merging

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

16

• Scenario 1:

Platooning + Merging Behavior

• n

the left lane is different from that on the right lane

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

17

• Scenario 2: Management of T-intersections

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

18

• Scenario 2:

T-Intersections A safety distance must be kept at all times w.r.t the preceding vehicle

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

19

Analysis of the communication channel (CCDF – Complementary Cumulative Distribution Function)

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

20

Mean and Variance of UD (Car)

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

21

Mean and Variance of UD (Truck)

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

22

Analysis of the communication channel

• For some vehicles, the probability of large delays is

significant (>10%).

• The UD degrades with distance.
• Occlusions have a strong effect on delays:
• Trucks

are less

• ccluded

given that their antennas are located at a height of 3 meters.

• Other findings: DCC in a highly congested channel

is making some of the vehicles get stuck in Restrictive state and are not able to regularly access the channel.

• CAM and DENM in GCDC at 25 Hz.

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

23

Effect of UD on Emergency Braking during CACC

Autonomous Cooperative Driving

• There is a probability between 0.01 – 0.001 of

collision with the leading vehicle is

• nly

communications are used for CACC in a fleet of more than 4 vehicles.

• The

channel load is responsible for low reliability.

• An additional sensor is needed (radar).

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

24

Motivation Autonomous Cooperative Driving GCDC Results Interaction with VRUs Conclusions and Future Work

Content

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

25

• DRIVERTIVE –

University of Alcalá’ s team

• Autonomous Cooperative Vehicle (Velodyne, Radar, 3D Vision, Laser,

DGPS, CANBus, Communications, fully automated)

Autonomous Cooperative Driving

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

26

DRIVERTIVE at GCDC 2016

GCDC Results

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

27

DRIVERTIVE – Winner of the Prize to the Best Team with Full Automation in GCDC 2016

GCDC Results

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

28

GCDC 2016 GCDC Results

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

29

Motivation Autonomous Cooperative Driving GCDC Results Interaction with VRUs Conclusions and Future Work

Content

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

30

Motivation

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

31

Motivation

• Pedestrian Path Prediction in the Automotive:

– Further improvement in state-of-the-art ADAS by means

• f action classification

– Walking, Stopping, Starting, Bending-in – Improvement of accuracy in 30-50 cm: – Difference between effective and non-effective intervention in emergency braking systems – Initiation of emergency braking 0.16 s in advance can potentially reduce severity of accidents injuries by 50% – Early recognition of pedestrians stopping actions can provide more accurate last-second active interventions Strong gains are expected in the performance and reliability of active pedestrian protection systems Strong gains are expected in the performance and reliability of active pedestrian protection systems Strong gains are expected in the performance and reliability of active pedestrian protection systems

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

32

Proposed Approach Global Scheme

Off-line Motion Capture System Recovery of 3D pose and position Lateral predicted position On-line Probabilistic Training

Knowledge of Pedestrians dynamics

Stereo Cameras Transformation to latent space + prediction Geometric processing 3D Pedestrian Pose Estimation

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

33

Pedestrian Pose Measurement

Pedestrian Skeleton considered in this research

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

34

Pedestrian Pose Measurement

Method for Joints Extraction - Example

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

35

Pedestrian Pose Measurement

Method for Joints Extraction – Results

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

36

Pedestrian Pose Measurement

Body parts detection using Deep Learning

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

37

General Method - Overview

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

38

Activity Recognition - Example

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

39

Experimental Results

Detection Delay - Summary

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

40

Experimental Results

Probabilistic Action Classification

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

41

Experimental Results

Probabilistic Action Classification

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

42

Experimental Results

Video sequence showing prediction results

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

43

Intelligent Interface with VRUs

GRAIL – GReen Assistant Interfacing Light

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

44

Intelligent Interface with VRUs

GRAIL – GReen Assistant Interfacing Light

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

45

Motivation Autonomous Cooperative Driving GCDC Results Interaction with VRUs Conclusions and Future Work

Content

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

Conclusions and Future Work

Conclusions

• Autonomous Cooperative Systems will pave the way to the massive

and robust deployment of self-driving cars.

• The V2V communication link is still a weakness that needs further

attention from the scientific community in order to provide real-time and robust communication capability among large fleets of vehicles.

• Anticipating the intentions of other traffic participants, such as VRUs

and vehicles, is essential for mimicking human drivers behavior.

Future Work

• Enhancement of V2V communication channel for large fleets of

vehicles (antenna placement, frequency of data, etc.).

• Context-based action prediction using Probabilistic Graphical

Models (Bayesian Networks) is under development for VRUs and vehicles.

• Gaze direction, group behavior.

9th Workshop on Planning, Perception, and Navigation for Intelligent Vehicles. Vancouver, Canada, 24th Sept. 2017

Cooperative Autonomous Driving and Interaction with Vulnerable Road Users Thanks for your kind attention!

Miguel Ángel Sotelo

miguel.sotelo@uah.es Full Professor University of Alcalá (UAH) SPAIN

9th Workshop on PPNIV – Keynote