Connected Automated Driving Overview, design, and technical - - PowerPoint PPT Presentation
PROTECTED Connected Automated Driving Overview, design, and technical challenges Gaurav Bansal Toyota InfoTechnology Center, USA March 28, 2018 Automated Driving PROTECTED HIGHWAY AUTOMATED DRIVING URBAN AUTOMATED DRIVING 1 Sensors are at
PROTECTED
March 28, 2018
PROTECTED
1
HIGHWAY AUTOMATED DRIVING URBAN AUTOMATED DRIVING
Sensors are at the heart of Automated Driving Technology
PROTECTED
2
ACCURACY/ RAN RANGE WEATHER COST URB RBAN/ NLOS SC SCEN ENARI RIO MAI AIN CHALLENGES MAPP APPING ∆ O ∆ O Mig ight no not exis xist for
all US S roa
Er Errors s will ill exi xist. LOCALIZATION X O O X GPS S (inc (including futu future GPS3) S3) has has ch challenges s in in lan lane le level loc localization for
urban can anyon. O X ∆ O LiDa LiDar base based loc localization can an be be cos
and do does s no not work
wel ell in in bad bad wea eather. PER ERCEP EPTION ∆ X O X Lim Limited range (ar (around 50 50-80 80 m). ). Majo ajor ch chal allenges s in in NLOS and and bad bad wea eather PATH PLA LANNING ∆ X O ∆ Ch Challenges in in Mer ergi ging, g, NLOS S Urban dr driving, Pla latooning
LEGEND O – good performance ∆ - room for improvement X - challenges
PROTECTED
3
ACCURACY/ RAN RANGE WEATHER COST URB RBAN/ NLOS SC SCEN ENARI RIO SO SOLVING CURRE RENT CHALLENGES MAPP APPING Da Data fr from lea lead vehicles s can an be be use used for
eal-time map ap up updates s and and correction of
errors. s. LOCALIZATION Hig ighly im improved loc localization by le leveragi ging GPS, S, LiD LiDar of
the ne neighboring con
s. PER ERCEP EPTION Non
sight vie iew, with ith robustness s for
any wea eather sce cenario. . Ra Rang nges up up to
PATH PLA LANNING Ultr ltra-low la latency Com
ith vehicles s whil hile mer erging, pla platooning, g, in intersection cr crossing
O - good performance
Connected Intelligence will help provide long range sensing, NLOS operations in a cost-effective manner with robustness for any weather condition
PROTECTED
Ad hoc networking technology that allows vehicles to communicate with each other, roadside devices, pedestrians, bicycles, trains, …
CURRENT STATUS:
elsewhere
Item Time 3D Position Position Accuracy Speed Heading Steering Wheel Angle Acceleration Brake Status Vehicle Size Event Flags Path History Path Prediction Other optional fields
4
Basic Safety Message
Basic Safety Message being transmitted
PROTECTED
Cooperative Automated Driving with highly improved mapping & localization, perception, and path planning
5
Local Sensors Local Sensors + V2X data
Cooperative
Perception Cooperative Mapping Cooperative Localization Cooperative Path Planner Controller Perception Mapping Localization Path Planner Controller
PROTECTED
Building real-time Automated Driving maps using the long-range of V2X
6
Significant improvement in road estimation as compared to using only local sensor data (Radar + Camera)
Measuring error in estimating road geometry using Camera, V2X and GPS
Data collected in Ann Arbor
Lane-level accuracy in road estimation at 200m
PROTECTED
When the vehicle is not changing lanes, its trajectory is a good indicator of lane geometry. In this scenario, V2V data from the remote vehicle can be used to augment accuracy of lane geometry estimation.
7 High detection accuracy achieved using Machine Learning Classifiers
PROTECTED
Improving positioning by leveraging GPS receivers of neighboring vehicles
8
Flow of data between host vehicle and neighbors Sensor Fusion block Measuring error in Vehicle Positioning
43.8% Error Reduction
PROTECTED
9
Enhancing sensing range by sharing data (incl. camera, LIDAR, radar, etc.) with neighboring vehicles
1 2 3
Pedestrian will be on the same section
seconds
Car will not see the the pedestrian for few more seconds Car and Bus communicate to let the car know about the pedestrian
PROTECTED
10
PROBLEM: Neural networks are unable to recognize all objects in certain conditions APPROACH: Combine multiple perspectives from neighboring cars of the same scenario
Car / biker not detected by Neural Network
Designed novel 3D Vision Fusion Algorithm to
allow bounding boxes to be updated with greater precision and
PROTECTED
11
neural network detects skateboarder
neural network does not detect skateboarder
Skateboarder detected Skateboarder not detected Other pedestrians detected
PROTECTED
12
Other pedestrians Skateboarder Skateboarder Other pedestrians CONFIDENCE LEVEL IN DETECTION
High confidence in FUSED VIEW No detection by Prius
Detected by Corolla
PROTECTED
13
Highway merging using V2X handshakes
25% Improvement in wait-time
Exchange V2V messages before merge to determine the merge order between on-ramp and highway vehicles Reducing the wait time on-ramp before merge
PROTECTED
14
Achieve near 100% accuracy your vehicle’s end-to-end driving by utilizing data from neighboring connected vehicles
V2X Convolutional Neural Network Steering angle Camera data, V2X data
What is End-to-end Driving? Deep Learning to predict steering angles, required force
PROTECTED
15
Ego
10 layer Video Neural Network
Input to neural network: Video frames Steering Angle output of video neural network
Baseline scenario: Ego vehicle predicts steering angle using forward facing camera data 1
vid
PROTECTED
Cooperative End-to-end Driving: V2V-based neural network - An alternative?
This is what the leading vehicle broadcasts: Latitude: 𝑀𝑏𝑢1 Longitude: 𝑀𝑝𝑜1 Heading: 𝜚1 This is what the Ego vehicle measures independently: Latitude: 𝑀𝑏𝑢0 Longitude: 𝑀𝑝𝑜0 Heading: 𝜚0
V2V Neural Network predicts the steering angle for Ego vehicle
2 layer V2V Neural Network
𝑀𝑏𝑢1 − 𝑀𝑏𝑢0 𝑀𝑝𝑜1 − 𝑀𝑝𝑜0 sin 𝜚1 − sin (𝜚0)
Latitude difference Longitude difference Sine of heading difference
Ego
2
Steering Angle
network
V2V
PROTECTED
Cooperative End-to-end Driving: Combining video-based and V2V-based networks Video-based prediction V2V-based prediction Weighing network
vid V2V F
vid
V2V
F
Weights
Final prediction
PROTECTED
using a neural network trained on integrated video and V2V data
Measured vs predicted steering angle
Trip Time [s] Steering angle [rad] 18
PROTECTED
19
Application Data Rate Details Cooperative Perception Processed – detected
0.5 Mbps 60 byte/object, 100
Cooperative Perception H.265/HEVC HD Camera
10 Mbps 640x360 resolution, compressed Cooperative Perception LIDAR - Raw 35 Mbps 6 verticle angels, 64 elements, 10 Hz horizontal rotation Cooperative Path Planning Coarse driving intention 0.05 Mbps Few 100 bytes (e.g. 500 bytes) / msg, 10 Hz Cooperative Path Planning Planned Trajectory 2.5 Mbps 32 byte / coordination, 10 msec resolution, 10 sec trajectory, 10 Hz
Total data rate per link: 50 Mbps.
Requirement #1:
Total system data rate target 10 Gbps
(Link data rate @50 Mbps x Target to support 200 links) Requirement #2:
Latency target1 = 20 msec
Ultra-low latency to support applications such as platooning
Latency target2 = 100 msec
For applications such as long range sensing, driving intention latency requirements would be relaxed (100 msec target)
For multiple links:
PROTECTED
Huge spectrum available at mmWave bands Advancement in CMOS technology enables low cost mmWave devices**
60 GHz WiFi (WirelessHD, IEEE 802.11ad) is already on the market
3 GHz 30 GHz
DSRC (75 MHz) Automotive radar 22-29 GHz 38-49 GHz 60 GHz WiFi 57-64 GHz Automotive radar 71-86 GHz
30 GHz 300 GHz
* United States radio spectrum frequency allocation chart as of 2011 ** T. S. Rappaport, J. N. Murdock, and F. Gutierrez, “State of the art in 60-GHz integrated circuits and systems for wireless communications,” Proceedings of the IEEE, vol. 99, no. 8, pp. 1390–1436, Aug 2011
Log-scale
20
PROTECTED
21
Use Case Category Throughput Latency Network type Cooperative Perception Traffic safety, efficiency High Low V2V Cooperative maneuvering Traffic safety, efficiency Moderate Ultra-low V2V Map download Traffic safety, efficiency High High V2I Live Perceptual data broadcast Traffic safety, efficiency High Low V2I Sensor data gathering Traffic safety, efficiency Moderate Moderate V2I Sending data for cloud processing Can be any Varied Varied V2I Media download Infotainment Varied High V2I Video steaming Infotainment High Low V2I
PROTECTED
ahead
networks
PROTECTED
gbansal@us.toyota-itc.com
23