Learning Steering for Parallel Autonomy: Handling Ambiguity in End-to-End Driving Alexander Amini Learning Steering for Parallel Autonomy Alexander Amini
Motivation Autonomous systems need the ability to handle a wide range of scenarios Night-time Driving No Lane Markings Rainy Weather Leveraging large datasets, we learn an underlying representation of driving based on human actually did Learning Steering for Parallel Autonomy Alexander Amini
Autonomous Driving Pipeline Separate problem into smaller sub-modules, tackle each independently Sensor Fusion Detection Localization Planning Actuation • What’s happening • Where are • Where am I relative • Where do I go? • What control signals around me? obstacles? to the obstacles? to take? [4-6] [1-3] [7, 8] [9, 10] [11, 12] Learning Steering for Parallel Autonomy Alexander Amini
End-to-End Learning Learn the control directly from raw sensor data Deep Neural Network Sensor Fusion Learned Model Actuation • What’s happening Underlying representation of how humans drive • What control signals around me? to take? [13-16] [1-3] [11, 12] Learning Steering for Parallel Autonomy Alexander Amini
End-to-End Learning pixel values Learn the control directly from raw sensor data Deep Neural Network Raw images: front Learned Model Actuation facing camera Underlying representation of how humans drive • What control signals to take? ! [13-16] [11, 12] Learning Steering for Parallel Autonomy Alexander Amini
End-to-End Learning pixel values steering Learn the control directly from raw sensor data Deep Neural Network Raw images: front Learned Model Actuation facing camera Underlying representation of how humans drive • What control signals to take? ! [13-16] [11, 12] Learning Steering for Parallel Autonomy Alexander Amini
Challenges Uncertainty Learning Steering for Parallel Autonomy Alexander Amini
Challenges Uncertainty Vision Learning Steering for Parallel Autonomy Alexander Amini
Challenges Uncertainty Vision Edge Cases Learning Steering for Parallel Autonomy Alexander Amini
Talk Outline Parallel Autonomy 1 Learning Steering for Parallel Autonomy Alexander Amini
Talk Outline Parallel Autonomy 1 Learning Bounds 2 Learning Steering for Parallel Autonomy Alexander Amini
Talk Outline Parallel Autonomy 1 Learning Bounds 2 3 Uncertainty Learning Steering for Parallel Autonomy Alexander Amini
Parallel Autonomy Shared robot-human control
Guardian Angel [17] Hyundai: Dad’s Sixth Sense. 2014. Learning Steering for Parallel Autonomy Alexander Amini
Parallel Autonomy: Architecture Human Input Hardware Low-Level- Drive-by-wire Shared Series Autonomy Tracking Interface Controller Control Learning Steering for Parallel Autonomy Alexander Amini
Parallel Autonomy: Hardware Learning Steering for Parallel Autonomy Alexander Amini
Parallel Autonomy: Hardware 5x LIDAR Laser Scanners [21-23] 3x GMSL Cameras [24] 1x GPS [25] 1x Inertial Measurement Unit [26] 2x Wheel Encoders Learning Steering for Parallel Autonomy [20] Alexander Amini
Parallel Autonomy: Hardware 5x LIDAR Laser Scanners [21-23] NVIDIA Drive PX2 [27] 3x GMSL Cameras [24] 1x GPS [25] 1x Inertial Measurement Unit [26] GPU enabled 2x Wheel Encoders computing platform Learning Steering for Parallel Autonomy Alexander Amini
Shared ≠ Binary Control Learning Steering for Parallel Autonomy Alexander Amini
Possible Approaches Direct actuation with motors CAN messages • Interference and contradictory • Responsiveness information form other ECUs • Reliability • Built in software safe guards • Difficulty designing for manual • Requires reprogramed ECUs override from Toyota (TRI) Learning Steering for Parallel Autonomy Alexander Amini
Possible Approaches Direct actuation with motors CAN messages • Interference and contradictory • Responsiveness information form other ECUs • Reliability • Built in software safe guards • Difficulty designing for manual • Requires reprogramed ECUs override from Toyota (TRI) Spoof input systems • Requires physical access to cables transmitting sensor data • Requires reverse engineering systems Learning Steering for Parallel Autonomy [20] Alexander Amini
Autonomous Modes Manual Parallel Autonomy Computer Learning Steering for Parallel Autonomy Alexander Amini
Learning Steering Bounds
Related Work: End-to-End Learning [13] [15] [16] Predict single control Compute policy from long Imitation Learning from the command given image frame short-term memory (LSTM) experts • No temporal information • Crowdsourced dataset • Simulated driving courses • Real world implementation • No simulation or real • Suffer from cascading world evaluation errors, oscillating actions Learning Steering for Parallel Autonomy Alexander Amini
Related Work: End-to-End Learning Differentiating Problem: Unable to integrate ambiguous decisions into higher level navigational control Learning Steering for Parallel Autonomy Alexander Amini
Learning a Steering Distribution Discretize action space of all steering commands to handle ambiguity Transform into continuous probability distributions and extract bounds [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 Learning Steering for Parallel Autonomy Alexander Amini
Discrete Action Learning Optimization through backpropogation ' ) 1 5 min 2 3 4 " log ( ! " ; ) "%& Single image Neural network Output distribution 4 " true distribution at frame 9 ! " ( , ((! " ; )) ((! " ; :) est. distribution at frame 9 from dataset with parameters ' # = ! " "%& ) [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 Learning Steering for Parallel Autonomy Alexander Amini
Multimodal Distributions We want to learn multimodal distributions but only have access to a single control that the human made Predicted Distribution 1 " = 0 [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 ! 0 Learning Steering for Parallel Autonomy Alexander Amini
Multimodal Distributions We want to learn multimodal distributions but only have access to a single control that the human made Human Distribution 1 ! 0 Predicted Distribution 1 " = 1 [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 ! 0 Learning Steering for Parallel Autonomy Alexander Amini
Multimodal Distributions We want to learn multimodal distributions but only have access to a single control that the human made Human Distribution 1 ! 0 Predicted Distribution 1 " = 2 [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 ! 0 Learning Steering for Parallel Autonomy Alexander Amini
Multimodal Distributions We want to learn multimodal distributions but only have access to a single control that the human made Human Distribution Human Distribution 1 1 ! 0 0 ! Predicted Distribution 1 " = 3 [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 ! 0 Learning Steering for Parallel Autonomy Alexander Amini
Multimodal Distributions We want to learn multimodal distributions but only have access to a single control that the human made Human Distribution Human Distribution 1 1 ! 0 0 ! Predicted Distribution 1 " = 4 [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 ! 0 Learning Steering for Parallel Autonomy Alexander Amini
Multimodal Distributions We want to learn multimodal distributions but only have access to a single control that the human made Human Distribution Human Distribution 1 1 ! 0 0 ! Predicted Distribution 1 " = 5 [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 ! 0 Learning Steering for Parallel Autonomy Alexander Amini
Multimodal Distributions We want to learn multimodal distributions but only have access to a single control that the human made Human Distribution Human Distribution 1 1 ! 0 0 ! Predicted Distribution 1 " → ∞ [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 ! 0 Learning Steering for Parallel Autonomy Alexander Amini
Advantages of this approach • Don’t need to see the same exact intersection multiple times in order to learn • We learn an underlying representation of drivable space under ambiguity • Not constrained to a pre-defined intersection models (T -intersection, 4-way, etc) [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 Learning Steering for Parallel Autonomy Alexander Amini
Dataset Collection • 7 hours of driving data • Greater Boston metropolitan area • Different seasons, times, weather • Highway & city roads Fine tuned on 1 minute of data of roads • without lane markers • Trained for 10 epochs • Data parallelism with multi-GPUs • ~1 hour on NVIDIA DGX-1 [28] [14] Amini et al. “Learning Steering Bounds for Parallel Autonomous Systems”. 2018 Learning Steering for Parallel Autonomy Alexander Amini
Recommend
More recommend
