Self-Driving Cars As Edge Computing Devices Matt Ranney - @mranney Uber ATG
Why Self-Driving? Self-driving Self-driving Uber matters for matters for matters to The world Uber self-driving
Vehicles at Scale Self-driving Systems Fleet Operations The Network
1400+ Uber ATG Maps Automotive CO DTW Employees Research + Labeling Development SEA TOR Core Core Development Development SF PIT
Side and rear facing cameras work in Top mounted lidar provides a 360° collaboration to construct a continuous 3-dimensional scan of the environment view of the vehicle’s surroundings Forward facing camera array focusing both near and far field, watching for braking vehicles, Roof mounted antenna crossing pedestrians, traffic lights, and signage provide GPS positioning and wireless data capabilities Automatic Emergency Braking system operates independently as a safeguard for certain situations requiring activation of the vehicle braking system 360° radar coverage detects vehicles and other obstacles Custom designed compute and storage Gateway Module serves as a gateway to allow for real-time processing of data while a the base Vehicle Platform from the Uber Self Modified base Vehicle Platform with fully integrated cooling solution keeps Driving System, translating messages and Uber-specific mounting provisions, components running optimally commanding the vehicle’s actuators (brakes, electrical harness, cooling interface, throttle, steering) interior trim, and software control API
Self-Driving Vehicle Basics Sensors Software Controls Camera Perception Steering LiDAR Prediction Braking Radar Motion Planning Propulsion Ultrasonic Control GPS Maps IMU Localization Wheel Encoder Routing Compute
What We Calibrate • Sensor Extrinsics • Camera Intrinsics • Lidar Intensity • Occlusion Masks
LTE Modem LTE Modem Telematics Module Switch Node Node Node Node Node Sensors Controls
Onboard Data Onboard OS Code Binaries Trained Models HD Maps Read Only Sensor Data Diagnostics Vehicle Telemetry Writable
O ffl oad Vehicle Depot Network Datacenter Log Ingest Performance Analytics Map Making Evaluation
O ffl oad Vehicle Depot Network Datacenter Log Ingest Performance Analytics Map Making Evaluation
End to End Latency Perception Latency Prediction Latency Planning Latency
O ffl oad Vehicle Depot Network Datacenter Log Ingest Performance Analytics Map Making Evaluation
O ffl oad Vehicle Depot Network Datacenter Log Ingest Performance Analytics Map Making Evaluation
Self-Driving Vehicle Basics Sensors Software Controls Camera Perception Steering LiDAR Prediction Braking Radar Motion Planning Propulsion Ultrasonic Control GPS Maps IMU Localization Wheel Encoder Routing Compute
Testing • Unit tests • Sanitizers: ASan, MSan, TSan, UBSan • Integration tests • Feedback cycles require end to end testing
Self-Driving Vehicle Basics Sensors Software Controls Camera Perception Steering LiDAR Prediction Braking Radar Motion Planning Propulsion Ultrasonic Control GPS Maps IMU Localization Wheel Encoder Routing Compute
Scenario Based Testing
Track Throughput • Time intensive • Space intensive • Want to test each diff independently
Write New Software Update ML Models Run Unit Tests Test on track
Simulation Sensor Data Vehicle Hardware Log / Results Software Under Test Hardware In the Loop (HIL) Commodity Sensor Data Log / Results Hardware Software Under Test Software In the Loop (SIL)
Log Based Simulation Logged Motion Vehicle Perception Prediction Controls Sensors Planning Model Vehicle Pose
Glossary Pose: position and orientation of an object Occlusion: one object blocks the perception of another Jerk: rate of change of acceleration, third derivate of position
Log Based Simulation Logged Motion Vehicle Perception Prediction Controls Sensors Planning Model Vehicle Pose
Log Divergence • Log itself is static • Log based simulator is (mostly) open loop • Perception output is localized to the map
Log Divergence • Log itself is static. • Log based simulator is (mostly) open loop. • Perception output is localized to the map. • Logged actors do not react.
Log Divergence • Log itself is static. • Log based simulator is (mostly) open loop. • Perception output is localized to the map. • Logged actors do not react. • For small divergence, this still works pretty well.
Initial Conditions pre-roll Interesting section t=0 t=10 t=20 Logged Motion Vehicle Perception Prediction Controls Sensors Planning Model Logged Pose Logged Motion Vehicle Perception Prediction Controls Sensors Planning Model Vehicle Pose
Virtual Simulation Partial Motion Vehicle Sim Engine Prediction Controls Perception Planning Model Vehicle Pose
Variations
Pass-Fail Result Jerk Deceleration Base-Diff Comparison
2019-10-05
Measuring Driving Behavior • Just reproducing a situation is not enough. We need a way to tell whether we are doing the right thing. • Measuring the correctness of driving is similarly challenging to building a simulator. • We built a framework we call S-R for “Situation - Response”. • The requirements for the types of driving behavior we want to measure are encoded and evaluated. • We can use parameter sweeps to validate and explore the requirements and our software’s performance.
Situation A 1 which expects Response X Situation A 2 which expects Response Y
How do we know that the simulator is predictive of real world performance?
Distribution of results Shows the area between the median log and our initial Simulation.
Sim with occlusion Distribution of results
Simulation in the Cloud • This is an ideal workload for a public cloud • Irregular demand • Each test needs thousands virtual vehicles when running, and 0 when done. • Some experiments utilize 100K tests • Some require 1M
Write New Software Update ML Models Run Unit Tests Test on track and Simulation Suite
Cloud Challenges ● Most batch schedulers do not handle job sizes of 1M tasks very well
Sim Manager
Cloud Challenges ● Most batch schedulers do not handle job sizes of 1M tasks very well ● Large containers and Docker write amplification
Large Containers Onboard OS Code Binaries Trained Models HD Maps Onboard Code Binaries Models Maps OS Dependencies Simulation Container
Write Amplification Container Registry Pull Layer Write Somewhere Extract Layer Write Somewhere tmpfs
Cloud Challenges ● Most batch schedulers do not handle job sizes of 1M tasks very well ● Large containers and Docker write amplification ● GPUs
GPUs Perception Prediction Planning Inference Inference Inference Inference GPU Code Cache Remote GPU Service
Cloud Challenges ● Most batch schedulers do not handle job sizes of 1M tasks very well ● Large containers and Docker write amplification ● GPUs ● Costs
Extract Logged Data Deploy to Full Fleet Identify Interesting Events Deploy to Test Fleet Write New Software Update ML Models Test with Simulation suite Test on track
Thanks
Recommend
More recommend
