SLIDE 1
Self-Driving Cars As Edge Computing Devices
Matt Ranney - @mranney Uber ATG
Why Self-Driving?
Self-driving matters for The world Self-driving matters for Uber Uber matters to self-driving
Self-Driving Cars As Edge Computing Devices Matt Ranney - @mranney - - PowerPoint PPT Presentation
Self-Driving Cars As Edge Computing Devices Matt Ranney - @mranney - - PowerPoint PPT Presentation
Self-Driving Cars As Edge Computing Devices Matt Ranney - @mranney Uber ATG Why Self-Driving? Self-driving Self-driving Uber matters for matters for matters to The world Uber self-driving Vehicles at Scale Self-driving Systems Fleet
SLIDE 2
SLIDE 3
Self-driving Systems Vehicles at Scale The Network Fleet Operations
SLIDE 4
1400+ Employees
Research + Development TOR Core Development PIT Core Development SF Maps CO Automotive DTW
Uber ATG
Labeling SEA
SLIDE 5
Side and rear facing cameras work in collaboration to construct a continuous view of the vehicle’s surroundings Modified base Vehicle Platform with Uber-specific mounting provisions, electrical harness, cooling interface, interior trim, and software control API Gateway Module serves as a gateway to the base Vehicle Platform from the Uber Self Driving System, translating messages and commanding the vehicle’s actuators (brakes, throttle, steering) Roof mounted antenna provide GPS positioning and wireless data capabilities Top mounted lidar provides a 360° 3-dimensional scan of the environment Forward facing camera array focusing both near and far field, watching for braking vehicles, crossing pedestrians, traffic lights, and signage 360° radar coverage detects vehicles and
- ther obstacles
Custom designed compute and storage allow for real-time processing of data while a fully integrated cooling solution keeps components running optimally Automatic Emergency Braking system
- perates independently as a safeguard for
certain situations requiring activation of the vehicle braking system
SLIDE 6
Self-Driving Vehicle Basics
Camera LiDAR Radar Ultrasonic GPS IMU Wheel Encoder Perception Prediction Motion Planning Control Maps Localization Routing Steering Braking Propulsion Sensors Software Controls Compute
SLIDE 7
SLIDE 8
- Sensor Extrinsics
- Camera Intrinsics
- Lidar Intensity
- Occlusion Masks
What We Calibrate
SLIDE 9
Sensors Controls LTE Modem LTE Modem Telematics Module Switch Node Node Node Node Node
SLIDE 10
Onboard Data
Onboard OS Code Binaries Trained Models HD Maps
Read Only
Sensor Data Diagnostics Vehicle Telemetry
Writable
SLIDE 11
Offload
Analytics Vehicle Map Making Depot Network Datacenter Log Ingest Performance Evaluation
SLIDE 12
Offload
Analytics Vehicle Map Making Depot Network Datacenter Log Ingest Performance Evaluation
SLIDE 13
SLIDE 14
SLIDE 15
SLIDE 16
End to End Latency Perception Latency Prediction Latency Planning Latency
SLIDE 17
SLIDE 18
Offload
Analytics Vehicle Map Making Depot Network Datacenter Log Ingest Performance Evaluation
SLIDE 19
SLIDE 20
SLIDE 21
SLIDE 22
SLIDE 23
SLIDE 24
Offload
Analytics Vehicle Map Making Depot Network Datacenter Log Ingest Performance Evaluation
SLIDE 25
Self-Driving Vehicle Basics
Camera LiDAR Radar Ultrasonic GPS IMU Wheel Encoder Perception Prediction Motion Planning Control Maps Localization Routing Steering Braking Propulsion Sensors Software Controls Compute
SLIDE 26
SLIDE 27
- Unit tests
- Sanitizers: ASan, MSan, TSan, UBSan
- Integration tests
- Feedback cycles require end to end testing
Testing
SLIDE 28
Self-Driving Vehicle Basics
Camera LiDAR Radar Ultrasonic GPS IMU Wheel Encoder Perception Prediction Motion Planning Control Maps Localization Routing Steering Braking Propulsion Sensors Software Controls Compute
SLIDE 29
SLIDE 30
SLIDE 31
Scenario Based Testing
SLIDE 32
- Time intensive
- Space intensive
- Want to test each diff independently
Track Throughput
SLIDE 33
Write New Software Update ML Models Run Unit Tests Test on track
SLIDE 34
Simulation
Sensor Data Vehicle Hardware Software Under Test Log / Results
Hardware In the Loop (HIL)
Sensor Data Commodity Hardware Software Under Test Log / Results
Software In the Loop (SIL)
SLIDE 35
Vehicle Model
Vehicle Pose
Logged Sensors
Log Based Simulation
Perception Prediction Motion Planning Controls
SLIDE 36
Pose: position and orientation of an object Occlusion: one object blocks the perception of another Jerk: rate of change of acceleration, third derivate of position
Glossary
SLIDE 37
Vehicle Model
Vehicle Pose
Logged Sensors
Log Based Simulation
Perception Prediction Motion Planning Controls
SLIDE 38
- Log itself is static
- Log based simulator is (mostly) open loop
- Perception output is localized to the map
Log Divergence
SLIDE 39
SLIDE 40
- Log itself is static.
- Log based simulator is (mostly) open loop.
- Perception output is localized to the map.
- Logged actors do not react.
Log Divergence
SLIDE 41
SLIDE 42
- Log itself is static.
- Log based simulator is (mostly) open loop.
- Perception output is localized to the map.
- Logged actors do not react.
- For small divergence, this still works pretty well.
Log Divergence
SLIDE 43
Initial Conditions
Vehicle Model
Logged Pose
Logged Sensors Perception Prediction Motion Planning Controls pre-roll Interesting section
t=0 t=10 t=20
Vehicle Model
Vehicle Pose
Logged Sensors Perception Prediction Motion Planning Controls
SLIDE 44
Vehicle Model
Vehicle Pose
Sim Engine
Virtual Simulation
Partial Perception Prediction Motion Planning Controls
SLIDE 45
SLIDE 46
Variations
SLIDE 47
SLIDE 48
SLIDE 49
Pass-Fail Result Jerk Deceleration Base-Diff Comparison
SLIDE 50
SLIDE 51
SLIDE 52
SLIDE 53
2019-10-05
SLIDE 54
SLIDE 55
SLIDE 56
SLIDE 57
SLIDE 58
- Just reproducing a situation is not enough. We need a way to tell whether we are doing
the right thing.
- Measuring the correctness of driving is similarly challenging to building a simulator.
- We built a framework we call S-R for “Situation - Response”.
- The requirements for the types of driving behavior we want to measure are encoded
and evaluated.
- We can use parameter sweeps to validate and explore the requirements and our
software’s performance.
Measuring Driving Behavior
SLIDE 59
Situation A2 which expects Response Y Situation A1 which expects Response X
SLIDE 60
How do we know that the simulator is predictive of real world performance?
SLIDE 61
SLIDE 62
SLIDE 63
SLIDE 64
Distribution of results
Shows the area between the median log and our initial Simulation.
SLIDE 65
Distribution of results
Sim with occlusion
SLIDE 66
- This is an ideal workload for a public cloud
- Irregular demand
- Each test needs thousands virtual vehicles when running, and 0 when done.
- Some experiments utilize 100K tests
- Some require 1M
Simulation in the Cloud
SLIDE 67
SLIDE 68
Write New Software Update ML Models Run Unit Tests and Simulation Suite Test on track
SLIDE 69
- Most batch schedulers do not handle job sizes of 1M tasks very well
Cloud Challenges
SLIDE 70
Sim Manager
SLIDE 71
- Most batch schedulers do not handle job sizes of 1M tasks very well
- Large containers and Docker write amplification
Cloud Challenges
SLIDE 72
Large Containers
Onboard OS Code Binaries Trained Models HD Maps
Onboard
Code Binaries Models Maps
Simulation Container
OS Dependencies
SLIDE 73
tmpfs
Write Amplification
Container Registry Pull Layer Write Somewhere Extract Layer Write Somewhere
SLIDE 74
- Most batch schedulers do not handle job sizes of 1M tasks very well
- Large containers and Docker write amplification
- GPUs
Cloud Challenges
SLIDE 75
GPUs
Perception Prediction Planning Inference Inference Inference Inference GPU Code Cache Remote GPU Service
SLIDE 76
- Most batch schedulers do not handle job sizes of 1M tasks very well
- Large containers and Docker write amplification
- GPUs
- Costs
Cloud Challenges
SLIDE 77
Extract Logged Data Identify Interesting Events Write New Software Update ML Models Test with Simulation suite Test on track Deploy to Test Fleet Deploy to Full Fleet
SLIDE 78