NVIDIA DRIVE December 2019
DRIVE PLATFORM Mass production INDUSTRY / MARKET SAFETY TECHNOLOGY REGULATIONS NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 2
THE DRIVE INITIATIVE End-to-End, Open Platform for Building Autonomous Vehicles DGX Saturn V Constellation Xavier DRIVE AV DRIVE IX Re-SIM The NVIDIA DRIVE™ Platform provides all major AI technologies for Driving, Development and Validation - from driving & cockpit computers & software to data collection & DNN training, to simulation tools. The system is an open platform with multiple programming points. Car makers are free to use or develop as much as they like and take advantage of an ecosystem of hundreds of partners. NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 3
NVIDIA DRIVE END-TO-END PLATFORM COLLECT DATA TRAIN MODELS SIMULATE DRIVE Cars Pedestrians Path Cars Pedestrians Path Lanes Signs Lights Lanes Signs Lights NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 4
DRIVE COMPUTER / ARCHITECTURE SCALABLE NOW & INTO THE FUTURE NCAP TO ROBOTAXI 5
DRIVE ROADMAP ORIN DRIVE Parker DRIVE PX2 DRIVE Xavier DRIVE Pegasus DRIVE Orin-X 1x Perf 20x Perf 23x Perf ~230x Perf >100x Perf 1x Efficiency 1x Efficiency 8x Efficiency 5x Efficiency ~20x Efficiency Samples – Q4’2015 Samples – Q2’2016 Samples – Q3’2018 Samples – Mid 2021 Samples – Q1’2018 Production – Q1’2017 Production – Q3’2016 Production – Q1’2019 Production – Q3’2019 Production - Q3’2022 Auto-Grade Super Energy-Efficient 6
ORIN – HIGHEST PERF SOC EVER What is driving the need for increasing performance in AV? Sensor Complexity - 20+ DNN Models New DNN Models- DNN GOPs/Frame More higher required for most CNN, RCNN, RNN increasing resolution demanding ODD cameras Integration of Future Proof OTA new features DNN Precision / functions EUNCAP / SW Patch Accuracy Centralized With 360° Computer Perception More perf equates to faster time to market 7
NVIDIA DRIVE One Architecture PEGASUS (L4/RoboTaxis) ORIN XAVIER (NCAP , L2+/L3) ORIN One AV Architecture for L2 -> Robo Taxi NVIDIA DRIVE AGX platform is an open platform; allows partners to do their own innovation NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 8
DRIVE ARCHITECTURE AUTOMOTIVE GRADE – INDUSTRIALIZED ECU’S NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 9
NVIDIA DRIVE SENSOR ECOSYSTEM 10
SENSOR ECOSYSTEM IMAGING SENSOR / CAMERA LIDAR AR0820, AR0144, AR0231, IMX390, IMX490, IMX424 RADAR GNSS/IMU ARS430 SC13S, MTi-G-710, SPAN-IGM A1 NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 11 https://developer.nvidia.com/drive/drive-core-sensors
NVIDIA DRIVE DRIVE SOFTWARE 12
NVIDIA DRIVE SOFTWARE ARCHITECTURE Open and modular software for AV development DRIVE AV DRIVE IX DRIVE Planning DRIVE Mapping Visualization AI CoPilot AI Assistant DRIVE Perception DRIVEWORKS DRIVE Networks DRIVE Calibration DRIVE Core DRIVE OS DRIVE AGX DEVELOPER KITS DRIVE HYPERION (Xavier/Pegasus) (Reference Architecture) NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 13
NVIDIA DRIVE DRIVE OS 14
DRIVE AGX AV Software Platform OEM/T1 Application Communication Services Data Distribution (DDS) 1 Foundation Services L1 Safety Services Security Services Power Management DriveWorks L3 Safety Services L2 Safety Services Diagnostics DRIVE OS OS Drivers, USB, cuDNN TensorRT NvMedia Open GL NvStreams File System, Network EGL 1.5 ES 3.2 CUDA Linux Linux BSP Hypervisor Software Hardware Xavier MCU CAN & GMSL RS 10G/1G/BR Ethernet PMIC NVLINK PCIe NOR eMMC UFS NVME I/O Flexray 232 (via CSI-2) Scalability; Radar Lidar GPS IMU Cameras TU104 +Xavier, NvME, or Optional Processor Hardware NVIDIA Software OS/3rd Party Software Notes: 1) For development only. NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 15 Optional Services
DRIVE AGX AV Software Platform Communication Services 1 Data Distribution (DDS) 1 Foundation Services DRIVE AV L1 Safety Services Security Services 1 Power Management L3 Safety Services L2 Safety Services Diagnostics DRIVE OS OS Drivers, USB 1 , cuDNN 1 TensorRT NvMedia NvStreams Open GL 1 File System, Network EGL 1.5 1 ES 3.2 CUDA QNX BSP Hypervisor Software Hardware Xavier MCU CAN & GMSL RS 10G/1G/BR Ethernet PMIC NVLINK PCIe NOR eMMC UFS NVME I/O Flexray 232 (via CSI-2) Scalability Scalability; Radar Lidar GPS IMU Cameras w/dGPU +Xavier, NvME, or Optional Processor Hardware NVIDIA Software OS/3rd Party Software NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. Notes: 1) Not for use in production in a safety context. 16 Optional Services
NVIDIA DRIVE AUTONOMOUS DRIVING APPLICATION 17
NVIDIA DRIVEWORKS SDK Data Abstraction | Calibration | Compute Modules DRIVE AV DRIVE IX DRIVEWORKS DRIVE Networks DRIVE Calibration Calibration DriveNet ClearSightNet OpenRoadNet Tools Self- PathNet MapNet LaneNet Calibration WaitNet SignNet LightNet Egomotion DRIVE Core Sensor Abstraction Image Processing DNN Frameworks Layer + Plugins Vehicle IO Point Cloud Processing Tools (Recorder) DRIVE OS DRIVE AGX DEVELOPER KITS DRIVE HYPERION NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 18
NVIDIA DRIVEWORKS SDK Foundation for AV SW Development on NVIDIA DRIVE SOFTWARE MODULES TOOLS - Open, modularized library of functions - Software dev tools ranging from sensor - Optimized for DRIVE AGX data capture to calibration and - Easy to integrate and build upon visualization DEVELOPER SUPPORT PRODUCTION GRADE - Sample apps - Architected for automotive safety - Documentation, tutorials - Compliant with industry standards, ( ISO - Active developer forum and community 26262/21448, MISRA) NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 19
DRIVE AV Actuation Behavior Planner Control Cloud Mapping Search Based Planning AI Based Prediction Learned Planning Lane Route Planner Safety Force Field Safety Checks LSA Image Planner HD Map World Model Localization Local World Model Map Perception Obstacle Perception Path Perception Wait Perception RoadRunner Map Creation Frame Packaging and and updates Serialization DriveNet PilotNet DriveNet OpenRoadNet LaneNet SubModule WaitNet Motion/Structure PathNet MapNet DNNs Drive AV Egomotion Estimation Low Level Processing DriveWorks Self-Calibration MapWorks Sensor Abstraction NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 20
SCALABLE PLATFORM One architecture – Highway and Urban AV Driving ON RAMP TO OFF RAMP URBAN: ADDRESS TO ADDRESS NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 21
ADVANCED FUNCTIONS Clear Sight Auto High Beam Parking NCAP NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 22
DRIVE AV Actuation Behavior Planner Control Cloud Mapping Search Based Planning AI Based Prediction Learned Planning Lane Route Planner Safety Force Field Safety Checks LSA Image Planner HD Map World Model Localization Local World Model Map Perception Obstacle Perception Path Perception Wait Perception RoadRunner Map Creation Frame Packaging and and updates Serialization DriveNet PilotNet DriveNet OpenRoadNet LaneNet SubModule WaitNet Motion/Structure PathNet MapNet DNNs Drive AV Egomotion Estimation Low Level Processing DriveWorks Self-Calibration MapWorks Sensor Abstraction NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 23
SENSOR SELF-CALIBRATION NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 24
EGOMOTION IMU/Odometry Ego-motion Radar Ego-motion Camera Ego-motion (SLAM) LIDAR Ego-motion NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 25
DRIVE AV Actuation Behavior Planner Control Cloud Mapping Search Based Planning AI Based Prediction Learned Planning Lane Route Planner Safety Force Field Safety Checks LSA Image Planner HD Map World Model Localization Local World Model Map Perception Obstacle Perception Path Perception Wait Perception RoadRunner Map Creation Frame Packaging and and updates Serialization DriveNet PilotNet DriveNet OpenRoadNet LaneNet SubModule WaitNet Motion/Structure PathNet MapNet DNNs Drive AV Egomotion Estimation Low Level Processing DriveWorks Self-Calibration MapWorks Sensor Abstraction NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 26
CAMERA OBSTACLE PERCEPTION NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 27
OBJECT DETECTION
SURROUND OBJECT TRACKING
DNN DISTANCE NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 31
FUTURE MOTION PREDICTION (RNN)
VELOCITY ESTIMATION NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 33
VELOCITY ESTIMATION NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 34
LIDAR – OBSTACLE PERCEPTION NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 35
LIDAR – OBSTACLE PERCEPTION NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 36
LIDAR FREESPACE NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 37
LIDAR MOTION DETECTION NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 38
RADAR TRACKING NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 39
PATH REPRESENTATION (THE LANE GRAPH) NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 40
LANENET + TRACKING + FUSION Tracking and Fusion Lane Assignment (OIPA) NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 41
PATHNET NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 42
FACTORED PILOTNET TURNS NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 43
Recommend
More recommend
