Real-Time Object Detection and Semantic Segmentation Geetank - PowerPoint PPT Presentation

Real-Time Object Detection and Semantic Segmentation Geetank Raipuria Wei Li Corporate Development manager Computer Vision Engineer NavInfo Europe Presented at GTC 2019 Session S9351

NavInfo - our growth benefits from AI Smart Mobility Provider 2015- AI developed by us provides: Future Autonomous • Perception for our autonomous solutions Driving • Can also benefit & apply to other industries • Support related companies/partners, like Chips NVIDIA AI Connected Navigation Vehicle Service (CVS) 2002- 2015 Map & Navigation Provider • Surveying and Mapping Knowledge • Image processing

NavInfo’s business growth path 1st Obtained More HAD map solutions Government License Working with more OEM Tencent New Expand the HD Map 1st ADAS Map of 1st On Mobile 1st Pedestrian autonomous solutions Shareholder development 1st R&D Navigation China to BMW Devices & Internet Navigation Map Map 1st Commercial Navigation Map 2002 2004 2006 2007 2009 2010 2011 2012 2014 2015 2016 2018 2019 2021 Provide NVIDIA HAD map for its localization service BMW L3 car 1st Navi Data Supplier 1st Commercial NDS Advanced ADAS Map 1st Commercial Provide Autonomous JV’s with NAVTEQ & applies NavInfo HD of China to BMW, Map Product to on SZ Stock Exchange Dynamic Traffic driving solution to TOYOTA Tsusho Market Daimler & VW Map in China Information several Chinese OEMs Provide Autopilot solution based on our own localization product

NavInfo’s Footprint Netherlands America China 1) NavInfo EU • Advanced AI • International • 21 Localization Research Lab Business • Bases for Data EU Business Expansion Expansion Collection and Singapore Technology 2) AIIM • Autonomous Services • Southeast Asia • 6 R&D Centers driving & Robotics Business solution provider ( Shanghai, Xi’an, Expansion 3) Mapscape Shenyang, Wuhan, Hefei, Shenzhen ) • Beijing Headquarters

Cooperation between NavInfo & NVIDIA NVIDIA’s new system - DRIVE Localization NVIDIA DRIVE Software use NavInfo’s HD Map to give customer a way of localizing their autonomous driving car. NavI nfo’s Perception Technology Artificial Intelligence NavInfo’s Training Model NavI nfo’s research lab developed a vision based system that can detect and classify objects in real time on NVIDIA Xavier NavInfo trains and optimizes its models on NVIDIA DGX-1 servers We support and benefit each other’s achievements, driven by AI, to generate better products and services to our customers and end users

NavInfo Service Offerings in Europe Autonomous driving and robotic solutions AI based solutions for different industries AI based algorithms

Results from TU/e sponsored by NavInfo Europe No. 4 in the world in the Semantic Segmentation Results with the highest speed and lowest error in Stereo Odometry (not Lidar, not SLAM) Reference: [1] Panagiotis Meletis and Gijs Dubbelman, "Training of Convolutional Networks on Multiple Heterogeneous Datasets for Street Scene Semantic Segmentation", In IEEE IV 2018. June 2018, 0 – 8.

Deep Learning for HD Mapping

HD Mapping Includes highly accurate lane and road features.

Deep Learning for HD Mapping Feature Extraction Deep learning provides automating feature extraction from video feed of • collection vehicles ( Semi ) Real Time Cameras, Automated Feature LIDARs Map Extraction Updating Localization

Real Time Object Detection Geetank Raipuria Andrei Pata Computer Vision Engineer Software Engineer Advanced Research Lab Advanced Research Lab NavInfo Europe NavInfo Europe

Real-Time Object Detection System Object Detector based on Deep Convolutional Neural Network architecture, to localize and classify road signs and traffic lights from a real-time camera feed Two Stage System: Best of both worlds High Accuracy Low Inference time

Real-Time Object Detection System Real Time Performance Features Supported 2-3x Speedup using Tensor RT 350+ supported classes including Traffic Signs About : 35 fps on INT8 on NVIDIA Xavier SoC Regulatory Signs • : 110 fps on Titan XP Warning Signs Inference at Full HD resolution (1920x1080) • Guide Signs • Information Signs • Able to extract and classify object as small as 25x25 pixels Road Work Signs • Robust to extreme lighting conditions Signboards Traffic Lights Digital Traffic Signs

Sample Detections

Sample Detections

Sample Detections

Online video available at https://youtu.be/-QtYF0XUZh0 Demo Realtime Object Detection Advanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China. We comply with all Chinese regulations regarding the processing of China data.

Real Time Semantic Segmentation Ahmed Badar Matti Jukola Computer Vision Engineer Software Engineer Advanced Research Lab Advanced Research Lab NavInfo Europe NavInfo Europe

Real-Time Semantic Segmentation Deep learning architecture to segment and extract road markings at pixel level Segmentation Network Camera input Segmented road markings Fully Convolutional Neural Network

Real-Time Semantic Segmentation Our model is based on a multi-branch Camera input Front-view predictions convolutional neural network architecture Top-view Front-view transformation transformation Top-down view Top-down predictions Multi-branch segmentation network

Real-Time Semantic Segmentation Currently supports 40 Road Marking Classes, including : Lane lines • Arrows • Text • Real Time Performance 3x Speedup using Tensor RT About : 90 fps on NVIDIA Xavier SoC : 300 fps on Titan XP Inference at 1024 × 384 image sizes This includes image transformations (top and front view)

Sample Segmentations Input Image Ground Truth Prediction

Sample Segmentations Input Image Ground Truth Prediction

Online video available at https://youtu.be/E4hU-COkHDo Demo Realtime Semantic Segmentation Advanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China. We comply with all Chinese regulations regarding the processing of China data.

Real Time Scene Understanding Elahe Arani Mahmoud Gamal Senior AI Researcher Computer Vision Engineer Advanced Research Lab Advanced Research Lab NavInfo Europe NavInfo Europe

Real-Time Scene Understanding A real time unified object detection and semantic segmentation for autonomous driving cars/HD mapping. DNN

Joint Object Detection and Segmentation Currently supports 40 Road Marking Classes and 350+ Road Sign classes including: Performance Inference at 512x512 image sizes Traffic Signs – About 45 FPS on Titan XP Gantry Signboards – Traffic Lights – Other Features Supported Digital Traffic Signs – Guard Rails – Lane Markings – Curbs – Text – Speed Limits on Road – Arrows –

Sample Detections

Sample Segmentation Input Image Ground Truth Prediction

Joint Detections and Segmentation Prediction (Dec) Ground Truth Prediction (Seg)

Occlusion Handling Input Image Prediction (Dec) Ground Truth Prediction (Seg)

Occlusion Handling Input Image Prediction (Dec) Ground Truth Prediction (Seg)

Online video available at https://youtu.be/NJVNFfueKb4 Demo Realtime Scene Understanding Advanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China. We comply with all Chinese regulations regarding the processing of China data.

Real-Time Object Detection and Semantic Segmentation Geetank Raipuria Wei Li Corporate development manager Computer Vision Engineer NavInfo Europe Presented at GTC 2019 Session S9351