Real-Time Object Detection and Semantic Segmentation Geetank - - PowerPoint PPT Presentation

▶
real time object detection and semantic segmentation
SMART_READER_LITE
LIVE PREVIEW

Real-Time Object Detection and Semantic Segmentation Geetank - - PowerPoint PPT Presentation

Aug 23, 2023 374 likes •739 views

Real-Time Object Detection and Semantic Segmentation Geetank Raipuria Wei Li Corporate Development manager Computer Vision Engineer NavInfo Europe Presented at GTC 2019 Session S9351 NavInfo - our growth benefits from AI Smart Mobility

slide-1
SLIDE 1

Real-Time Object Detection and Semantic Segmentation

NavInfo Europe Wei Li

Corporate Development manager Presented at GTC 2019 Session S9351

Geetank Raipuria

Computer Vision Engineer

slide-2
SLIDE 2

NavInfo - our growth benefits from AI

2015- Future 2002- 2015

  • Surveying and Mapping Knowledge
  • Image processing

Smart Mobility Provider

Chips

Autonomous Driving Connected Vehicle Service (CVS) Navigation

Map & Navigation Provider

AI developed by us provides:

  • Perception for our autonomous solutions
  • Can also benefit & apply to other industries
  • Support related companies/partners, like

NVIDIA

AI

slide-3
SLIDE 3

NavInfo’s business growth path

2002 2004 2006 2007 2009 2010 2011 2012 2015 2016 2018 2014

1st Obtained Government License 1st R&D Navigation Map 1st Commercial Navigation Map 1st On Mobile Devices & Internet 1st Commercial Dynamic Traffic Information 1st Pedestrian Navigation Map 1st Navi Data Supplier

  • n SZ Stock Exchange

1st ADAS Map of China to BMW BMW L3 car applies NavInfo HD Map in China Provide NVIDIA HAD map for its localization service Provide Autonomous driving solution to several Chinese OEMs Provide Autopilot solution based on our

  • wn localization

product 1st Commercial NDS Map Product to Market Tencent New Shareholder Advanced ADAS Map

  • f China to BMW,

Daimler & VW Expand the HD Map development

2021 2019

More HAD map solutions Working with more OEM autonomous solutions JV’s with NAVTEQ & TOYOTA Tsusho

slide-4
SLIDE 4

China

  • 21 Localization

Bases for Data Collection and Technology Services

  • 6 R&D Centers

( Shanghai, Xi’an, Shenyang, Wuhan, Hefei, Shenzhen )

  • Beijing

Headquarters

Netherlands

1) NavInfo EU

  • Advanced AI

Research Lab

  • EU Business

Expansion 2) AIIM

  • Autonomous

driving & Robotics solution provider 3) Mapscape

America

  • International

Business Expansion

Singapore

  • Southeast Asia

Business Expansion

NavInfo’s Footprint

slide-5
SLIDE 5

Cooperation between NavInfo & NVIDIA

NavInfo’s Training Model

NavInfo trains and optimizes its models on NVIDIA DGX-1 servers

NavInfo’s Perception Technology

NVIDIA DRIVE Software use NavInfo’s HD Map to give customer a way of localizing their autonomous driving car.

NVIDIA’s new system - DRIVE Localization

NavInfo’s research lab developed a vision based system that can detect and classify objects in real time on NVIDIA Xavier

Artificial Intelligence

We support and benefit each other’s achievements, driven by AI, to generate better products and services to our customers and end users

slide-6
SLIDE 6

NavInfo Service Offerings in Europe

Autonomous driving and robotic solutions AI based algorithms AI based solutions for different industries

slide-7
SLIDE 7

Results from TU/e sponsored by NavInfo Europe

Results with the highest speed and lowest error in Stereo Odometry (not Lidar, not SLAM)

  • No. 4 in the world in the Semantic Segmentation

Reference: [1] Panagiotis Meletis and Gijs Dubbelman, "Training of Convolutional Networks on Multiple Heterogeneous Datasets for Street Scene Semantic Segmentation", In IEEE IV 2018. June 2018, 0–8.

slide-8
SLIDE 8

Deep Learning for HD Mapping

slide-9
SLIDE 9

HD Mapping

Includes highly accurate lane and road features.

slide-10
SLIDE 10

Deep Learning for HD Mapping Feature Extraction

  • Deep learning provides automating feature extraction from video feed of

collection vehicles

Cameras, LIDARs Real Time Feature Extraction

(Semi)

Automated Map Updating

Localization

slide-11
SLIDE 11

Real Time Object Detection

Geetank Raipuria Computer Vision Engineer Advanced Research Lab NavInfo Europe Andrei Pata Software Engineer Advanced Research Lab NavInfo Europe

slide-12
SLIDE 12

Object Detector based on Deep Convolutional Neural Network architecture, to localize and classify road signs and traffic lights from a real-time camera feed

Real-Time Object Detection System

Two Stage System: Best of both worlds High Accuracy Low Inference time

slide-13
SLIDE 13

Features Supported 350+ supported classes including Traffic Signs

  • Regulatory Signs
  • Warning Signs
  • Guide Signs
  • Information Signs
  • Road Work Signs

Signboards Traffic Lights Digital Traffic Signs

Real-Time Object Detection System

Real Time Performance 2-3x Speedup using Tensor RT About : 35 fps on INT8 on NVIDIA Xavier SoC : 110 fps on Titan XP Inference at Full HD resolution (1920x1080) Able to extract and classify object as small as 25x25 pixels Robust to extreme lighting conditions

slide-14
SLIDE 14

Sample Detections

slide-15
SLIDE 15

Sample Detections

slide-16
SLIDE 16

Sample Detections

slide-17
SLIDE 17

Demo Realtime Object Detection

Advanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China. We comply with all Chinese regulations regarding the processing of China data.

Online video available at https://youtu.be/-QtYF0XUZh0

slide-18
SLIDE 18

Real Time Semantic Segmentation

Matti Jukola Software Engineer Advanced Research Lab NavInfo Europe Ahmed Badar Computer Vision Engineer Advanced Research Lab NavInfo Europe

slide-19
SLIDE 19

Deep learning architecture to segment and extract road markings at pixel level

Camera input Segmentation Network Segmented road markings

Fully Convolutional Neural Network Real-Time Semantic Segmentation

slide-20
SLIDE 20

Multi-branch segmentation network

Our model is based

  • n a multi-branch

convolutional neural network architecture

Top-view transformation

Camera input Top-down view Top-down predictions Front-view predictions

Front-view transformation

Real-Time Semantic Segmentation

slide-21
SLIDE 21

Real-Time Semantic Segmentation

Real Time Performance 3x Speedup using Tensor RT About : 90 fps on NVIDIA Xavier SoC : 300 fps on Titan XP Inference at 1024 × 384 image sizes This includes image transformations (top and front view) Currently supports 40 Road Marking Classes, including:

  • Lane lines
  • Arrows
  • Text
slide-22
SLIDE 22

Sample Segmentations

Input Image Prediction Ground Truth

slide-23
SLIDE 23

Sample Segmentations

Input Image Prediction Ground Truth

slide-24
SLIDE 24

Demo Realtime Semantic Segmentation

Advanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China. We comply with all Chinese regulations regarding the processing of China data.

Online video available at https://youtu.be/E4hU-COkHDo

slide-25
SLIDE 25

Real Time Scene Understanding

Elahe Arani Senior AI Researcher Advanced Research Lab NavInfo Europe Mahmoud Gamal Computer Vision Engineer Advanced Research Lab NavInfo Europe

slide-26
SLIDE 26

Real-Time Scene Understanding

A real time unified object detection and semantic segmentation for autonomous driving cars/HD mapping.

DNN

slide-27
SLIDE 27

Joint Object Detection and Segmentation

Currently supports 40 Road Marking Classes and 350+ Road Sign classes including: – Traffic Signs – Gantry Signboards – Traffic Lights – Digital Traffic Signs – Lane Markings – Text – Arrows Performance Inference at 512x512 image sizes About 45 FPS on Titan XP Other Features Supported – Guard Rails – Curbs – Speed Limits on Road

slide-28
SLIDE 28

Sample Detections

slide-29
SLIDE 29

Input Image Ground Truth Prediction

Sample Segmentation

slide-30
SLIDE 30

Prediction (Dec) Ground Truth Prediction (Seg)

Joint Detections and Segmentation

slide-31
SLIDE 31

Occlusion Handling

Input Image Prediction (Dec) Ground Truth Prediction (Seg)

slide-32
SLIDE 32

Occlusion Handling

Input Image Prediction (Dec) Ground Truth Prediction (Seg)

slide-33
SLIDE 33

Demo Realtime Scene Understanding

Advanced Research Lab/NIEU focusses on the development of the AI algorithm, the data processing is done completely in China. We comply with all Chinese regulations regarding the processing of China data.

Online video available at https://youtu.be/NJVNFfueKb4

slide-34
SLIDE 34

Real-Time Object Detection and Semantic Segmentation

NavInfo Europe Wei Li

Corporate development manager Presented at GTC 2019 Session S9351

Geetank Raipuria

Computer Vision Engineer