Perception at the edge to heal the HERE HD Live Map Vlad Shestak - - PowerPoint PPT Presentation

▶
perception at the edge to heal the here hd live map
SMART_READER_LITE
LIVE PREVIEW

Perception at the edge to heal the HERE HD Live Map Vlad Shestak - - PowerPoint PPT Presentation

Feb 22, 2024 202 likes •501 views

Perception at the edge to heal the HERE HD Live Map Vlad Shestak & Josh Finken Edge Perception March 20, 2019 HERE in numbers HERE Maps on board of 100M 9,000 + vehicles and counting Employees in 56 countries focused on delivering the

slide-1
SLIDE 1

Perception at the edge to heal the HERE HD Live Map

Vlad Shestak & Josh Finken Edge Perception March 20, 2019

slide-2
SLIDE 2

HERE in numbers

Countries mapped Years of experience transforming location technology

9,000+

Employees in 56 countries focused on delivering the world’s best map and location technologies HERE Maps on board of

100M

vehicles and counting TB map data collected per day

4of5

In-car navigation systems in Europe and North America use HERE maps

+

collecting data for maps HERE cars 3D data points per second per car

700,000

555k+ km

for Autonomous Driving HD Live Map covering

GTC 2019 1

slide-3
SLIDE 3

Key building blocks for autonomous driving

HD maps are essential in autonomous solutions

Sensors Perception Stack Decision Making

Intelligent sensors to see, feel and sense surroundings Real-time information for localization and path planning Deep learning and sensor fusion for decision making

HD Maps

Highly detailed, helping pinpoint a car’s location and understand its surroundings

GTC 2019 2

slide-4
SLIDE 4

HERE HD Live Map

Data layers providing key benefits

Published in NDS and Protobuf formats

Localization Enhanced sensor functionality Localization Route planning Rules of the road Enhanced sensor functionality Route planning Rules of the road Enhanced sensor functionality

Layer Data Benefits

Road side objects (furniture)

Ex: road signs, barriers

Lane level features

Ex: Lane lines, lane widths, lane markings, etc.

HERE’s Standard Definition (Infotainment) map with ADAS attributes

Ex: Road topology, direction of travel, elevation, slop, etc.

HD Localization Model HD Lane Model Road Model

GTC 2019 3

slide-5
SLIDE 5

Industry Capture Hardware

Novatel GPS Antenna Velodyne HDL-32 LiDAR 80 MP Cameras Novatel Span CPT 7 IMU

GTC 2019 4

slide-6
SLIDE 6

Typical Urban Scene

GTC 2019 5

slide-7
SLIDE 7

GTC 2019 6

slide-8
SLIDE 8

Active Learning Loop

Selection Queries Learn a Model Machine Learning Model Human Annotator

and/or

Machine Model Unlabeled Pool Labeled Library

GTC 2019 7

Billions of images

slide-9
SLIDE 9

Maintenance Localization Build Out Quality

HERE’s Maintenance Strategy to HD Maps

Industrial Capture OneMap Alliance Agnostic Crowd- sourced Quality Index

  • The world isn’t static, it’s constantly shifting and evolving
  • Mapping systems that support autonomous driving functionalities require rapid

map updates to reflect real-world changes

  • Freshness can only be achieved through a constant and broad flow of

sensor data

  • Map maintenance is transitioning from a systematic industrial capture (HERE True)

mapping model to a self-healing crowd-source model

GTC 2019 8

slide-10
SLIDE 10

A high definition, continuously updated map requires the crowd

HD Live Map

Harness crowdsourced sensor data… …for near real-time updates

GTC 2019 9

slide-11
SLIDE 11

HERE’s self-healing map process

Map update Publish Aggregation Localization Change detection Observations Features Internal HD map Mapltes | RWOs HD Live Map

Points

(signs, signals, pavement markings…)

Polylines

(lane markings, road edges…)

Many to one

Drive paths

HD map tiles sensor observations 3rd Party Sources HERE True Pipeline

Sensor collection & ingestion Aggregation in the cloud Creation of a new feature Update map & publish

GTC 2019 10

slide-12
SLIDE 12

HERE’s self-healing map process

Map update Publish Aggregation Localization Change detection Observations Features Internal HD map Maplets | RWOs HD Live Map

Points

(signs, signals, pavement markings…)

Polylines

(lane markings, road edges…)

Many to one

Drive paths

HD map tiles sensor observations 3rd Party Sources HERE True Pipeline

GTC 2019 11

slide-13
SLIDE 13

HERE’s self-healing map process

Map update Publish Aggregation Localization Change detection Observations Features Internal HD map Maplets | RWO HD Live Map

Points

(signs, signals, pavement markings…)

Polylines

(lane markings, road edges…)

Many to one

Drive paths

HD map tiles sensor observations 3rd Party Sources HERE True Pipeline

GTC 2019 12

slide-14
SLIDE 14

Aggregation

GTC 2019 13

slide-15
SLIDE 15

Aggregation

GTC 2019 14

slide-16
SLIDE 16

HERE’ self-healing map process

Map update Publish Aggregation Localization Change detection Observations Features Internal HD map Maplets | RWOs HD Live Map

Points

(signs, signals, pavement markings…)

Polylines

(lane markings, road edges…)

Many to one

Drive paths

HD map tiles sensor observations 3rd Party Sources HERE True Pipeline

GTC 2019 15

slide-17
SLIDE 17

HERE’s self-healing map process

Map update Publish Aggregation Localization Change detection Observations Features Internal HD map Maplets | RWOs

Points

(signs, signals, pavement markings…)

Polylines

(lane markings, road edges…)

Many to one

Drive paths

HD map tiles sensor observations 3rd Party Sources HERE True Pipeline

NDS Protobuf

HD Live Map

GTC 2019 16

slide-18
SLIDE 18

Cloud and edge analytics: What to do where, best?

Map update Publish Aggregation Localization Change detection Observations Features Internal HD map Maplets | RWOs HD Live Map

Points

(signs, signals, pavement markings…)

Polylines

(lane markings, road edges…)

Many to one

Drive paths

HD map tiles sensor observations 3rd Party Sources HERE True Pipeline

GTC 2019 17

slide-19
SLIDE 19

In-Vehicle Processing

HERE Backend In-Vehicle

HERE Maplets HD Live Map Change Detection (optional) Cameras Lidar Radar Ultrasonic … Edge Perception Stack

GTC 2019 18

slide-20
SLIDE 20

Cyclops

Off the Shelf, At the Edge, In the Vehicle

GTC 2019 19

slide-21
SLIDE 21

Cyclops

Off the Shelf, At the Edge, In the Vehicle

GTC 2019 20

Standard Configuration

  • Android smartphone
  • Imagery at 30 Hz
  • Android Fused GPS positions at 1 Hz
  • MEMS readings at 50 Hz
  • Efficiently packed
  • Wireless stream
  • NVIDIA GPU compute
slide-22
SLIDE 22

Cyclops: RTK GNSS

GTC 2019 21

Optional configuration: positioning via RTK

  • Lightweight digital antenna
  • Connects directly to an Android phone via USB
  • Greatly improved GNSS accuracy
  • 10Hz position update rate

rate

slide-23
SLIDE 23

Real-Time Perception Architecture

GTC 2019 22

  • Peer-to-peer set of decentralized microservices
  • Nodes run stand-alone or deployed as a set of services
  • Wireless: UDP, TCP and HTTP

Change or Cloud DNN Model Runtime Tracking and Sensor Fusion Observation Reconstruction Maplet Generation

Compute: Jetson AGX Xavier Optional Laptop: visualization Sensor stream: smartphone

slide-24
SLIDE 24

Cyclops: Maplets

GTC 2019 23

Sensoris: Sensor Interface Specification

  • International, standardized interface for exchange
  • Within the vehicle
  • Vehicle to Cloud
  • Cloud to Cloud
  • Is at the core of HERE Maplets

Maplets:

  • Structurally combines in-vehicle observations and related

accuracy requirements

  • Sensor-agnostic, low data footprint for instant data

transmission

  • Populate into SENSORIS
  • Python and C++ implementations available
slide-25
SLIDE 25

Cyclops: Demonstration Setup

GTC 2019 24

Sensor stream and visualization

  • Xiaomi Mi 8: sensor stream and wifi network
  • RTK GNSS positioning
  • Optional laptop: monitoring and visualization

Xavier in-vehicle

  • Power via USB Type-C (cigarette-lighter adapter)
  • Headless via an Intel 8265 wifi chip, antennas
  • Edge Perception software stack
slide-26
SLIDE 26

Cyclops: Demonstration

GTC 2019 25

slide-27
SLIDE 27

Cyclops: Assessment

GTC 2019 26

pole sign

slide-28
SLIDE 28

Stronger with our partners

GTC 2019 27