Wei Liu 1 , Yifan Gong 1 , Hao Wu 1 , Jidong Zhai 2 , Jiangming Jin 1 - - PowerPoint PPT Presentation

wei liu 1 yifan gong 1 hao wu 1 jidong zhai 2 jiangming
SMART_READER_LITE
LIVE PREVIEW

Wei Liu 1 , Yifan Gong 1 , Hao Wu 1 , Jidong Zhai 2 , Jiangming Jin 1 - - PowerPoint PPT Presentation

Memory-Centric Communication Mechanism for Real- time Autonomous Navigation Applications Wei Liu 1 , Yifan Gong 1 , Hao Wu 1 , Jidong Zhai 2 , Jiangming Jin 1 1. TuSimple Inc. 2. Tsinghua University, BNRist Agenda 1. Background &


slide-1
SLIDE 1

Memory-Centric Communication Mechanism for Real- time Autonomous Navigation Applications

Wei Liu1, Yifan Gong1, Hao Wu1, Jidong Zhai2, Jiangming Jin1

1. TuSimple Inc. 2. Tsinghua University, BNRist

slide-2
SLIDE 2

Agenda

  • 1. Background & Motivation
  • 2. Virtual Layer based Architecture
  • 3. Pre-record Memory Allocation Algorithm
  • 4. Evaluation
slide-3
SLIDE 3

Background & Motivation

Artificial Inteligence: CV, NLP, RL Autonomous Navigation System Resource Isolation & Modularity for fast deploy. System consists of Many Processes(C++/Python)

slide-4
SLIDE 4

Background & Motivation

Multi-Process

Resource Isolation Modularity Multi-language Latency Memory Throghput advantage disadvantage braking distance

  • ut-of-date

perception result Inter-Process Communication

slide-5
SLIDE 5

Background & Motivation

+ 풂 풍풄 풖풍 풂‷ 㰍䅁 ੪ ≈ ퟐ%

slide-6
SLIDE 6

Background & Motivation

IPC latency breakdown: Memory Allocation (De)Serialization Memory Copy (De)Convert

slide-7
SLIDE 7

Background & Motivation

IPC latency breakdown: Memory Allocation (De)Serialization Memory Copy (De)Convert

slide-8
SLIDE 8

Background & Motivation

IPC latency breakdown: Memory Allocation (De)Serialization Memory Copy (De)Convert

data movement

slide-9
SLIDE 9

Memory-Centric Mechanism

Memory Allocation Data Movement Virtual Layer based Architecture Pre-record Memory Allocation problem solution Z-framework

slide-10
SLIDE 10

Virtual Layer based Architecture

ROS2-like Architecture Z-framework

slide-11
SLIDE 11

Virtual Layer based Architecture

Z-framework Python Support Problem Socket-base IPC Problem

slide-12
SLIDE 12

Virtual Layer based Architecture

Z-framework Python Support Problem Socket-base IPC Problem

slide-13
SLIDE 13

Virtual Layer based Architecture

Process-1 Process-2 Process-3

slide-14
SLIDE 14

Pre-record Memory Allocation Algorithm

Low Efficiency for Memory Allocation A Memory Cost Model Memory Pool Techniques page fault Virtual memory Physical memory page-fault

slide-15
SLIDE 15

Pre-record Memory Allocation Algorithm

ObjectMap

slide-16
SLIDE 16

Other Optimization Techniques

Efficient Data Assigment Efficient Data Extraction Distributed Environment Supporting

slide-17
SLIDE 17

Evaluation

Total IPC Costs of Different Sizes of Images Image IPC Latency from 256KB to 8MB Image IPC Latency from 256KB to 8MB Mainly Solved: memory allocation overhead Mainly Solved: data movement overhead

slide-18
SLIDE 18

Evaluation

Total IPC Costs of Different Types of Messages

slide-19
SLIDE 19

Evaluation

Performance Improvement with Virtual Layer based Architecture Virtual Layer Technique Pre-record Technique benefit by:

slide-20
SLIDE 20

Evaluation

Workflow Latency Workflow Throughput

Real-world workflow test

slide-21
SLIDE 21

Thanks