Wei Liu 1 , Yifan Gong 1 , Hao Wu 1 , Jidong Zhai 2 , Jiangming Jin 1 - PowerPoint PPT Presentation

Memory-Centric Communication Mechanism for Real- time Autonomous Navigation Applications Wei Liu 1 , Yifan Gong 1 , Hao Wu 1 , Jidong Zhai 2 , Jiangming Jin 1 1. TuSimple Inc. 2. Tsinghua University, BNRist

Agenda 1. Background & Motivation 2. Virtual Layer based Architecture 3. Pre-record Memory Allocation Algorithm 4. Evaluation

Background & Motivation Artificial Inteligence: CV, NLP, RL Autonomous Navigation System Resource Isolation & Modularity for fast deploy. System consists of Many Processes(C++/Python)

Background & Motivation advantage disadvantage result Resource Isolation braking Latency distance Multi-Process Modularity out-of-date Memory perception Throghput Multi-language Inter-Process Communication

Background & Motivation 푰� � ≈ ퟐ�% 푰� � + �풂 풍풄 풖풍 풂‷ 㰍䅁 ੪

Background & Motivation IPC latency breakdown: Memory Allocation (De)Serialization Memory Copy (De)Convert

Background & Motivation IPC latency breakdown: Memory Allocation (De)Serialization Memory Copy (De)Convert

Background & Motivation IPC latency breakdown: Memory Allocation (De)Serialization Memory Copy (De)Convert data movement

Memory-Centric Mechanism problem Data Movement Memory Allocation Virtual Layer based Architecture Pre-record Memory Allocation solution Z-framework

Virtual Layer based Architecture ROS2-like Architecture Z-framework

Virtual Layer based Architecture Python Support Problem Socket-base IPC Problem Z-framework

Virtual Layer based Architecture Python Support Problem Socket-base IPC Problem Z-framework

Virtual Layer based Architecture Process-1 Process-2 Process-3

Pre-record Memory Allocation Algorithm page-fault Low Efficiency for Memory Allocation page fault A Memory Cost Model Virtual Physical memory memory Memory Pool Techniques

Pre-record Memory Allocation Algorithm ObjectMap

Other Optimization Techniques Efficient Data Assigment Efficient Data Extraction Distributed Environment Supporting

Evaluation Image IPC Latency from 256KB to 8MB Mainly Solved: memory allocation overhead Image IPC Latency from 256KB to 8MB Mainly Solved: data movement overhead Total IPC Costs of Different Sizes of Images

Evaluation Total IPC Costs of Different Types of Messages

Evaluation benefit by: Virtual Layer Technique Pre-record Technique Performance Improvement with Virtual Layer based Architecture

Evaluation Real-world workflow test Workflow Latency Workflow Throughput

Thanks