TVM TVM f for ed or edge c e com omputin ting p g pla latf - - PowerPoint PPT Presentation
TVM TVM f for ed or edge c e com omputin ting p g pla latf tform orm NTT Software Inno nnovation n Ce Center Ka Kazutaka Mo Morita In Inference in 5G era Edge Devices Offload MEC (Mobile edge computing) server Offload
NTT Software Inno nnovation n Ce Center
Edge Devices
Base station MEC (Mobile edge computing) server
Cloud Internet
Offload Offload ~10 10 ms ms la latency
Device Edge
GPU
Cloud Device Edge Device
AI chip CPU 5G data Big data
Computing resource Inference with data
Edge is one of the targets of AI accelerators High-end server- spec accelerators are available AI chip is unavailable for low-end devices Real-time inference with big data
5G
Interaction with other devices
Object segmentation inference Plane detection
Object detection inference can also provide collider from moving real world objects bouncing object
Occlusion Point cloud
Ma Many Inferenc nce tasks Inferenc nce with h bi big da data in n the he cloud ud
HYPER-REALITY: https://vimeo.com/166807261
Cloud
Point cloud data Captured images
Internet
Developing framework for edge computing
Device SDK Device
Developer Edge Cloud
Offload inference if necessary, based on device and communication status Distribute runtimes to device, edge, and cloud
data data
He Heter erogen eneo eous ru runti time with th
floa
g suppor
Dy Dyna namic r run untime Sma Smart NI NIC support
Auto tuning support would be also nice
Switch based on device and communication status Execute on edge via RPC
Edge Edge
Smart NIC CPU CPU GPU NIC FPGA
No overhead of PCIe communication or host memory access
Device Device
On device On edge On device On edge
Scheduler