TVM Deep Learning on Bare-Metal Devices Pratyush Patel No OS - PowerPoint PPT Presentation

μ TVM Deep Learning on Bare-Metal Devices Pratyush Patel

No OS stack

Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR LLVM, CUDA VTA AutoVTA Hardware FPGA ASIC Fleet

Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR Most bare-metal devices LLVM, CUDA VTA AutoVTA do not support LLVM Hardware FPGA ASIC Fleet

Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR LLVM, CUDA VTA C, C++ AutoVTA Hardware FPGA ASIC Fleet

Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR Upstreamed! LLVM, CUDA VTA C, C++ AutoVTA Hardware FPGA ASIC Fleet

Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR Upstreamed! LLVM, CUDA VTA C, C++ AutoVTA Hardware FPGA ASIC Fleet Many other backends

μ TVM builds upon AutoTVM μ TVM Runtime μ Device C Code Generator API send program run optimize

A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API run

A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API run IR -> code infer.c

A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API run IR -> code vendor 
 gcc infer.c infer.o

A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API run IR -> code vendor 
 ld linker gcc remap infer.c infer.o infer

A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API custom loader run IR -> code vendor 
 ld linker gcc remap infer.c infer.o infer

A closer look at μ TVM μ TVM Runtime telnet JTAG μ Device C Code OpenOCD Generator API send program custom loader run IR -> code vendor 
 ld linker gcc remap infer.c infer.o infer

Next steps • Iron out interfaces with actual hardware. • Optimize with AutoTVM. • Support restricted and configurable model sizes. • Enable custom data types such as fixed-point precision formats. Get in touch! Pratyush Patel — patelp1@cs.uw.edu