TVM Deep Learning on Bare-Metal Devices Pratyush Patel No OS - - PowerPoint PPT Presentation

▶

tvm

TVM Deep Learning on Bare-Metal Devices Pratyush Patel No OS - - PowerPoint PPT Presentation

Nov 17, 2023 1.23k likes •1.46k views

TVM Deep Learning on Bare-Metal Devices Pratyush Patel No OS stack Extend TVM to support bare-metal devices Optimization High-Level Differentiable IR AutoTVM Tensor Expression IR LLVM, CUDA VTA AutoVTA Hardware FPGA ASIC Fleet

slide-1

SLIDE 1

μTVM

Deep Learning on Bare-Metal Devices

Pratyush Patel

slide-2

SLIDE 2

slide-3

SLIDE 3

slide-4

SLIDE 4

slide-5

SLIDE 5

No OS stack

slide-6

SLIDE 6

Extend TVM to support bare-metal devices

Optimization AutoTVM AutoVTA High-Level Differentiable IR Tensor Expression IR VTA

FPGA ASIC

Hardware Fleet LLVM, CUDA

slide-7

SLIDE 7

Extend TVM to support bare-metal devices

Optimization AutoTVM AutoVTA High-Level Differentiable IR Tensor Expression IR VTA

FPGA ASIC

Hardware Fleet LLVM, CUDA

Most bare-metal devices do not support LLVM

slide-8

SLIDE 8

Extend TVM to support bare-metal devices

Optimization AutoTVM AutoVTA High-Level Differentiable IR Tensor Expression IR VTA

FPGA ASIC

Hardware Fleet LLVM, CUDA C, C++

slide-9

SLIDE 9

Extend TVM to support bare-metal devices

Optimization AutoTVM AutoVTA High-Level Differentiable IR Tensor Expression IR VTA

FPGA ASIC

Hardware Fleet LLVM, CUDA C, C++

Upstreamed!

slide-10

SLIDE 10

Extend TVM to support bare-metal devices

Optimization AutoTVM AutoVTA High-Level Differentiable IR Tensor Expression IR VTA

FPGA ASIC

Hardware Fleet LLVM, CUDA C, C++

Many other backends Upstreamed!

slide-11

SLIDE 11

μTVM builds upon AutoTVM

μTVM Runtime C Code Generator μDevice API run send program

ptimize

slide-12

SLIDE 12

A closer look at μTVM

μTVM Runtime C Code Generator μDevice API OpenOCD telnet JTAG run

slide-13

SLIDE 13

A closer look at μTVM

μTVM Runtime C Code Generator μDevice API infer.c OpenOCD telnet JTAG IR->code run

slide-14

SLIDE 14

A closer look at μTVM

μTVM Runtime C Code Generator μDevice API infer.c infer.o vendor  gcc OpenOCD telnet JTAG IR->code run

slide-15

SLIDE 15

A closer look at μTVM

μTVM Runtime C Code Generator μDevice API infer.c infer.o vendor  gcc remap infer OpenOCD ld linker telnet JTAG IR->code run

slide-16

SLIDE 16

A closer look at μTVM

μTVM Runtime C Code Generator μDevice API infer.c infer.o vendor  gcc remap infer OpenOCD ld linker telnet JTAG custom loader IR->code run

slide-17

SLIDE 17

A closer look at μTVM

μTVM Runtime C Code Generator μDevice API infer.c infer.o vendor  gcc remap infer OpenOCD ld linker telnet JTAG custom loader IR->code send program run

slide-18

SLIDE 18

Next steps

Iron out interfaces with actual hardware.
Optimize with AutoTVM.
Support restricted and configurable model sizes.
Enable custom data types such as fixed-point precision formats.

Get in touch! Pratyush Patel — patelp1@cs.uw.edu