IR An intermediate representation for transforming and optimizing - - PowerPoint PPT Presentation

µIR – An intermediate representation for transforming and optimizing the microarchitecture of application accelerators

1

Amirali Sharifian1, Reza Hojabr1, Navid Rahimi1, Sihao Liu2, Apala Guha1 Tony Nowatzki2 and Arrvindh Shriraman1

Simon Fraser University1, UCLA2

https://github.com/sfu-arch/uir

The Accelerator Flowchart

2

Study the Application

Object Detection Model

The Accelerator Flowchart

3

Study the Application

Object Detection Model

Design Hardware RTL Design

f()

Valid Input Weights Output Clock Reset Valid

The Accelerator Flowchart

4

Study the Application

Object Detection Model

Design Hardware RTL Design

f()

Valid Input Weights Output Clock Reset Valid

New Software

Compiler

Feature Dimension (Transistor Count) — Source: IBS 125 250 375 500

65nm 45/40 nm 28nm 20nm 16/14nm

Architecture Software Validation

80 160 240 320

Cost(M$)

Problems With Design Flowchart

5

ISA-based Flowchart

6

Compile Execute

✓Isolate Application from Architecture

ISA


Application

GPU

ISA-based Flowchart

7

Compile Execute

✓Isolate Application from Architecture

ISA


Application

GPU

ISA-based Flowchart

8

Accelerator

?

✓Isolate Application from Architecture

• Not expressive enough to create hardware

Compile Execute

ISA


Application

ISA-based Flowchart

9

?

✓Isolate Application from Architecture

• Not expressive enough to create hardware
• Not precise enough to explore hardware

Compile Execute

ISA


Application

µIR — A New Accelerator Flowchart

10

✓End-to-End flow — Existing software for behavior/functionality

Applications

Auto-Synthesis

Behavior CREATE

RTL

Microarch.
 Representation (µIR)

µIR — A New Accelerator Flowchart

11

✓End-to-End flow — Existing software for behavior/functionality ✓Reduce effort — Compiler to extract behaviour

Applications

Auto-Synthesis

Behavior CREATE

RTL

Microarch.
 Representation (µIR)

µIR — A New Accelerator Flowchart

12

✓End-to-End flow — Existing software for behavior/functionality ✓Reduce effort — Compiler to extract behaviour ✓Design exploration — New model for exploring architectures

Applications

Auto-Synthesis

Behavior CREATE

Microarch.
 Representation (µIR)

µIR — A New Accelerator Flowchart

13

✓End-to-End flow — Existing software for behavior/functionality ✓Reduce effort — Compiler to extract behaviour ✓Design exploration — New model for exploring architectures ✓Extensibility — Extensible to capture domain information

Applications

Auto-Synthesis

Behavior CREATE

Microarch.
 Representation (µIR)

µIR — A New Accelerator Flowchart

14

✓End-to-End flow — Existing software for behavior/functionality ✓Reduce effort — Compiler to extract behaviour ✓Design exploration — New model for exploring architectures ✓Extensibility — Extensible to capture domain information

Applications

Auto-Synthesis

Behavior CREATE

Microarch.
 Representation (µIR)

15

Application Compile

GPU TPU

LLVM IR

/Cilk

µIR Graph

µIR A New Accelerator Flowchart

16

Application Compile LLVM IR

/Cilk

Hierarchical data flow Structure Graph

µIR Graph

µIR A New Accelerator Flowchart

17

Application Compile LLVM IR

/Cilk

Hierarchical data flow Structure Graph

µIR Graph

µIR A New Accelerator Flowchart

18

µOpt

Application Compile LLVM IR

/Cilk

Hierarchical data flow Structure Graph

µIR Graph

µIR A New Accelerator Flowchart

19

µOpt

Application Compile LLVM IR

/Cilk

Chisel

µLib

• Motivation
• µIR behaviour graph
• µIR structural graph
• Evaluation
• Summary

20

parallel_for(i=0; I<(M-W); i++) parallel_for(j=0; j<W; j++)

• utput[i] += input[i+j] * weight[j];

µIR Behavioural Graph (1/3)

X +=

Output Input Weights

Conv =

N−W

X

i=0 W

X

j=0

Wij ∗ INij

parallel_for(i=0; I<(M-W); i++) parallel_for(j=0; j<W; j++)

• utput[i] += input[i+j] * weight[j];

µIR Behavioural Graph (2/3)

• Hierarchical and heterogeneous task graph
• Decompose the input program to task blocks.
• Graph of task blocks
• Implement arbitrary heterogenous parallel and

serial

22

for i for j Body

Spawn Sync

µIR Behavioural Graph (3/3)

• µIR Dataflow graph for each task block
• Pipelined (single and multi-cycle)
• Typed (e.g., FP precision)
• Non-deterministic cycle ops (e.g., Mem ops)
• Permits local transformations because tasks

are asynchronous

23

LD LD

X +

LD ST

• utput[i] += input[i+j] * weight[j];

?

• Motivation
• µIR behaviour graph
• µIR structural graph
• Evaluation
• Summary

24

µIR Structural Graph : Stage 0 — Unoptimized

25

DRAM

MAC

µIR Structural Graph : Stage 1 — Locality

26

DRAM

MAC

Buffer

• Define custom memory hierarchy — Buffer, FIFO

µIR Structural Graph : Stage 1 — Locality

27

DRAM

MAC

Buffer

• Define custom memory hierarchy — Buffer, FIFO

µIR Structural Graph : Stage 2 — Tiling

28

DRAM

• Define custom memory hierarchy — Buffer, FIFO
• Tiling asynchronous task blocks

MAC MAC

µIR Structural Graph : Stage 3 — Pipelining

29

DRAM

• Define custom memory hierarchy — Buffer, FIFO
• Tiling asynchronous task blocks
• Start pipelining the operands

MAC MAC MAC MAC

µIR Structural Graph : Stage 3 — Pipelining

30

DRAM

• Define custom memory hierarchy — Buffer, FIFO
• Tiling asynchronous task blocks
• Start pipelining the operands
• Define custom operator

Systolic Array Systolic Array

µIR Structural Graph : Stage 4 Higer-Order Ops

• Extensible IR
• Introduce new ops and new types
• Existing stages/transformations have to work
• µIR components are generic: Dataflow nodes, buffers

and memory network

31

DRAM Tensor 
 Network

GEMM

• Motivation
• µIR behaviour graph
• µIR structural graph
• Evaluation
• Summary

32

µIR Targets

• Cycle accurate simulation (e.g., gem5)
• C++ driver and a Python binding for ease of use
• Cache-based memory interface

33

Simulator

https://github.com/sfu-arch/uir-sim https://github.com/sfu-arch/uir-fpga

FPGA

• FPGA
• Arria 10 Soc
• Scripted process

µIR Iterative Optimizations — Stacking

• Normalized to the baseline accelerator

34

0.25 0.5 0.75 1

STENCIL IMA.SC GEMM FFT SPMV CONV. DENSE16 SOFTM16

Banking, Localization, Op-Fusion, Tiling

Between: 20% — 80%

µIR Productivity

35

#Changes µIR #Changes Firrtl Ratio

Saxpy

9 73

9.3

Stencil

12 142

12.4

Image Sca.

10 84

8.4

36

Available Now!

https://github.com/sfu-arch/uir

µIR — lib Simulator

https://github.com/sfu-arch/uir-sim https://github.com/sfu-arch/uir-fpga

FPGA

https://github.com/sfu-arch/uir-docker

Docker

Thanks to Open-Source community!