Proposing a Fast and Scalable Systolic Array for Matrix - - PowerPoint PPT Presentation
Proposing a Fast and Scalable Systolic Array for Matrix Multiplication Bahar Asgari , , Ra Ramyad Ha Hadidi, , Hy Hyesoon esoon Ki Kim Click to edit Master subtitle style Matrix Multiplication 2 Matrix multiplication is the key operation
2
} Includes several similar operations (i.e., multiply and accumulation) } Captures high data reuse rate
F F F
…
C
K F W
H
C
F F
=
W
H
K
K
F 2.C
× = K
W.H W.H
F 2.C
Convolution:
Matrix Multiplication:
3
} Non-stationary
} None of the operands are stationary
a MAC unit
4
} Non-stationary
} None of the operands are stationary } Phase 1: } only processing } Time steps: 1 }
5
} Non-stationary
} None of the operands are stationary } Phase 1 } only processing } Time steps: 2 }
6
} Non-stationary
} None of the operands are stationary } Phase 1: } only processing } Time steps: 3 }
7
} Non-stationary
} None of the operands are stationary } Phase 1: } only processing } Time steps: 4 }
8
} Non-stationary
} None of the operands are stationary } Phase 1: } only processing } Time steps: 5 }
9
} Non-stationary
} None of the operands are stationary } Phase 1: } only processing } Time steps: n + m
10
} Non-stationary
} None of the operands are stationary } Phase 2: } processing and offloading } Time steps: n + m + 1
Phase 1
11
} Non-stationary
} None of the operands are stationary } Phase 3: } only offloading } Time steps: n + m + p - 2 + 1 }
Phase 2 Phase 1
12
} Non-stationary
} None of the operands are stationary } Phase 3: } only offloading } Time steps: n + m + p - 2 + 2 }
Phase 2 Phase 1
13
} Non-stationary
} None of the operands are stationary } Phase 3: } only offloading } Time steps: n + m + p - 2 + n }
Phase 2 Phase 1
14
} Non-stationary
} None of the operands are stationary } Phase 3: } only offloading } Time steps: 2n + m + p - 2 }
15
} Stationary
} One operand (here, B) is stationary
a MAC unit
16
} Stationary
} One operand (here, B) is stationary
} only loading B } Time steps: 1
17
} Stationary
} One operand (here, B) is stationary
} only loading B } Time steps: m - 1
18
} Stationary
} One operand (here, B) is stationary
} loading B and processing } Time steps: m - 1 + 1
Phase 1
19
} Stationary
} One operand (here, B) is stationary
} only processing } Time steps: m + 1
Phase 1 &2
20
} Stationary
} One operand (here, B) is stationary
} only processing } Time steps: m + m - 1
Phase 1 &2
21
Phase 1 &2&3
} Stationary
} One operand (here, B) is stationary
} processing and offloading } Time steps: 2m - 1 + 1
22
} Stationary
} One operand (here, B) is stationary
} processing and offloading } Time steps: 2m - 1 + 2
Phase 1 &2&3
23
} Stationary
} One operand (here, B) is stationary
} processing and offloading } Time steps: 2m - 1 + 3
Phase 1 &2&3
24
} Stationary
} One operand (here, B) is stationary
} processing and offloading } Time steps: 2m - 1 + n + p - 2
Phase 1 &2&3
25
} Stationary
} One operand (here, B) is stationary
} only offloading } Time steps: 2m -1 + n + p - 2 + 1
Phase 4 Phase 1 &2&3
26
} Stationary
} One operand (here, B) is stationary
} only offloading } Time steps: n + 2m + p - 2
27
31
} Multiplication } Additions
} Latency increases sublinearly with the input size
}
Multiplier array
}
Adder-tree array
This can be done in log(m) for m numbers
a multiplier
an adder tree
32
a multiplier
an adder tree
33
} only loading B } Time steps: 1
34
} only loading B } Time steps: m-1
35
} loading B and multiplication } Time steps: m - 1 + 1
Phase 1
36
} multiplication and addition } Time steps: m + 1
Phase 1 &2
37
} multiplication and addition } Time steps: m + 2
Phase 1 &2
38
} multiplication and addition } Time steps: m + 3
Phase 1 &2
39
} multiplication and addition } Time steps: m + 4
Phase 1 &2
40
} only addition } Time steps: m + n + p - 2 + 1
Phase 1 &2 Phase 3
41
} only addition } Time steps: m + n + p - 2 + 2
Phase 1 &2 Phase 3
42
} only addition } Time steps: m + n + p - 2 + 3
Phase 1 &2 Phase 3
43
} only addition } Time steps: m + n + p - 2 + log (m)
Phase 1 &2 Phase 3
44
} only addition } Time steps: n + m + log(m) + p - 2
46
} ZYNQ XC7z020 } Vivado HLS
} DNNs (VGG16, VGGS, AlexNet, CifarNet, ResNet50)
} Latency } Energy consumption
47
} 1.99x faster than non-stationary while consuming 2.12x less energy } 1.83x faster than stationary while consuming 2.27x less energy
1.99 1.83
48
} because of their unique interconnections that satisfies the unique requirement of data
reuse in matrix multiplication.
} Latency is the key factor for single-batch inference!
}
It is faster than both prior proposals when the size of the operands grows