NEAR DATA PROCESSING Mahdi Nazm Bojnordi Assistant Professor - - PowerPoint PPT Presentation
NEAR DATA PROCESSING Mahdi Nazm Bojnordi Assistant Professor School of Computing University of Utah CS/ECE 7810: Advanced Computer Architecture Requirement for Your Presentations Prepare for exactly 20m talk followed by 5m Q&A
¨ Prepare for exactly 20m talk followed by 5m Q&A ¨ Required components in your presentation
¤ Summary of the work
n Clearly present key ideas, mechanisms, and results
¤ Strength and weaknesses
n Slides highlighting the strengths and weaknesses
¤ Discussion
n Future research directions n Alternative ways to solving the problem
¨ Why processing in memory …
Near Data Processing Emerging Applications
Graph analytics, deep neural nets, etc.
Scaling Limitations
End of Denard scaling
New Technologies
3D stacking, resistive memories, etc.
[ref: GraphLab, ZME]
Near Data Processing
¨ Throughput
¤ High processing
¨ Power
¤ Thermal constraints
¨ Flexibility
¤ Must amortize manufacturing
¨ HMC: hybrid memory cube
[ref: micron]
¨ Current DDR4 maxes out at 25.6 GB/sec ¨ High Bandwidth Memory (HBM) led by AMD and NVIDIA ¤ Supports 1,024 bit-wide bus at125 GB/sec ¨ Hybrid Memory Cube (HMC) consortium led by Intel ¤ Claimed that 400 GB/sec possible ¨ Both based on stacked memory chips ¤ Limited capacity (won’t replace DRAM), but much higher than on-
¨ 3D crosspoint promises virtually unlimited memory
¤ Non-volatile ¤ 10x higher density ¤ Main limit for now 6GB/sec interface
Line Driver SA SA SA SA Decoder
¨ How to map applications to NDP technology?
Near Data Processing Emerging Applications
Graph analytics, deep neural nets, etc.
Scaling Limitations
End of Denard scaling
New Technologies
3D stacking, resistive memories, etc.
[ref: GraphLab, ZME]
¨ Example
¤ Stacks run memory intensive code ¤ Multiple stacks linked to host processor (SerDes) HMC
¨ Communication within and across stacks
¤ Fine-grained vs. coarse-grained blocks [PACT’15]
¨ Flexibility, area and power efficiency ¨ HRL: heterogeneous reconfigurable logic
[HPCA’16]
Configurable Logic Block
control logic
sigmoid, tanh, etc. Output MUX Block
(tree, cascading, parallel)
low cost and flexible Functional Unit
arithmetic/logic ops
and retiming
¨ 2.2x performance/Watt over FPGA ¨ Within 92% of ASIC performance
[HPCA’16]
¨ Numerous critical problems in science and engineering can be
Massively Parallel Boltzmann Machine
Approximate Heuristic Algorithms
Genetic Algorithms Ant Colony Optimization Semi-Definite Programming Simulated Annealing Tabu Search Communication Networks 10010 01 1 1001 Data Mining DNA Analysis Artificial Intelligence Pharmaceuticals
Combinatorial Optimization Problems Traveling Salesman Knapsack Scheduling Machine Learning Bin Packing
¨ Two-state units connected with real-valued edge weights form
¨ Goal: iteratively update the state or weight variables to
xj
The Boltzmann Machine
Σ x0 x3 w3,j w0,j
1 1 + eδ/C
Control Parameter
δ = (2xj-1) Σxiwi,j
¨ Network energy is minimized by adjusting either the edge
¨ Iterative matrix-vector multiplication between weights and
Data Movement Functional Units … … … Memory Arrays
w0,0 w0,1 … w1,0 … … x0 x1 … Σ, ×, 1 1 + ex
The Boltzmann Machine
¨ An RRAM cell comprises an access transistor and a resistive
RRAM Cell Wordline Bitline The Boltzmann Machine Functional Units … … … RRAM Arrays
V
RRAM: Resistive RAM (source: HP, 2009)
¨ A read is performed by activating a wordline and measuring
I = V/R1 V ‘1’
R1 The Boltzmann Machine Functional Units … … … RRAM Arrays
¨ Key Idea: exploit current summation on the RRAM bitlines to
‘1’ ‘1’ ‘1’ ‘1’ I =ΣV/Ri V
The Boltzmann Machine Functional Units … … … RRAM Arrays
¨ Memory cells represent the weights and state variables are
I =ΣV/Ri
w01 w02 w03 w04
I =ΣW0i V
X1 X2 X3 X4 X0
I =ΣX0XiW0i
The Boltzmann Machine Functional Units … … … RRAM Arrays
¨ Hierarchical organization with configurable reduction tree is
Mat Subbank H-Tree Bank Reduction Tree Controller Chip
Software configures the
initiates the optimization by writing to a memory mapped control register. To maintain ordering, accesses to the accelerator are made uncacheable by the processor. DDR3 reads and writes are used for configuration and data transfer.
Accelerator DIMM
Controller CPU
D R A M
Software configures the
initiates the optimization by writing to a memory mapped control register. To maintain ordering, accesses to the accelerator are made uncacheable by the processor. DDR3 reads and writes are used for configuration and data transfer.
Accelerator DIMM
Controller CPU
D R A M
Controller
Software configures the
initiates the optimization by writing to a memory mapped control register. To maintain ordering, accesses to the accelerator are made uncacheable by the processor. DDR3 reads and writes are used for configuration and data transfer.
Accelerator DIMM Model (m×n) n m
CPU
D R A M
Software configures the
initiates the optimization by writing to a memory mapped control register. To maintain ordering, accesses to the accelerator are made uncacheable by the processor. DDR3 reads and writes are used for configuration and data transfer.
Accelerator DIMM
Controller Start Ready CPU
D R A M
Software configures the
initiates the optimization by writing to a memory mapped control register. To maintain ordering, accesses to the accelerator are made uncacheable by the processor. DDR3 reads and writes are used for configuration and data transfer.
Accelerator DIMM
Controller Start Ready CPU
D R A M
0.01 0.1 1 0.01 0.1 1
Execution Time Normalized to the Single Threaded Kernel System Energy Normalized to the Single Threaded Baseline
Multi-threaded Kernel PIM Accelerator Memristive Accelerator