SLIDE 1
PHI: ARCHITECTURAL SUPPORT FOR SYNCHRONIZATION- AND - - PowerPoint PPT Presentation
PHI: ARCHITECTURAL SUPPORT FOR SYNCHRONIZATION- AND - - PowerPoint PPT Presentation
PHI: ARCHITECTURAL SUPPORT FOR SYNCHRONIZATION- AND BANDWIDTH-EFFICIENT COMMUTATIVE SCATTER UPDATES Anurag Mukkara , Nathan Beckmann, Daniel Sanchez MICRO 2019 Scatter updates are common but inefficient 2 Scatter updates are common but
SLIDE 2
SLIDE 3
Scatter updates are common but inefficient
2
¨ Scatter updates are common in sparse algorithms
¤ e.g., in push graph algorithms, vertices scatter updates to
- utgoing neighbors
SLIDE 4
Scatter updates are common but inefficient
2
¨ Scatter updates are common in sparse algorithms
¤ e.g., in push graph algorithms, vertices scatter updates to
- utgoing neighbors
¨ Current memory hierarchies are optimized for reads
¤ Scatter updates suffer from high synchronization and high
memory bandwidth
SLIDE 5
Scatter updates are common but inefficient
2
¨ Scatter updates are common in sparse algorithms
¤ e.g., in push graph algorithms, vertices scatter updates to
- utgoing neighbors
¨ Current memory hierarchies are optimized for reads
¤ Scatter updates suffer from high synchronization and high
memory bandwidth
Cache Core Memory Fetch Fetch WB WB
SLIDE 6
Scatter updates are common but inefficient
2
¨ Scatter updates are common in sparse algorithms
¤ e.g., in push graph algorithms, vertices scatter updates to
- utgoing neighbors
¨ Current memory hierarchies are optimized for reads
¤ Scatter updates suffer from high synchronization and high
memory bandwidth
¨ Key insight: Many scatter updates are commutative and can
be reordered for performance
Cache Core Memory Fetch Fetch WB WB
SLIDE 7
Scatter updates are common but inefficient
2
¨ Scatter updates are common in sparse algorithms ¤ e.g., in push graph algorithms, vertices scatter updates to outgoing
neighbors
¨ Current memory hierarchies are optimized for reads ¤ Scatter updates suffer from high synchronization and high memory
bandwidth
¨ Key insight: Many scatter updates are commutative and can be
reordered for performance
¨ PHI extends the cache hierarchy to exploit temporal and spatial
locality of commutative scatter updates
Cache Core Memory Fetch Fetch WB WB
SLIDE 8
Scatter updates are common but inefficient
2
¨ Scatter updates are common in sparse algorithms ¤ e.g., in push graph algorithms, vertices scatter updates to outgoing
neighbors
¨ Current memory hierarchies are optimized for reads ¤ Scatter updates suffer from high synchronization and high memory
bandwidth
¨ Key insight: Many scatter updates are commutative and can be
reordered for performance
¨ PHI extends the cache hierarchy to exploit temporal and spatial
locality of commutative scatter updates
Cache Core Memory Fetch Fetch WB WB Cache Core Memory Push Push
SLIDE 9
Scatter updates are common but inefficient
2
¨ Scatter updates are common in sparse algorithms ¤ e.g., in push graph algorithms, vertices scatter updates to outgoing
neighbors
¨ Current memory hierarchies are optimized for reads ¤ Scatter updates suffer from high synchronization and high memory
bandwidth
¨ Key insight: Many scatter updates are commutative and can be
reordered for performance
¨ PHI extends the cache hierarchy to exploit temporal and spatial
locality of commutative scatter updates
Cache Core Memory Fetch Fetch WB WB Cache Core Memory Push Push Merge
SLIDE 10
PHI gives large benefits
3
SLIDE 11
PHI gives large benefits
3
¨ PageRank algorithm on UK web graph ¨ 16-core processor with 32MB cache, 4 memory controllers
SLIDE 12
PHI gives large benefits
3
Push Pull UB PHI
0.0 0.2 0.4 0.6 0.8 1.0
Memory traffic
3.5x
Memory traffic
¨ PageRank algorithm on UK web graph ¨ 16-core processor with 32MB cache, 4 memory controllers
SLIDE 13
PHI gives large benefits
3
Push Pull UB PHI
1 2 3 4 5 6 7
Speedup over Push Push Pull UB PHI
0.0 0.2 0.4 0.6 0.8 1.0
Memory traffic
3.5x
Performance Memory traffic
¨ PageRank algorithm on UK web graph ¨ 16-core processor with 32MB cache, 4 memory controllers
SLIDE 14
Agenda
4
¨ Background ¨ PHI Design ¨ Evaluation
SLIDE 15
Scatter updates are important
5
SLIDE 16
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses
SLIDE 17
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates
SLIDE 18
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src)
SLIDE 19
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates 1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src)
SLIDE 20
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates 1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src)
SLIDE 21
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates 1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src)
SLIDE 22
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates ¨ Pull mode: Indirect accesses are gather reads
1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src) for dst in vertices: for src in inNeighbors(dst): vertex(dst) += vertex(src)
SLIDE 23
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates ¨ Pull mode: Indirect accesses are gather reads
1 2 4 3 1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src) for dst in vertices: for src in inNeighbors(dst): vertex(dst) += vertex(src)
SLIDE 24
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates ¨ Pull mode: Indirect accesses are gather reads
1 2 4 3 1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src) for dst in vertices: for src in inNeighbors(dst): vertex(dst) += vertex(src)
SLIDE 25
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates ¨ Pull mode: Indirect accesses are gather reads
1 2 4 3 1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src) for dst in vertices: for src in inNeighbors(dst): vertex(dst) += vertex(src)
SLIDE 26
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates ¨ Pull mode: Indirect accesses are gather reads
1 2 4 3 1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src) for dst in vertices: for src in inNeighbors(dst): vertex(dst) += vertex(src)
SLIDE 27
Scatter updates are important
5
¨ Sparse algorithms perform push or pull-based indirect accesses ¨ Push mode: Indirect accesses are scatter updates ¨ Pull mode: Indirect accesses are gather reads ¨ Important to support scatter updates efficiently
¤ Push mode performs less work when few vertices are active
¤ Some algorithms do not admit a pull implementation
1 2 4 3 1 2 4 3
for src in vertices: for dst in outNeighbors(src): vertex(dst) += vertex(src) for dst in vertices: for src in inNeighbors(dst): vertex(dst) += vertex(src)
SLIDE 28
Scatter updates are inefficient on conventional hierarchies
6
SLIDE 29
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
SLIDE 30
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
SLIDE 31
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
1 2 4 3
SLIDE 32
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
1 2 4 3 Core Cache Shared Cache Memory Core Cache
… …
SLIDE 33
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
1 2 4 3 Core Cache Shared Cache Memory Core Cache
… …
2
SLIDE 34
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
1 2 4 3 Core Cache Shared Cache Memory Core Cache
… …
2
SLIDE 35
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
1 2 4 3 Core Cache Shared Cache Memory Core Cache
… …
2
SLIDE 36
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
1 2 4 3 Core Cache Shared Cache Memory Core Cache
… …
2
SLIDE 37
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
1 2 4 3
Push UB 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Memory requests per edge
Updates Updates Destination Vertex Source Vertex CSR
Core Cache Shared Cache Memory Core Cache
… …
2
Push PageRank on uk-2005 graph
SLIDE 38
Scatter updates are inefficient on conventional hierarchies
6
¨ Poor temporal and spatial locality when inputs do not fit in cache
¤Wasteful data transfers from main memory
¨ Multiple threads update the same vertex
¤Cache line ping-ponging
1 2 4 3
Push UB 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Memory requests per edge
93% of traffic due to scatter updates 10x more traffic than compulsory
Updates Updates Destination Vertex Source Vertex CSR
Core Cache Shared Cache Memory Core Cache
… …
2
Push PageRank on uk-2005 graph
SLIDE 39
Prior hardware support for scatter updates
7
SLIDE 40
Prior hardware support for scatter updates
7
¨ Remote memory operations (RMOs) send and perform update
- perations at a fixed location (e.g., shared cache banks)
SLIDE 41
Prior hardware support for scatter updates
7
¨ Remote memory operations (RMOs) send and perform update
- perations at a fixed location (e.g., shared cache banks)
¤Avoids cache-line ping ponging
SLIDE 42
Prior hardware support for scatter updates
7
¨ Remote memory operations (RMOs) send and perform update
- perations at a fixed location (e.g., shared cache banks)
¤Avoids cache-line ping ponging
¨ COUP [MICRO’15] modifies the coherence protocol to perform
commutative operations in a distributed fashion
SLIDE 43
Prior hardware support for scatter updates
7
¨ Remote memory operations (RMOs) send and perform update
- perations at a fixed location (e.g., shared cache banks)
¤Avoids cache-line ping ponging
¨ COUP [MICRO’15] modifies the coherence protocol to perform
commutative operations in a distributed fashion
¨ Both RMOs and COUP do not improve locality
SLIDE 44
Prior hardware support for scatter updates
7
¨ Remote memory operations (RMOs) send and perform update
- perations at a fixed location (e.g., shared cache banks)
¤Avoids cache-line ping ponging
¨ COUP [MICRO’15] modifies the coherence protocol to perform
commutative operations in a distributed fashion
¨ Both RMOs and COUP do not improve locality
¤Bottlenecked by memory traffic with large inputs
SLIDE 45
PHI builds on Update Batching (UB)
8
Propagation Blocking [IPDPS’17], MILK [PACT’16]
SLIDE 46
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution
Propagation Blocking [IPDPS’17], MILK [PACT’16]
SLIDE 47
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Propagation Blocking [IPDPS’17], MILK [PACT’16]
SLIDE 48
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Propagation Blocking [IPDPS’17], MILK [PACT’16]
SLIDE 49
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Propagation Blocking [IPDPS’17], MILK [PACT’16]
Destination Ids
11 5 9 0 7 4 6 8 3 12
SLIDE 50
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution ¨ Binning phase: Logs updates to memory, dividing them into cache-fitting
slices (bins) of vertices
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
- 1. Binning Phase
Propagation Blocking [IPDPS’17], MILK [PACT’16]
Destination Ids
11 5 9 0 7 4 6 8 3 12
SLIDE 51
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution ¨ Binning phase: Logs updates to memory, dividing them into cache-fitting
slices (bins) of vertices
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Bin 0
A B 5 A
…….
D 3
- 1. Binning Phase
Propagation Blocking [IPDPS’17], MILK [PACT’16]
Destination Ids
11 5 9 0 7 4 6 8 3 12
SLIDE 52
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution ¨ Binning phase: Logs updates to memory, dividing them into cache-fitting
slices (bins) of vertices
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Bin 0
A B 5 A
…….
D 3 A 9 B B 7
Bin 1
11
…….
D 12
- 1. Binning Phase
Propagation Blocking [IPDPS’17], MILK [PACT’16]
Destination Ids
11 5 9 0 7 4 6 8 3 12
SLIDE 53
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution ¨ Binning phase: Logs updates to memory, dividing them into cache-fitting
slices (bins) of vertices
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Bin 0
A B 5 A
…….
D 3 A 9 B B 7
Bin 1
11
…….
D 12
- 1. Binning Phase
Main memory
Propagation Blocking [IPDPS’17], MILK [PACT’16]
Destination Ids
11 5 9 0 7 4 6 8 3 12
SLIDE 54
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution ¨ Binning phase: Logs updates to memory, dividing them into cache-fitting
slices (bins) of vertices
¨ Accumulation phase: Reads and applies logged updates bin-by-bin
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Bin 0
A B 5 A
…….
D 3 A 9 B B 7
Bin 1
11
…….
D 12
- 1. Binning Phase
- 2. Accumulation Phase
Main memory
Propagation Blocking [IPDPS’17], MILK [PACT’16]
Destination Ids
11 5 9 0 7 4 6 8 3 12
SLIDE 55
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution ¨ Binning phase: Logs updates to memory, dividing them into cache-fitting
slices (bins) of vertices
¨ Accumulation phase: Reads and applies logged updates bin-by-bin
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Bin 0
A B 5 A
…….
D 3 A 9 B B 7
Bin 1
11
…….
D 12
- 1. Binning Phase
- 2. Accumulation Phase
Main memory
Propagation Blocking [IPDPS’17], MILK [PACT’16]
Destination Ids
11 5 9 0 7 4 6 8 3 12
SLIDE 56
PHI builds on Update Batching (UB)
8
¨ Maximizes spatial locality of memory transfers using two-phase execution ¨ Binning phase: Logs updates to memory, dividing them into cache-fitting
slices (bins) of vertices
¨ Accumulation phase: Reads and applies logged updates bin-by-bin
Cache fitting slice
8 16
. .
Destination Vertices
. .
Source Vertices
A B
. .
C D
Bin 0
A B 5 A
…….
D 3 A 9 B B 7
Bin 1
11
…….
D 12
- 1. Binning Phase
- 2. Accumulation Phase
Main memory
Propagation Blocking [IPDPS’17], MILK [PACT’16]
Destination Ids
11 5 9 0 7 4 6 8 3 12
SLIDE 57
Update Batching tradeoffs
9
SLIDE 58
Update Batching tradeoffs
9
¨ Perfect spatial locality for all main memory transfers
¤Compulsory memory traffic for all data structures
SLIDE 59
Update Batching tradeoffs
9
¨ Perfect spatial locality for all main memory transfers
¤Compulsory memory traffic for all data structures
¨ Binning phase ignores temporal locality
¤Generates large stream of updates even with structured inputs
SLIDE 60
Update Batching tradeoffs
9
Push UB PHI 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Memory requests per edge Updates Destination Vertex Source Vertex CSR
Unstructured input
¨ Perfect spatial locality for all main memory transfers
¤Compulsory memory traffic for all data structures
¨ Binning phase ignores temporal locality
¤Generates large stream of updates even with structured inputs
Push PageRank on uk-2005 graph
SLIDE 61
Update Batching tradeoffs
9
Push UB PHI 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Memory requests per edge Updates Destination Vertex Source Vertex CSR
Unstructured input
¨ Perfect spatial locality for all main memory transfers
¤Compulsory memory traffic for all data structures
¨ Binning phase ignores temporal locality
¤Generates large stream of updates even with structured inputs
Push PageRank on uk-2005 graph
SLIDE 62
Update Batching tradeoffs
9
Push UB PHI 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Memory requests per edge Updates Destination Vertex Source Vertex CSR
Unstructured input
¨ Perfect spatial locality for all main memory transfers
¤Compulsory memory traffic for all data structures
¨ Binning phase ignores temporal locality
¤Generates large stream of updates even with structured inputs
Push PageRank on uk-2005 graph
SLIDE 63
Update Batching tradeoffs
9
Push UB PHI 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Memory requests per edge Push UB PHI 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Memory requests per edge Updates Destination Vertex Source Vertex CSR
Unstructured input Structured input
¨ Perfect spatial locality for all main memory transfers
¤Compulsory memory traffic for all data structures
¨ Binning phase ignores temporal locality
¤Generates large stream of updates even with structured inputs
Push PageRank on uk-2005 graph
SLIDE 64
Agenda
10
¨ Background ¨ PHI Design ¨ Evaluation
SLIDE 65
Key techniques of PHI
11
SLIDE 66
Key techniques of PHI
11
¨ In-cache update buffering and coalescing
¤Exploits temporal locality
SLIDE 67
Key techniques of PHI
11
¨ In-cache update buffering and coalescing
¤Exploits temporal locality
¨ Selective update batching
¤Achieves high spatial locality
SLIDE 68
Key techniques of PHI
11
¨ In-cache update buffering and coalescing
¤Exploits temporal locality
¨ Selective update batching
¤Achieves high spatial locality
Bandwidth efficient
SLIDE 69
Key techniques of PHI
11
¨ In-cache update buffering and coalescing
¤Exploits temporal locality
¨ Selective update batching
¤Achieves high spatial locality
¨ Hierarchical buffering and coalescing
¤Enables update parallelism ¤Eliminates synchronization overheads
Bandwidth efficient Synchronization efficient
SLIDE 70
In-cache buffering and coalescing
12
SLIDE 71
In-cache buffering and coalescing
12
¨ Buffer updates in cache without ever
accessing main memory
SLIDE 72
In-cache buffering and coalescing
12
¨ Buffer updates in cache without ever
accessing main memory
¨ Treat cache as a large coalescing
buffer for updates
SLIDE 73
In-cache buffering and coalescing
12
¨ Buffer updates in cache without ever
accessing main memory
¨ Treat cache as a large coalescing
buffer for updates
¨ Reduction ALU in cache bank performs
coalescing
SLIDE 74
In-cache buffering and coalescing
12
Cache
¨ Buffer updates in cache without ever
accessing main memory
¨ Treat cache as a large coalescing
buffer for updates
¨ Reduction ALU in cache bank performs
coalescing
Core Memory
0xFOO
10
SLIDE 75
In-cache buffering and coalescing
12
UPDATE 0xFOO, +4
Cache
¨ Buffer updates in cache without ever
accessing main memory
¨ Treat cache as a large coalescing
buffer for updates
¨ Reduction ALU in cache bank performs
coalescing
Core Memory
0xFOO
10
SLIDE 76
In-cache buffering and coalescing
12
UPDATE 0xFOO, +4
Cache
¨ Buffer updates in cache without ever
accessing main memory
¨ Treat cache as a large coalescing
buffer for updates
¨ Reduction ALU in cache bank performs
coalescing
Core Memory
4
0xFOO 0xFOO
10
SLIDE 77
In-cache buffering and coalescing
12
Cache
¨ Buffer updates in cache without ever
accessing main memory
¨ Treat cache as a large coalescing
buffer for updates
¨ Reduction ALU in cache bank performs
coalescing
Core Memory
4
0xFOO 0xFOO
10
SLIDE 78
In-cache buffering and coalescing
12
Cache
¨ Buffer updates in cache without ever
accessing main memory
¨ Treat cache as a large coalescing
buffer for updates
¨ Reduction ALU in cache bank performs
coalescing
Core Memory
4
UPDATE 0xFOO, +2 0xFOO 0xFOO
10
SLIDE 79
In-cache buffering and coalescing
12
Cache
¨ Buffer updates in cache without ever
accessing main memory
¨ Treat cache as a large coalescing
buffer for updates
¨ Reduction ALU in cache bank performs
coalescing
Core Memory
4
UPDATE 0xFOO, +2
6
0xFOO 0xFOO
10
SLIDE 80
Handling cache evictions
13
SLIDE 81
Handling cache evictions
13
¨ PHI adapts to the amount of spatial locality in the evicted line
SLIDE 82
Handling cache evictions
13
¨ PHI adapts to the amount of spatial locality in the evicted line ¨ Cache controller performs update batching selectively
¤Achieves good spatial locality in all cases
SLIDE 83
Handling cache evictions
13
¨ PHI adapts to the amount of spatial locality in the evicted line ¨ Cache controller performs update batching selectively
¤Achieves good spatial locality in all cases
¨ Key insight: Update batching is a good tradeoff only when the
evicted line has poor spatial locality
SLIDE 84
Case 1: Evicted line has few updates
14
SLIDE 85
Case 1: Evicted line has few updates
14
¨ Log updates to temporary buffers (stored in cache)
SLIDE 86
Case 1: Evicted line has few updates
14
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
SLIDE 87
Case 1: Evicted line has few updates
14
Cache Memory
7
0xA4:
3
0xF8:
F00 4
0x10:
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
SLIDE 88
Case 1: Evicted line has few updates
14 Evict 0xA4
Cache Memory
7
0xA4:
3
0xF8:
F00 4
0x10:
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
SLIDE 89
Case 1: Evicted line has few updates
14 Evict 0xA4
Cache Memory
3
0xF8:
F00 4
0x10: INV
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
SLIDE 90
Case 1: Evicted line has few updates
14 Evict 0xA4
Cache Memory
3
0xF8:
F00 4 A48 7
0x10: INV
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
SLIDE 91
Case 1: Evicted line has few updates
14
Cache Memory
3
0xF8:
F00 4 A48 7
0x10: INV
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
SLIDE 92
Case 1: Evicted line has few updates
14
Cache Memory
3
0xF8:
F00 4 A48 7
0x10: INV
Evict 0xF8
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
SLIDE 93
Case 1: Evicted line has few updates
14
Cache Memory
3
0xF8:
F00 4 A48 7
0x10: INV
Evict 0xF8 Evict 0x10
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
SLIDE 94
Case 1: Evicted line has few updates
14
Cache Memory
3
0xF8:
F00 4 A48 7
0x10: INV
Evict 0xF8 Evict 0x10
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
SLIDE 95
Case 1: Evicted line has few updates
14
Cache Memory
3
0xF8:
F00 4 A48 7
0x10: INV
Evict 0xF8 Evict 0x10
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
F00 4 A48 7
0x10:
SLIDE 96
Case 1: Evicted line has few updates
14
Cache Memory
3
0xF8: INV
Evict 0xF8
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
INV
F00 4 A48 7
0x10:
SLIDE 97
Case 1: Evicted line has few updates
14
Cache Memory
INV INV
¨ Log updates to temporary buffers (stored in cache) ¨ These buffers are later evicted to memory when full
INV
4
0xF0:
Invalid line Buffered-updates line
F00 4 A48 7
0x10:
Line with batched updates
F84 3
0x11:
F00 4 A48 7
0x10:
SLIDE 98
Case 1: Evicted line has many valid updates
15
___
SLIDE 99
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
___
SLIDE 100
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
Cache Memory
4 6 3
0xF0:
___ 5 6 1 8
0xBC:
7 9 2
0xDF: INV
4
0xF0:
Invalid line Buffered-updates line
1 2 1 7
0xF0:
SLIDE 101
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
Cache Memory
4 6 3
0xF0:
Evict 0xF0
___ 5 6 1 8
0xBC:
7 9 2
0xDF: INV
4
0xF0:
Invalid line Buffered-updates line
1 2 1 7
0xF0:
SLIDE 102
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
Cache Memory
4 6 3
0xF0:
Evict 0xF0
___ 5 6 1 8
0xBC:
7 9 2
0xDF: INV
4
0xF0:
Invalid line Buffered-updates line
1 2 1 7
0xF0:
SLIDE 103
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
Cache Memory
4 6 3
0xF0:
Evict 0xF0
___ 5 6 1 8
0xBC:
7 9 2
0xDF: INV
4
0xF0:
Invalid line Buffered-updates line
1 2 1 7
0xF0:
SLIDE 104
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
Cache Memory
4 6 3
0xF0:
Evict 0xF0
___ 5 6 1 8
0xBC:
7 9 2
0xDF: INV
4
0xF0:
Invalid line Buffered-updates line
1 2 1 7
0xF0:
MERGE
SLIDE 105
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
Cache Memory
4 6 3
0xF0:
Evict 0xF0
___ 5 6 1 8
0xBC:
7 9 2
0xDF: INV
4
0xF0:
Invalid line Buffered-updates line
1 2 1 7
0xF0:
MERGE
SLIDE 106
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
Cache Memory
4 6 3
0xF0:
Evict 0xF0
___ 5 6 1 8
0xBC:
7 9 2
0xDF: INV
4
0xF0:
Invalid line Buffered-updates line
5 8 4 7
0xF0:
MERGE
SLIDE 107
Case 1: Evicted line has many valid updates
15
¨ Fetch line from main memory and merge updates
Cache Memory
Evict 0xF0
___
INV
5 6 1 8
0xBC:
7 9 2
0xDF: INV
4
0xF0:
Invalid line Buffered-updates line
5 8 4 7
0xF0:
MERGE
SLIDE 108
PHI avoids synchronization costs
16
SLIDE 109
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol
SLIDE 110
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1 Memory
SLIDE 111
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1 Update 0xF04, +4 Memory
SLIDE 112
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
Memory
SLIDE 113
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
Update 0xF08, +3 Memory
SLIDE 114
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 0 3
0xF0:
Memory
SLIDE 115
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 0 3
0xF0:
Update 0xB5C, +2 Memory
SLIDE 116
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 0 3
0xF0:
Evict 0xF0 Update 0xB5C, +2 Memory
SLIDE 117
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 0 3
0xF0:
0 0 3
0xF0:
Evict 0xF0 Update 0xB5C, +2 Memory
SLIDE 118
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 0 3
0xF0:
Update 0xB5C, +2 Memory
SLIDE 119
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 0 3
0xF0:
0 0 0 2
0xB5:
Memory
SLIDE 120
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 0 3
0xF0:
0 0 0 2
0xB5:
Update 0xA00, +1 Memory
SLIDE 121
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 0 3
0xF0:
0 0 0 2
0xB5:
Update 0xA00, +1 Evict 0xF0 Memory
SLIDE 122
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
4
0xF0:
0 4 3
0xF0:
0 0 0 2
0xB5:
Update 0xA00, +1 Evict 0xF0 Memory
SLIDE 123
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
0 4 3
0xF0:
0 0 0 2
0xB5:
Update 0xA00, +1 Memory
SLIDE 124
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
1 0 0
0xA0:
0 4 3
0xF0:
0 0 0 2
0xB5:
Memory
SLIDE 125
PHI avoids synchronization costs
16
¨ Private caches buffer and coalesce
updates locally, push them to shared cache on evictions
¤No need for a coherence protocol
¨ Private caches do not perform update
batching
¤Simply evict buffered-update lines to
shared cache
Core 1 Private Cache 0 Shared Cache Core 0 Private Cache 1
1 0 0
0xA0:
0 4 3
0xF0:
0 0 0 2
0xB5:
Memory
SLIDE 126
PHI has minimal hardware costs
17
SLIDE 127
PHI has minimal hardware costs
17
¨ Per-line buffered updates bit
¤0.17% additional storage with 64-byte lines
SLIDE 128
PHI has minimal hardware costs
17
¨ Per-line buffered updates bit
¤0.17% additional storage with 64-byte lines
¨ Reduction unit for each cache bank
¤Supports 64-bit floating-point and integer additions, logical operations ¤0.06% of chip area in a 16-core system (0.09mm2 in 45 nm)
SLIDE 129
Agenda
18
¨ Background ¨ PHI Design ¨ Evaluation
SLIDE 130
Evaluation methodology
19
SLIDE 131
Evaluation methodology
19
¨ Event-driven simulation using ZSim
SLIDE 132
Evaluation methodology
19
¨ Event-driven simulation using ZSim ¨ 16-core processor
¤Haswell-like OOO cores ¤32 MB L3 cache ¤4 memory controllers
……
Core0
Private Cache 0
Memory
Shared Cache
Core15
Private Cache 15
SLIDE 133
Evaluation methodology
19
¨ Event-driven simulation using ZSim ¨ 16-core processor
¤Haswell-like OOO cores ¤32 MB L3 cache ¤4 memory controllers
¨ Graph applications
¤ PageRank, PageRank Delta,
Connected Components, Radii Estimation
¨ Degree Counting (No Pull)
¨ SpMV
……
Core0
Private Cache 0
Memory
Shared Cache
Core15
Private Cache 15
SLIDE 134
Evaluation methodology
19
¨ Event-driven simulation using ZSim ¨ 16-core processor
¤Haswell-like OOO cores ¤32 MB L3 cache ¤4 memory controllers
¨ Graph applications
¤ PageRank, PageRank Delta,
Connected Components, Radii Estimation
¨ Degree Counting (No Pull)
¨ SpMV ¨ Large real world inputs
¤Up to 100 million vertices ¤Up to 1 billion edges
……
Core0
Private Cache 0
Memory
Shared Cache
Core15
Private Cache 15
SLIDE 135
PHI improves performance significantly
20
SLIDE 136
PHI improves performance significantly
20
Push Pull UB Push-RMO PHI
SLIDE 137
PHI improves performance significantly
20
Push Pull UB Push-RMO PHI
SLIDE 138
PHI improves performance significantly
20
¨ Pull and UB show mixed results
Push Pull UB Push-RMO PHI
SLIDE 139
PHI improves performance significantly
20
¨ Pull and UB show mixed results ¨ Push-RMO improves performance by avoiding synchronization costs
Push Pull UB Push-RMO PHI
SLIDE 140
PHI improves performance significantly
20
¨ Pull and UB show mixed results ¨ Push-RMO improves performance by avoiding synchronization costs ¨ PHI consistently outperforms other schemes
Push Pull UB Push-RMO PHI
SLIDE 141
PHI reduces memory traffic
21
SLIDE 142
PHI reduces memory traffic
21
Push Pull UB PHI
SLIDE 143
PHI reduces memory traffic
21
Push Pull UB PHI
SLIDE 144
PHI reduces memory traffic
21
¨ Pull incurs higher memory traffic for non-all-active algorithms (CC, RE)
Push Pull UB PHI
SLIDE 145
PHI reduces memory traffic
21
¨ Pull incurs higher memory traffic for non-all-active algorithms (CC, RE) ¨ UB increases memory traffic when input has good locality
Push Pull UB PHI
SLIDE 146
PHI reduces memory traffic
21
¨ Pull incurs higher memory traffic for non-all-active algorithms (CC, RE) ¨ UB increases memory traffic when input has good locality ¨ PHI reduces memory traffic over UB by exploiting temporal locality
Push Pull UB PHI
SLIDE 147
Conclusion
22
SLIDE 148
Conclusion
22
¨ Scatter updates are inefficient on conventional hierarchies
SLIDE 149
Conclusion
22
¨ Scatter updates are inefficient on conventional hierarchies ¨ PHI extends the cache hierarchy to make commutative scatter updates
efficient
SLIDE 150
Conclusion
22
¨ Scatter updates are inefficient on conventional hierarchies ¨ PHI extends the cache hierarchy to make commutative scatter updates
efficient
¨ Exploits both temporal and spatial locality
SLIDE 151
Conclusion
22
¨ Scatter updates are inefficient on conventional hierarchies ¨ PHI extends the cache hierarchy to make commutative scatter updates
efficient
¨ Exploits both temporal and spatial locality ¨ Incurs low memory traffic and minimal synchronization
SLIDE 152
Conclusion
22
¨ Scatter updates are inefficient on conventional hierarchies ¨ PHI extends the cache hierarchy to make commutative scatter updates
efficient
¨ Exploits both temporal and spatial locality ¨ Incurs low memory traffic and minimal synchronization