SIGNET: NETWORK-ON-CHIP FILTERING FOR COARSE VECTOR DIRECTORIES - - PowerPoint PPT Presentation
SIGNET: NETWORK-ON-CHIP FILTERING FOR COARSE VECTOR DIRECTORIES Natalie Enright Jerger University of Toronto Interaction of Coherence and Network 2 Cache coherence protocol drives network-on-chip traffic Scalable coherence protocols
Natalie Enright Jerger DATE 2010
Cache coherence protocol drives network-on-chip
Scalable coherence protocols needed for many-core
Consider interconnection network optimizations to
2
Natalie Enright Jerger DATE 2010
Introduction Network-on-Chip Challenges with Scalable Coherence
SigNet Architecture: Network filtering solution Evaluation Conclusion
3
Natalie Enright Jerger DATE 2010
4
Natalie Enright Jerger DATE 2010
Broadcast
Good latency Poor scaling due to
4
Natalie Enright Jerger DATE 2010
Broadcast
Good latency Poor scaling due to
4
Natalie Enright Jerger DATE 2010
Broadcast
Good latency Poor scaling due to
4
Directory
Good scalability due to point to
Storage overheads
Natalie Enright Jerger DATE 2010
Broadcast
Good latency Poor scaling due to
4
Store Miss
Directory
Good scalability due to point to
Storage overheads
Natalie Enright Jerger DATE 2010
Broadcast
Good latency Poor scaling due to
4
Store Miss ForwardX
Directory
Good scalability due to point to
Storage overheads
Natalie Enright Jerger DATE 2010
Broadcast
Good latency Poor scaling due to
4
Store Miss ForwardX Response
Directory
Good scalability due to point to
Storage overheads
Natalie Enright Jerger DATE 2010
5
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
5
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
5
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
5
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
5
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
5
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
5
Cores 1, 2, 5 & 15 share cache line
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 3rd sharer: overflow 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 3rd sharer: overflow 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 3rd sharer: overflow 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 3rd sharer: overflow 2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 3rd sharer: overflow
3 sharers but 6 invalidations
2 pointers
Natalie Enright Jerger DATE 2010
Directory protocol storage
Single bit per core in sharing
256 cores ➔ 32 Bytes of
Coarse Vector Directories
DiriCVr
i: # of pointers r: # of cores in region
Example: Dir2CV2
Requires 1/2 as much storage
5
Cores 1, 2, 5 & 15 share cache line Only 2 sharers: 1 & 15 3rd sharer: overflow
3 sharers but 6 invalidations
2 pointers
Natalie Enright Jerger DATE 2010
For many applications: number of sharers is small
2 to 3
When number of sharers exceeds i, pointers overflow
Directory entry operates in coarse mode
1 bit represents multiple cores Imprecise sharing list
Extra processors will receive and acknowledge invalidation Consumes network power Requires additional cache lookups
6
Natalie Enright Jerger DATE 2010
Increase in coarseness leads to significant contention
7
Natalie Enright Jerger DATE 2010
Increased load results in decreased effectiveness of
Increased dynamic power consumption
8
Natalie Enright Jerger DATE 2010
Coarse vectors increase
Increase completion time
All acknowledgments must
Delay subsequent requests
9
Inval Complete Local Remote
SPECjbb SPECweb TPC-H TPC-W Barnes Ocean Radiosity Raytrace
Natalie Enright Jerger DATE 2010
Coarseness reduces directory storage
But with significant potential impact on network
Due to extraneous invalidations Safely remove extraneous invalidations
Save power and reduce network contention
Place cache summary information in routers
Counting Bloom filters used for cache summary signatures Use summary information to filter network packets
10
Natalie Enright Jerger DATE 2010
Counting Bloom filters summarize cache information in
Signature of cache contents
Bloom Filter Hit
Core exists between current node and destination that is
Bloom Filter Miss
None of the downstream caches are caching lines that map
11
Natalie Enright Jerger DATE 2010
Bloom Filter Insertion
Cache misses increment counter as they travel to
Bloom Filter Deletion
Writeback and invalidation acknowledgments
12
Natalie Enright Jerger DATE 2010
13 !"#$%& '"()#$*+",&
234$05& .//"0*$"1& '1"667*1&634$05&
8,)#$&9& 8,)#$&:& ;#$)#$&9& ;#$)#$&:& <%0"=%&
24>&?& 24>&2& 24>&@& 24>&A&
8,)#$&7#B%16&
.0C&D%66*>%& E"1(*+",&
8,)#$&7#B%16&
Natalie Enright Jerger DATE 2010
Header flits traverse modified pipeline
If the packet needs to check/update signature
14
!"# $%# &'()*'+,-.#
/0# ,0#
3'4*#5-6# ,-.#3-6# 0(7#89.# :);<#
/0# ,0#
,-.#8-99#
BW RC VA SA ST LT
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
Requests follow X-Y path Responses follow Y-X path
15
1 Cache miss to A Send request to directory 2 At each hop, insert address into input port signature 3 Directory operating in coarse mode Bit for region set Directory responds to request
Natalie Enright Jerger DATE 2010
16
Natalie Enright Jerger DATE 2010
16
4 Write request for A
Natalie Enright Jerger DATE 2010
16
4 Write request for A
Natalie Enright Jerger DATE 2010
16
4 Write request for A
Natalie Enright Jerger DATE 2010
16
4 Write request for A 5 Directory forwards invalidation requests
Natalie Enright Jerger DATE 2010
16
4 Write request for A 5 Directory forwards invalidation requests
Natalie Enright Jerger DATE 2010
16
4 Write request for A 5 Directory forwards invalidation requests 6 North Port Filter Hit West Port Filter Miss
Natalie Enright Jerger DATE 2010
16
4 Write request for A 5 Directory forwards invalidation requests 6 North Port Filter Hit West Port Filter Miss 7 Spawn 2 Acks
Natalie Enright Jerger DATE 2010
16
4 Write request for A 5 Directory forwards invalidation requests 6 North Port Filter Hit West Port Filter Miss 7 Spawn 2 Acks
Natalie Enright Jerger DATE 2010
16
4 Write request for A 5 Directory forwards invalidation requests 6 North Port Filter Hit West Port Filter Miss 7 Spawn 2 Acks
Natalie Enright Jerger DATE 2010
16
4 Write request for A 5 Directory forwards invalidation requests 6 North Port Filter Hit West Port Filter Miss 7 Spawn 2 Acks
Natalie Enright Jerger DATE 2010
16
4 Write request for A 5 Directory forwards invalidation requests 6 North Port Filter Hit West Port Filter Miss 7 Spawn 2 Acks 8 Invalidations reach 2 destination cores and acks sent to directory
Natalie Enright Jerger DATE 2010
Recall: Full-map directory requires 32 bytes per
50% overhead per cache line
Evaluation uses 8K entry Bloom filters at each output
Reduces overhead to 12.5% to 25% per cache line
Depends on size of counters and number of pointers in coarse
17
Natalie Enright Jerger DATE 2010
All cores caching a block must receive invalidation
Bloom filter can have false positives
Lessens performance benefit but correct Cannot have false negatives
Differences in utilization due to location and memory
18
Natalie Enright Jerger DATE 2010
19
Natalie Enright Jerger DATE 2010
Create synthetic benchmarks based on characteristics of
20
Natalie Enright Jerger DATE 2010
Filters effectively reduce network contention
Lower average packet latency
Additional research needed to further close gap with
21
Normalized Average Packet Latency
Natalie Enright Jerger DATE 2010
SigNet improves invalidation completion time Comparison with Pruning Caches in paper
22
Cycles for Invalidations to Complete
Natalie Enright Jerger DATE 2010
Interconnection Network Support
Pruning Caches In-network coherence filters
Cache Coherence Optimizations with Bloom filters
Jetty filters: reduce cache snoops Tagless Coherence Directories
Reduce storage overheads
23
Natalie Enright Jerger DATE 2010
Characterize impact of CV directories
Significant power consumption and performance
Interconnect support to facilitate scalable cache
SigNet: network filters to reduce extraneous
24
Natalie Enright Jerger DATE 2010
Questions?
25