SLIDE 1
A Fully Associative Software-Managed Cache Design
Erik G. Hallnor and Steven K. Reindhardt
Presented By: Maryam Sadooghi-Alvandi
Motivation
- Two Trends:
- Growing DRAM access latency
- Multi-megabyte on-chip caches
A Fully Associative Software-Managed Cache Design Erik G. Hallnor - - PowerPoint PPT Presentation
A Fully Associative Software-Managed Cache Design Erik G. Hallnor - - PowerPoint PPT Presentation
A Fully Associative Software-Managed Cache Design Erik G. Hallnor and Steven K. Reindhardt Presented By: Maryam Sadooghi-Alvandi Motivation Two Trends: Growing DRAM access latency Multi-megabyte on-chip caches Motivation Two
SLIDE 2
SLIDE 3
Motivation
- Two Trends:
- Growing DRAM access latency
- Multi-megabyte on-chip caches
Re-examine how caches are
- rganized and managed
SLIDE 4
Motivation (II)
- Cache-DRAM relationship similar to
DRAM-disk storage relationship
SLIDE 5
Virtual Memory
- Two mechanisms used in Virtual Memory:
- Full Associativity
- Software Management
SLIDE 6
Virtual Memory
- Two mechanisms used in Virtual Memory:
- Full Associativity
- Software Management
Apply these to Caches
SLIDE 7
Goal
- A fully associative software-managed cache
SLIDE 8
The System
- Consists of 2 parts:
- The Indirect Index Cache (IIC)
- The Generational Replacement Algorithm
SLIDE 9
Potential Uses of A Software-Managed Cache
- More sophisticated replacement algorithms
- Reduced penalty for locking data
- Arbitrary partitioning of the data store
SLIDE 10
Conventional Caches
TAG TAG
TAG SET OFFSET
MATCH? MATCH?
SLIDE 11
Conventional Caches
TAG TAG
TAG SET OFFSET
MATCH? MATCH?
SLIDE 12
Conventional Caches
TAG TAG
TAG SET OFFSET
MATCH? MATCH?
SLIDE 13
Full Associativity
DATA ARRAY
SLIDE 14
Full Associativity
DATA ARRAY
Data block may be placed in any location
SLIDE 15
Full Associativity
DATA ARRAY
Data block may be placed in any location
SLIDE 16
Full Associativity
DATA ARRAY
Data block may be placed in any location
PC
SLIDE 17
Full Associativity
DATA ARRAY
Data block may be placed in any location
PC
INDEX
SLIDE 18
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
SLIDE 19
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
SLIDE 20
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
SLIDE 21
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
SLIDE 22
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
TAG STATUS INDEX REPL.
SLIDE 23
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
SLIDE 24
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
SLIDE 25
MATCH? MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE
INDEX
TE
SLIDE 26
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
SLIDE 27
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE TE
SLIDE 28
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE TE
SLIDE 29
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE TE
SLIDE 30
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE TE
SLIDE 31
INDEX
MATCH?
IIC
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE TE
MATCH?
SLIDE 32
MATCH?
IIC: Storage Overhead
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
TAG STATUS INDEX REPL.
TE TE
SLIDE 33
MATCH?
IIC: Storage Overhead
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
TAG STATUS INDEX REPL.
TE TE
1
SLIDE 34
MATCH?
IIC: Storage Overhead
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
TAG STATUS INDEX REPL.
TE TE
1 2
SLIDE 35
MATCH?
IIC: Storage Overhead
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
TAG STATUS INDEX REPL.
TE TE
1 2 3
SLIDE 36
MATCH?
IIC: Storage Overhead
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
TAG STATUS INDEX REPL.
TE TE
1 2 3 4 4
SLIDE 37
MATCH?
IIC: Storage Overhead
TAG
TAG OFFSET
HASH MATCH? MATCH? MATCH? MATCH?
TE TE TE TE
TAG STATUS INDEX REPL.
TE TE
1 2 3 4 4 5
SLIDE 38
IIC: Timing Overhead
- Accessing tag and data arrays sequentially
- Traversing the hash chain
- Overhead of software management
SLIDE 39
Generational Replacement: Motivation
L1 L2
SLIDE 40
Generational Replacement: Motivation
repeatedly referenced
L1 L2
SLIDE 41
Generational Replacement: Motivation
repeatedly referenced
L1 L2
L2 does not see all references
SLIDE 42
Generational Replacement: Motivation
- LRU looks at when block was last accessed
- GR emphasizes how many times a block is
accessed over when it was accessed
- Priority for staying given to blocks that are
repeatedly accessed
- even though they may not be the most recently
used
- Cope with long intervals between accesses
SLIDE 43
FRESH POOL
On A Hit ...
1
POOL 0 POOL 1 POOL 2 POOL 3
least frequently used most frequently used
SLIDE 44
FRESH POOL
On A Hit ...
1
POOL 0 POOL 1 POOL 2 POOL 3
HIT
least frequently used most frequently used
SLIDE 45
FRESH POOL
On A Hit ...
1
POOL 0 POOL 1 POOL 2 POOL 3
1
HIT
least frequently used most frequently used
SLIDE 46
FRESH POOL
On A Miss ...
1 1
POOL 0 POOL 1 POOL 2 POOL 3
least frequently used most frequently used
SLIDE 47
FRESH POOL
On A Miss ...
1 1
POOL 0 POOL 1 POOL 2 POOL 3
least frequently used most frequently used
SLIDE 48
FRESH POOL
On A Miss ...
1 1
POOL 0 POOL 1 POOL 2 POOL 3
least frequently used most frequently used
SLIDE 49
FRESH POOL
On A Miss ...
1 1
POOL 0 POOL 1 POOL 2 POOL 3
least frequently used most frequently used
SLIDE 50
FRESH POOL
On A Miss ...
1 1
POOL 0 POOL 1 POOL 2 POOL 3
least frequently used most frequently used
SLIDE 51
FRESH POOL
On A Miss ...
1 1
POOL 0 POOL 1 POOL 2 POOL 3
least frequently used most frequently used
SLIDE 52
FRESH POOL
On A Miss ...
1 1
POOL 0 POOL 1 POOL 2 POOL 3
least frequently used most frequently used
SLIDE 53
FRESH POOL
On A Miss ...
1 1
POOL 0 POOL 1 POOL 2 POOL 3
EVICT
SLIDE 54
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
SLIDE 55
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
HIT
1
SLIDE 56
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
HIT
1 1
SLIDE 57
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
1
SLIDE 58
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
1
MISS
2
SLIDE 59
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
1
MISS
2
SLIDE 60
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
1
MISS
2
SLIDE 61
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
1
MISS
2
SLIDE 62
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
1
SLIDE 63
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
1
HIT
3
SLIDE 64
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
HIT
3 3
SLIDE 65
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3
SLIDE 66
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3
HIT
4
SLIDE 67
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3
HIT
4
1
4
SLIDE 68
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3
1
4
SLIDE 69
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3
1
4
MISS
5
SLIDE 70
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3
1
4
MISS
5
SLIDE 71
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3
1
4
MISS
5
SLIDE 72
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3
1
4
MISS
5
SLIDE 73
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3 4
MISS
5
SLIDE 74
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3 4
MISS
5
SLIDE 75
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3 4
SLIDE 76
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3 4
MISS
6
SLIDE 77
EVICT
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
B3
3 4
MISS
6
SLIDE 78
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
3 4
MISS
6
SLIDE 79
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
3 4
MISS
6
SLIDE 80
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
3 4
MISS
6
SLIDE 81
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
3 4
SLIDE 82
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
3 4
MISS
7
SLIDE 83
EVICT
B2
GR vs. LRU
B1 1
POOL 0 POOL 1 POOL 2
3 4
MISS
7
SLIDE 84
GR: Timing Overhead
- Number of operations on a miss:
proportional to the number of priority queues
SLIDE 85
GR: Storage Overhead
- FIFO queues: doubly linked lists
- Two 12-bit pointers per block
- Head and tail pointers per queue
- Two full timestamps per queue
- 8-bit timestamps per block
SLIDE 86
Evaluation
- Windows NT system address traces on
Intel Architecture platform
- S/390 trace running OLTP workload
SLIDE 87
Simulations
- 64KB, 2-way, split L1 cache with 32B blocks
- 1MB L2 cache
SLIDE 88
Miss vs. Associativity
40,000 60,000 80,000 100,000 120,000 140,000 160,000 4 8 16 32 64 128 256 FA
LRU 128 LRU 256 LRU 512 OPT 128 OPT 256 OPT 512
BETTER
TPCC_LONG
SLIDE 89
Miss vs. Associativity
40,000 60,000 80,000 100,000 120,000 140,000 160,000 4 8 16 32 64 128 256 FA
LRU 128 LRU 256 LRU 512 OPT 128 OPT 256 OPT 512
BETTER
LRU
}
TPCC_LONG
SLIDE 90
Miss vs. Associativity
40,000 60,000 80,000 100,000 120,000 140,000 160,000 4 8 16 32 64 128 256 FA
LRU 128 LRU 256 LRU 512 OPT 128 OPT 256 OPT 512
BETTER
OPT
}
LRU
}
TPCC_LONG
SLIDE 91
Miss vs. Associativity
50,000,000 100,000,000 150,000,000 200,000,000 250,000,000 4 8 16 32 64 128 256 FA
LRU 128 LRU 256 LRU 512 OPT 128 OPT 256 OPT 512
BETTER
OLTP1W
SLIDE 92
Miss vs. Associativity
40,000 95,000 150,000 205,000 260,000 315,000 370,000 4 8 16 32 64 128 256 FA
LRU 128 LRU 256 LRU 512 OPT 128 OPT 256 OPT 512
BETTER
SPECWEB
SLIDE 93
Miss vs. Associativity
40,000 95,000 150,000 205,000 260,000 315,000 370,000 4 8 16 32 64 128 256 FA
LRU 128 LRU 256 LRU 512 OPT 128 OPT 256 OPT 512
BETTER
SPECWEB
LRU
}
SLIDE 94
Miss vs. Associativity
40,000 95,000 150,000 205,000 260,000 315,000 370,000 4 8 16 32 64 128 256 FA
LRU 128 LRU 256 LRU 512 OPT 128 OPT 256 OPT 512
BETTER
SPECWEB
OPT
}
LRU
}
SLIDE 95
Results (I)
BETTER
5 10 15 20 25 Millions of Misses L R U 4
- W
A Y + P G C O L O R + V I C T I M L R U 8
- W
A Y L R U 1 6
- W
A Y C L O C K F A G E N F A L R U F A O P T F A OLTP1W
SLIDE 96
Miss vs. Associativity
BETTER
5 10 15 20 25 Millions of Misses L R U 4
- W
A Y + P G C O L O R + V I C T I M L R U 8
- W
A Y L R U 1 6
- W
A Y C L O C K F A G E N F A L R U F A O P T F A OLTP1W
SLIDE 97
Results (II)
BETTER
50 100 150 200 250 300 350 400 Thousands of Misses L R U 4
- W
A Y + P G C O L O R + V I C T I M L R U 8
- W
A Y L R U 1 6
- W
A Y C L O C K F A G E N F A L R U F A O P T F A SPECWEB
SLIDE 98
Miss vs. Associativity
BETTER
50 100 150 200 250 300 350 400 Thousands of Misses L R U 4
- W
A Y + P G C O L O R + V I C T I M L R U 8
- W
A Y L R U 1 6
- W
A Y C L O C K F A G E N F A L R U F A O P T F A SPECWEB
SLIDE 99
Results (III)
BETTER
20 40 60 80 100 120 140 160 180 Thousands of Misses L R U 4
- W
A Y + P G C O L O R + V I C T I M L R U 8
- W
A Y L R U 1 6
- W
A Y C L O C K F A G E N F A L R U F A O P T F A TPCC_LONG
SLIDE 100
Thank you.
SLIDE 101
Generational Replacement: Motivation
- L1 hits filtered from L2’s observed
reference stream
- A block not referenced between 2 misses