Memory Hierarchy and Direct Map Caches Lecture 11 CDA 3103 - - PowerPoint PPT Presentation
Memory Hierarchy and Direct Map Caches Lecture 11 CDA 3103 06-25-2014 5.1 Introduction Principle of Locality Programs access a small proportion of their address space at any time Temporal locality Items accessed recently are
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 2
Programs access a small proportion of
Temporal locality
Items accessed recently are likely to be
e.g., instructions in a loop, induction variables
Spatial locality
Items near those accessed recently are likely
E.g., sequential instruction access, array data
§5.1 Introduction
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 3
Memory hierarchy Store everything on disk Copy recently accessed (and nearby)
Main memory
Copy more recently accessed (and
Cache memory attached to CPU
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 4
Block (aka line): unit of copying
May be multiple words
If accessed data is present in
Hit: access satisfied by upper level
Hit ratio: hits/accesses
If accessed data is absent
Miss: block copied from lower level
Time taken: miss penalty Miss ratio: misses/accesses
= 1 – hit ratio
Then accessed data supplied from
upper level
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 5
Static RAM (SRAM)
0.5ns – 2.5ns, $2000 – $5000 per GB
Dynamic RAM (DRAM)
50ns – 70ns, $20 – $75 per GB
Magnetic disk
5ms – 20ms, $0.20 – $2 per GB
Ideal memory
Access time of SRAM Capacity and cost/GB of disk
§5.2 Memory Technologies
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 6
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 7
Data stored as a charge in a capacitor
Single transistor used to access the charge Must periodically be refreshed
Read contents and write back Performed on a DRAM “row”
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 8
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 9
Bits in a DRAM are organized as a
DRAM accesses an entire row Burst mode: supply successive words from a
Double data rate (DDR) DRAM
Transfer on rising and falling clock edges
Quad data rate (QDR) DRAM
Separate DDR inputs and outputs
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 10
50 100 150 200 250 300 '80 '83 '85 '89 '92 '96 '98 '00 '04 '07 Trac Tcac Year Capacity $/GB 1980 64Kbit $1500000 1983 256Kbit $500000 1985 1Mbit $200000 1989 4Mbit $50000 1992 16Mbit $15000 1996 64Mbit $10000 1998 128Mbit $4000 2000 256Mbit $1000 2004 512Mbit $250 2007 1Gbit $50
Row buffer
Allows several words to be read and refreshed in
Synchronous DRAM
Allows for consecutive accesses in bursts without
Improves bandwidth
DRAM banking (DDR3 etc)
Allows simultaneous access to multiple DRAMs Improves bandwidth DIMM (Dual inline memory modules [4-16 DRAM’s])
A DIMM using DDR4-3200 SDRAM can transfer at 8 x 3200
= 25600 megabytes per second
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 12
4-word wide memory
Miss penalty = 1 + 15 + 1 = 17 bus cycles Bandwidth = 16 bytes / 17 cycles = 0.94 B/cycle
4-bank interleaved memory
Miss penalty = 1 + 15 + 4×1 = 20 bus cycles Bandwidth = 16 bytes / 20 cycles = 0.8 B/cycle
Chapter 6 — Storage and Other I/O Topics — 13
Nonvolatile semiconductor storage
100× – 1000× faster than disk Smaller, lower power, more robust But more $/GB (between disk and DRAM)
§6.4 Flash Storage
Chapter 6 — Storage and Other I/O Topics — 14
NOR flash: bit cell like a NOR gate
Random read/write access Used for instruction memory in embedded systems
NAND flash: bit cell like a NAND gate
Denser (bits/area), but block-at-a-time access Cheaper per GB Used for USB keys, media storage, …
Flash bits wears out after 1000’s of accesses
Not suitable for direct RAM or disk replacement Wear leveling: remap data to less used blocks
Chapter 6 — Storage and Other I/O Topics — 15
Nonvolatile, rotating magnetic storage
§6.3 Disk Storage
Chapter 6 — Storage and Other I/O Topics — 16
Each sector records
Sector ID Data (512 bytes, 4096 bytes proposed) Error correcting code (ECC)
Used to hide defects and recording errors
Synchronization fields and gaps
Access to a sector involves
Queuing delay if other accesses are pending Seek: move the heads Rotational latency Data transfer Controller overhead
Chapter 6 — Storage and Other I/O Topics — 17
Given
512B sector, 15,000rpm, 4ms average seek
Average read time
4ms seek time
If actual average seek time is 1ms
Average read time = 3.2ms
Chapter 6 — Storage and Other I/O Topics — 18
Manufacturers quote average seek time
Based on all possible seeks Locality and OS scheduling lead to smaller actual
Smart disk controller allocate physical sectors on
Present logical sector interface to host SCSI, ATA, SATA
Disk drives include caches
Prefetch sectors in anticipation of access Avoid seek and rotational delay
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 19
Cache memory
The level of the memory hierarchy closest to
Given accesses X1, …, Xn–1, Xn
§5.3 The Basics of Caches
How do we know if
Where do we look?
Dr Dan Garcia
Processor (active) Control (“brain”) Datapath (“brawn”) Computer Memory (passive) (where programs, data live when running) Devices Input Output Keyboard, Mouse Display, Printer Disk, Network
Dr Dan Garcia
Dr Dan Garcia
holds data in register file (~100 Bytes) Registers accessed on nanosecond timescale
More capacity than registers (~Gbytes) Access time ~50-100 ns Hundreds of clock cycles per memory access?!
HUGE capacity (virtually limitless) VER
1 10 100 1000 10000 Performance
Year
DRAM 7%/year (2X/10yrs)
Processor-Memory Performance Gap (grows 50%/year)
µProc 55%/year (2X/1.5yr)
1989 first Intel CPU with cache on chip 1998 Pentium III has two cache levels on chip
Dr Dan Garcia
Dr Dan Garcia
Increasing distance from the processor in access time L1$ L2$ Main Memory
1,024+ bytes (disk sector = page)
Secondary Memory (Relative) size of the memory at each level
Dr Dan Garcia
Processor
4-8 bytes (word)
Inclusive– what is in L1$ is a subset of what is in L2$ is a subset of what is in MM that is a subset of is in SM
1 to 4 blocks 8-32 bytes (block)
Second Level Cache (SRAM)
Datapath On-Chip Components Control
Secondary Memory (Disk Or Flash)
RegFile Main Memory (DRAM) Data Cache Instr Cach e ITLB DTLB
Dr Dan Garcia
Speed (#cycles): ½’s 1’s 10K’s 10’s M’s 100’s G’s 10,000’s T’s lowest Size (bytes): Cost: 100’s highest
Dr Dan Garcia
Smaller Faster More expensive subset of lower levels (contains most recently used
Dr Dan Garcia
essentially limitless capacity
, very slow to retrieve a book
able is main memory
smaller capacity: means you must return book when table fills up easier and faster to find a book there once you’ve already
retrieved it
smaller capacity: can have very few open books fit on table; again,
when table fills up, you must close a book
much, much faster to retrieve data
Keep as many recently used books open on table as possible
since likely to use again
Also keep as many books on table as possible, since faster than
going to library
Dr Dan Garcia
T
Spatial Locality: if we use a piece of memory
Dr Dan Garcia
If a memory location is referenced then it will tend
If a memory location is referenced, the locations
Dr Dan Garcia
(Remember that cache is subset of memory
By compiler (or assembly level programmer)
By the cache controller hardware
By the operating system (virtual memory) Virtual to physical address mapping assisted by
By the programmer (files)
Dr Dan Garcia
Dr Dan Garcia
Therefore, we only need to look in a single
Block is the unit of transfer between cache and
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 34
Location determined by address Direct mapped: only one choice
(Block address) modulo (#Blocks in cache)
#Blocks is a
Use low-order
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 35
How do we know which particular block is
Store block address as well as the data Actually, only need the high-order bits Called the tag
What if there is no data in a location?
Valid bit: 1 = present, 0 = not present Initially 0
Memory location 0, 4, 8, ... 4 blocks
Memory Address
finds out the right block, and loads it all!
How does it know right block? How do we select the byte?
1 101?
block it originated from?
Memory Address
What do you do at baggage claim?
Dr Dan Garcia
Do we need the entire address? What do all these tags have in common? What did we do with the immediate when we were branch addressing, always count by bytes?
It’suseful to draw memory with the
MemoryAddress
(addresses shown)
same width as the block size
Cache#
Dr Dan Garcia
1
2
3
Dr Dan Garcia
Dr Dan Garcia
specifies the cache index (which “row”/block of
once we’ve found correct block, specifies which
the remaining bits after offset and index are
WIDTH (size of one block, B/block) HEIGHT (# of blocks)
Dr Dan Garcia
T ag Index Offset
Dr Dan Garcia
Sound familiar?
need to specify correct byte within a block block contains 2 bytes
= 21bytes
need 1bit to specify correct byte
need to specify correct block in cache cache contains 8 B= 23 bytes block contains 2 B= 21bytes # blocks/cache
= bytes/cache bytes/block = 23 bytes/cache 21bytes/block = 22 blocks/cache
need 2 bits to specify this many blocks
Dr Dan Garcia
Dr Dan Garcia
tag length = addr length – offset - index
so tag is leftmost 29 bits of memory address T
All bytes within block need same address (4b) Index must be same for every address within a
A.
B.
ABC 1: FFF 1: FFT 2: FTF 2: 3: 3: 4: 5: FTT TFF TFT TTF TTT
C.
Dr Dan Garcia
B.
ABC 1: FFF 1: FFT 2: FTF 2: 3: 3: 4: 5: FTT TFF TFT TTF TTT
C.
Dr Dan Garcia
Dr Dan Garcia
each successively lower level contains “most used”
exploits temporal & spatial locality do the common case fast, worry less about the
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 48
8-blocks, 1 word/block, direct mapped Initial state
Index V Tag Data 000 N 001 N 010 N 011 N 100 N 101 N 110 N 111 N
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 49
Index V Tag Data 000 N 001 N 010 N 011 N 100 N 101 N 110 Y 10 Mem[10110] 111 N Word addr Binary addr Hit/miss Cache block 22 10 110 Miss 110
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 50
Index V Tag Data 000 N 001 N 010 Y 11 Mem[11010] 011 N 100 N 101 N 110 Y 10 Mem[10110] 111 N Word addr Binary addr Hit/miss Cache block 26 11 010 Miss 010
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 51
Index V Tag Data 000 N 001 N 010 Y 11 Mem[11010] 011 N 100 N 101 N 110 Y 10 Mem[10110] 111 N Word addr Binary addr Hit/miss Cache block 22 10 110 Hit 110 26 11 010 Hit 010
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 52
Index V Tag Data 000 Y 10 Mem[10000] 001 N 010 Y 11 Mem[11010] 011 Y 00 Mem[00011] 100 N 101 N 110 Y 10 Mem[10110] 111 N Word addr Binary addr Hit/miss Cache block 16 10 000 Miss 000 3 00 011 Miss 011 16 10 000 Hit 000
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 53
Index V Tag Data 000 Y 10 Mem[10000] 001 N 010 Y 10 Mem[10010] 011 Y 00 Mem[00011] 100 N 101 N 110 Y 10 Mem[10110] 111 N Word addr Binary addr Hit/miss Cache block 18 10 010 Miss 010
Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 54
Dr Dan Garcia
Dr Dan Garcia
1.
2.
processor
Memory
I. Memory reads 99 at address 1022ten II. Memory sends 99 to Cache III. Cache replaces word with new 99 IV. Cache sends 99 to processor
3.
Dr Dan Garcia
Dr Dan Garcia
Dr Dan Garcia
Dr Dan Garcia
Dr Dan Garcia
Hit, Miss, Miss w. replace Values returned: a ,b, c, d, e, ..., k, l
000000000000000000 0000000011 0000
000000000000000000 0000000001 1100
Dr Dan Garcia
0x00000030 = e 0x0000001c = d
Dr Dan Garcia
Dr Dan Garcia
Dr Dan Garcia
address: tag index