Computer Systems Lecture 16 Caching Introduction CS 230 - Spring - PowerPoint PPT Presentation

CS 230 – Introduction to Computers and Computer Systems Lecture 16 – Caching Introduction CS 230 - Spring 2020 3-1

MEM – Memory Access  If instruction is lw or sw  load from or store to memory  How?  accessing memory is slow  much slower than registers  how do we do the MEM stage quickly? CS 230 - Spring 2020 3-2

Memory Hierarchy  Going down the hierarchy memory is  cheaper  bigger  slower  further away from CPU CS 230 - Spring 2020 3-3

Typical Memory Hierarchy  Registers  Cache  Main memory  SSD/HDD  Network  Off-site archive  (tape, optical disk, etc.) CS 230 - Spring 2020 3-4

Registers  Very expensive, thus limited  Access at instruction speed  very low latency  very high throughput  can access in less than a cycle in ID and WB  Not persistent  lose values on power-off CS 230 - Spring 2020 3-5

Main Memory  Cheap and large  several gigabytes  Noticeable access latency  can be approximately 100x slower than registers  Not persistent  Random Access Memory (RAM)  send address to memory controller on memory bus  memory controller responds with value CS 230 - Spring 2020 3-6

Disk  Hard Disk Drive (HDD) or Solid State Drive (SSD)  Very large storage  hundreds of gigabytes to multiple terabytes  Very long access latency  thousands of times slower than main memory  Persistent CS 230 - Spring 2020 3-7

Memory Stalls  How long does the MEM stage actually take?  Delay until response from memory controller CS 230 - Spring 2020 3-8

Locality  The set of data used during certain time period is called the working set  Assumption:  size(working set) < size(all data/memory)  working set much smaller than available memory  Example: books kept on your desk vs. books on shelves in library (Dewey Decimal System) CS 230 - Spring 2020 3-9

Locality Principle  Temporal locality  same data item likely to be used again soon  example: loop  library example continued: you’ll likely need the same book again  Spatial locality  close data item likely to be used soon  example: iteration through array  library example continued: books beside ones you are using are likely relevant CS 230 - Spring 2020 3-10

Caching  A cache is a small amount of fast memory used to keep recently used (and nearby) data quickly accessible  exploits the locality principle  Basic challenges  limited fast memory: replacement strategy  shared remote data: invalidation  Data vs. Instruction cache  in CS 230 we focus on data cache CS 230 - Spring 2020 3-11

Terminology  When a memory access ( lw ) happens the CPU checks the cache first:  hit = data found in the cache  miss = data not found, go get it from main memory  Timing  time if hit: hit time  time if missed: miss penalty  Recall byte-addressable  each byte has it’s own address CS 230 - Spring 2020 3-12

Memory Caching  Cache is organized into cache-blocks  also called cache lines  multiple words/bytes per block ( block size )  load from main memory to cache in blocks  library example continued: page vs. book  How to know whether data is present?  each block from main memory has an identity tag  library example continued: Dewey Decimal Index  Where do we put newly loaded cache-blocks? CS 230 - Spring 2020 3-13

Direct-Mapped Cache  Assume a computer with M cache-blocks and a block size of B bytes  Want to load address P  Address P goes into cache-block C = ( P / B ) mod M  Typically M and B are powers of 2  Discard remainder of P / B  The tag of cache-block C will be T = P /( BM )  Discard remainder CS 230 - Spring 2020 3-14

Direct-Mapped Cache  Consider a computer with a word size of 32 -bits, 8 cache-blocks, and a block size of 4 words  4 * 32 -bits = 16 byte cache-block size ( B )  M = 8  Consider address P = 0x00000113  P = 275 10  C = (P/B)MOD M = (275/16)MOD 8 = 1 10  T = P/(BM) = 275/(16*8) = 2 10 CS 230 - Spring 2020 3-15

Direct-Mapped Cache Continued  Conveniently we can also look directly at bits:  example: M = 8 , B = 16 , 32 -bit machine word  consider address P = 0x00000113 00000000000000000000000100010011 So address 0x113 goes in cache block 1 with tag 2  Least significant 4 bits are within block since log 2 16 = 4  Next 3 bits are cache-block number since log 2 8 = 3  Remaining 25 bits are tag since 32 - log 2 ( 8 * 16 ) = 25 CS 230 - Spring 2020 3-16

Direct-Mapped Cache Blocks Numbers in Cache XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX Addresses in Memory (XXXX means any value) CS 230 - Spring 2020 3-17

Cache Entries  Many memory blocks map to same cache-block  Tag identifies which one is present  How do we tell if a cache-block actually has data in it?  add valid bit  1 ( Y es) if data is loaded, 0 ( N o) if empty  What if we load into already valid cache-block  overwrite cache-block with new value  called an eviction CS 230 - Spring 2020 3-18

Example Cache Diagram  Assume M = 8, B = 2 bytes, 8-bit machine word Index Valid Tag Data 000 N 001 N 010 N 011 N 100 N 101 N 110 N 111 N CS 230 - Spring 2020 3-19

Example Address Binary addr Hit/miss Cache block 44 10 0010 110 0 Miss 110 Index Valid Tag Data 000 N 001 N 010 N The block in 011 N memory 100 N containing all address 0010110X 101 N where X is any 110 Y 0010 Mem[0010110X] value 111 N CS 230 - Spring 2020 3-20

Example Address Binary addr Hit/miss Cache block 53 10 0011 010 1 Miss 010 Index Valid Tag Data 000 N 001 N 010 Y 0011 Mem[0011010X] 011 N 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N CS 230 - Spring 2020 3-21

Example Address Binary addr Hit/miss Cache block 45 10 0010 110 1 Hit 110 52 10 0011 010 0 Hit 010 Index Valid Tag Data 000 N 001 N 2 Hits! The blocks 010 Y 0011 Mem[0011010X] we were looking 011 N for were already in 100 N the cache. 101 N 110 Y 0010 Mem[0010110X] 111 N CS 230 - Spring 2020 3-22

Example Address Binary addr Hit/miss Cache block 33 10 0010 000 1 Miss 000 6 10 0000 011 0 Miss 011 Index Valid Tag Data 000 Y 0010 Mem[0010000X] 001 N 010 Y 0011 Mem[0011010X] 011 Y 0000 Mem[0000011X] 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N CS 230 - Spring 2020 3-23

Example Address Binary addr Hit/miss Cache block 33 10 0010 000 1 Hit 000 Index Valid Tag Data 000 Y 0010 Mem[0010000X] 001 N 010 Y 0011 Mem[0011010X] 011 Y 0000 Mem[0000011X] 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N CS 230 - Spring 2020 3-24

Example Address Binary addr Hit/miss Cache block 37 10 0010 010 1 Miss 010 Index Valid Tag Data 000 Y 0010 Mem[0010000X] 001 N The block that was here before got 010 Y 0010 Mem[0010010X] evicted 011 Y 0000 Mem[0000011X] 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N CS 230 - Spring 2020 3-25

Hit Chance  How many of the accesses were hits?  previous example had 8 accesses: 3 hits, 5 misses  3/8 = 0.375 10 hit chance  5/8 = 0.625 10 miss chance CS 230 - Spring 2020 3-26

Try it Yourself Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24 CS 230 - Spring 2020 3-27

Try it Yourself Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24 Index Valid Tag Data 000 N 001 N 010 N 011 N 100 N 101 N 110 N 111 N CS 230 - Spring 2020 3-28

Try it Yourself Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80 , 0x3E, 0x83, 0x3C, 0x22, 0x24 miss 100 000 00 tag block xx Index Valid Tag Data 000 Y 100 Mem[100000XX] 001 N 010 N 011 N 100 N 101 N 110 N 111 N CS 230 - Spring 2020 3-29

Try it Yourself Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E , 0x83, 0x3C, 0x22, 0x24 miss miss 001 111 10 tag block xx Index Valid Tag Data 000 Y 100 Mem[100000XX] 001 N 010 N 011 N 100 N 101 N 110 N 111 Y 001 Mem[001111XX] CS 230 - Spring 2020 3-30

Try it Yourself Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83 , 0x3C, 0x22, 0x24 miss miss hit 100 000 11 tag block xx Index Valid Tag Data 000 Y 100 Mem[100000XX] 001 N 010 N 011 N 100 N 101 N 110 N 111 Y 001 Mem[001111XX] CS 230 - Spring 2020 3-31