Computer Systems Lecture 16 Caching Introduction CS 230 - Spring - - PowerPoint PPT Presentation

CS 230 - Spring 2020 3-1

CS 230 – Introduction to Computers and Computer Systems Lecture 16 – Caching Introduction

CS 230 - Spring 2020 3-2

MEM – Memory Access

 If instruction is lw or sw

 load from or store to memory

 How?

 accessing memory is slow  much slower than registers  how do we do the MEM stage quickly?

CS 230 - Spring 2020 3-3

Memory Hierarchy

 Going down the

hierarchy memory is

 cheaper  bigger  slower  further away from CPU

CS 230 - Spring 2020 3-4

Typical Memory Hierarchy

 Registers  Cache  Main memory  SSD/HDD  Network  Off-site archive

 (tape, optical disk, etc.)

CS 230 - Spring 2020 3-5

Registers

 Very expensive, thus limited  Access at instruction speed

 very low latency  very high throughput  can access in less than a cycle in ID and WB

 Not persistent

 lose values on power-off

CS 230 - Spring 2020 3-6

Main Memory

 Cheap and large

 several gigabytes

 Noticeable access latency

 can be approximately 100x slower than registers

 Not persistent  Random Access Memory (RAM)

 send address to memory controller on memory bus  memory controller responds with value

CS 230 - Spring 2020 3-7

Disk

 Hard Disk Drive (HDD) or Solid State Drive (SSD)  Very large storage

 hundreds of gigabytes to multiple terabytes

 Very long access latency

 thousands of times slower than main memory

 Persistent

CS 230 - Spring 2020 3-8

Memory Stalls

 How long does the MEM stage actually take?  Delay until response from memory controller

CS 230 - Spring 2020 3-9

Locality

 The set of data used during certain time period

is called the working set

 Assumption:

 size(working set) < size(all data/memory)  working set much smaller than available memory

 Example: books kept on your desk vs. books on

shelves in library (Dewey Decimal System)

CS 230 - Spring 2020 3-10

Locality Principle

 Temporal locality

 same data item likely to be used again soon  example: loop  library example continued: you’ll likely need the

same book again

 Spatial locality

 close data item likely to be used soon  example: iteration through array  library example continued: books beside ones you

are using are likely relevant

CS 230 - Spring 2020 3-11

Caching

 A cache is a small amount of fast memory used

to keep recently used (and nearby) data quickly accessible

 exploits the locality principle

 Basic challenges

 limited fast memory: replacement strategy  shared remote data: invalidation

 Data vs. Instruction cache

 in CS 230 we focus on data cache

CS 230 - Spring 2020 3-12

Terminology

 When a memory access (lw) happens the CPU

checks the cache first:

 hit = data found in the cache  miss = data not found, go get it from main memory

 Timing

 time if hit: hit time  time if missed: miss penalty

 Recall byte-addressable

 each byte has it’s own address

CS 230 - Spring 2020 3-13

Memory Caching

 Cache is organized into cache-blocks

 also called cache lines  multiple words/bytes per block (block size)  load from main memory to cache in blocks  library example continued: page vs. book

 How to know whether data is present?

 each block from main memory has an identity tag  library example continued: Dewey Decimal Index

 Where do we put newly loaded cache-blocks?

CS 230 - Spring 2020 3-14

Direct-Mapped Cache

 Assume a computer with M cache-blocks and a

block size of B bytes

 Want to load address P  Address P goes into cache-block C = (P/B) mod M

 Typically M and B are powers of 2  Discard remainder of P/B

 The tag of cache-block C will be T = P/(BM)

 Discard remainder

CS 230 - Spring 2020 3-15

Direct-Mapped Cache

 Consider a computer with a word size of 32-bits,

8 cache-blocks, and a block size of 4 words

 4 * 32-bits = 16 byte cache-block size (B)  M = 8

 Consider address P = 0x00000113

 P = 27510  C = (P/B)MOD M = (275/16)MOD 8 = 110  T = P/(BM) = 275/(16*8) = 210

CS 230 - Spring 2020 3-16

Direct-Mapped Cache Continued

 Conveniently we can also look directly at bits:

 example: M = 8, B = 16, 32-bit machine word  consider address P = 0x00000113

00000000000000000000000100010011 So address 0x113 goes in cache block 1 with tag 2

 Least significant 4 bits are within block since log216 = 4  Next 3 bits are cache-block number since log28 = 3  Remaining 25 bits are tag since 32 - log2(8 * 16) = 25

CS 230 - Spring 2020 3-17

Direct-Mapped Cache

Addresses in Memory (XXXX means any value) Blocks Numbers in Cache

XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX

CS 230 - Spring 2020 3-18

Cache Entries

 Many memory blocks map to same cache-block  Tag identifies which one is present  How do we tell if a cache-block actually has

data in it?

 add valid bit  1 (Yes) if data is loaded, 0 (No) if empty

 What if we load into already valid cache-block

 overwrite cache-block with new value  called an eviction

CS 230 - Spring 2020 3-19

Example Cache Diagram

 Assume M = 8, B = 2 bytes, 8-bit machine word

Index Valid Tag Data 000 N 001 N 010 N 011 N 100 N 101 N 110 N 111 N

CS 230 - Spring 2020 3-20

Example

Address Binary addr Hit/miss Cache block 4410 0010 110 0 Miss 110 Index Valid Tag Data 000 N 001 N 010 N 011 N 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N The block in memory containing all address 0010110X where X is any value

CS 230 - Spring 2020 3-21

Example

Address Binary addr Hit/miss Cache block 5310 0011 010 1 Miss 010 Index Valid Tag Data 000 N 001 N 010 Y 0011 Mem[0011010X] 011 N 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N

CS 230 - Spring 2020 3-22

Example

Address Binary addr Hit/miss Cache block 4510 0010 110 1 Hit 110 5210 0011 010 0 Hit 010 Index Valid Tag Data 000 N 001 N 010 Y 0011 Mem[0011010X] 011 N 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N 2 Hits! The blocks we were looking for were already in the cache.

CS 230 - Spring 2020 3-23

Example

Address Binary addr Hit/miss Cache block 3310 0010 000 1 Miss 000 610 0000 011 0 Miss 011 Index Valid Tag Data 000 Y 0010 Mem[0010000X] 001 N 010 Y 0011 Mem[0011010X] 011 Y 0000 Mem[0000011X] 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N

CS 230 - Spring 2020 3-24

Example

Address Binary addr Hit/miss Cache block 3310 0010 000 1 Hit 000 Index Valid Tag Data 000 Y 0010 Mem[0010000X] 001 N 010 Y 0011 Mem[0011010X] 011 Y 0000 Mem[0000011X] 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N

CS 230 - Spring 2020 3-25

Example

Address Binary addr Hit/miss Cache block 3710 0010 010 1 Miss 010 Index Valid Tag Data 000 Y 0010 Mem[0010000X] 001 N 010 Y 0010 Mem[0010010X] 011 Y 0000 Mem[0000011X] 100 N 101 N 110 Y 0010 Mem[0010110X] 111 N The block that was here before got evicted

CS 230 - Spring 2020 3-26

Hit Chance

 How many of the accesses were hits?

 previous example had 8 accesses: 3 hits, 5 misses

 3/8 = 0.37510 hit chance  5/8 = 0.62510 miss chance

CS 230 - Spring 2020 3-27

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

CS 230 - Spring 2020 3-28

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

Index Valid Tag Data 000 N 001 N 010 N 011 N 100 N 101 N 110 N 111 N

CS 230 - Spring 2020 3-29

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

Index Valid Tag Data 000 Y 100 Mem[100000XX] 001 N 010 N 011 N 100 N 101 N 110 N 111 N

miss

100 000 00

tag block xx

CS 230 - Spring 2020 3-30

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

Index Valid Tag Data 000 Y 100 Mem[100000XX] 001 N 010 N 011 N 100 N 101 N 110 N 111 Y 001 Mem[001111XX]

miss miss

001 111 10

tag block xx

CS 230 - Spring 2020 3-31

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

Index Valid Tag Data 000 Y 100 Mem[100000XX] 001 N 010 N 011 N 100 N 101 N 110 N 111 Y 001 Mem[001111XX]

miss miss hit

100 000 11

tag block xx

CS 230 - Spring 2020 3-32

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

Index Valid Tag Data 000 Y 100 Mem[100000XX] 001 N 010 N 011 N 100 N 101 N 110 N 111 Y 001 Mem[001111XX]

miss miss hit hit

001 111 00

tag block xx

CS 230 - Spring 2020 3-33

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

Index Valid Tag Data 000 Y 001 Mem[001000XX] 001 N 010 N 011 N 100 N 101 N 110 N 111 Y 001 Mem[001111XX]

miss miss hit hit miss+evic

001 000 10

tag block xx

CS 230 - Spring 2020 3-34

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

Index Valid Tag Data 000 Y 001 Mem[001000XX] 001 Y 001 Mem[001001XX] 010 N 011 N 100 N 101 N 110 N 111 Y 001 Mem[001111XX]

miss miss hit hit miss+evic miss

001 001 00

tag block xx

CS 230 - Spring 2020 3-35

Try it Yourself

Draw a cache diagram showing what the cache would look like after loading the following addresses from memory. What is the hit chance of this cache for this access sequence? Assume M = 8, B = 4 bytes, 8-bit machine word. 0x80, 0x3E, 0x83, 0x3C, 0x22, 0x24

miss miss hit hit miss+evic miss

6 accesses 2 hits 4 misses 2/6 = 0.33310 hit chance