cache memory
play

Cache Memory Chapter 17 S. Dandamudi Outline Introduction - PowerPoint PPT Presentation

Mar 18, 2023 •236 likes •802 views

Cache Memory Chapter 17 S. Dandamudi Outline Introduction Types of cache misses How cache memory works Types of caches Why cache memory works Example implementations Pentium Cache design basics PowerPC

  1. Cache Memory Chapter 17 S. Dandamudi

  2. Outline • Introduction • Types of cache misses • How cache memory works • Types of caches • Why cache memory works • Example implementations ∗ Pentium • Cache design basics ∗ PowerPC • Mapping function ∗ MIPS ∗ Direct mapping • Cache operation summary ∗ Associative mapping • Design issues ∗ Set-associative mapping ∗ Cache capacity • Replacement policies ∗ Cache line size • Write policies ∗ Degree of associatively • Space overhead 2003  S. Dandamudi Chapter 17: Page 2 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  3. Introduction • Memory hierarchy ∗ Registers ∗ Memory ∗ Disk ∗ … • Cache memory is a small amount of fast memory ∗ Placed between two levels of memory hierarchy » To bridge the gap in access times – Between processor and main memory (our focus) – Between main memory and disk (disk cache) ∗ Expected to behave like a large amount of fast memory 2003  S. Dandamudi Chapter 17: Page 3 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  4. Introduction (cont’d) 2003  S. Dandamudi Chapter 17: Page 4 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  5. How Cache Memory Works • Prefetch data into cache before the processor needs it ∗ Need to predict processor future access requirements » Not difficult owing to locality of reference • Important terms ∗ Miss penalty ∗ Hit ratio ∗ Miss ratio = (1 – hit ratio) ∗ Hit time 2003  S. Dandamudi Chapter 17: Page 5 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  6. How Cache Memory Works (cont’d) Cache read operation 2003  S. Dandamudi Chapter 17: Page 6 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  7. How Cache Memory Works (cont’d) Cache write operation 2003  S. Dandamudi Chapter 17: Page 7 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  8. Why Cache Memory Works • Example for (i=0; i<M; i++) for(j=0; j<N; j++) X[i][j] = X[i][j] + K; ∗ Each element of X is double (eight bytes) ∗ Loop is executed (M * N) times » Placing the code in cache avoids access to main memory – Repetitive use (one of the factors) – Temporal locality » Prefetching data – Spatial locality 2003  S. Dandamudi Chapter 17: Page 8 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  9. How Cache Memory Works (cont’d) 300 250 Execution time (ms Column-order 200 150 100 Row-order 50 0 500 600 700 800 900 1000 Matrix size 2003  S. Dandamudi Chapter 17: Page 9 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  10. Cache Design Basics • On every read miss ∗ A fixed number of bytes are transferred » More than what the processor needs – Effective due to spatial locality • Cache is divided into blocks of B bytes » b -bits are needed as offset into the block b = log 2 B » Block are called cache lines • Main memory is also divided into blocks of same size ∗ Address is divided into two parts 2003  S. Dandamudi Chapter 17: Page 10 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  11. Cache Design Basics (cont’d) B = 4 bytes b = 2 bits 2003  S. Dandamudi Chapter 17: Page 11 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  12. Cache Design Basics (cont’d) • Transfer between main memory and cache ∗ In units of blocks ∗ Implements spatial locality • Transfer between main memory and cache ∗ In units of words • Need policies for ∗ Block placement ∗ Mapping function ∗ Block replacement ∗ Write policies 2003  S. Dandamudi Chapter 17: Page 12 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  13. Cache Design Basics (cont’d) Read cycle operations 2003  S. Dandamudi Chapter 17: Page 13 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  14. Mapping Function • Determines how memory blocks are mapped to cache lines • Three types ∗ Direct mapping » Specifies a single cache line for each memory block ∗ Set-associative mapping » Specifies a set of cache lines for each memory block ∗ Associative mapping » No restrictions – Any cache line can be used for any memory block 2003  S. Dandamudi Chapter 17: Page 14 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  15. Mapping Function (cont’d) Direct mapping example 2003  S. Dandamudi Chapter 17: Page 15 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  16. Mapping Function (cont’d) • Implementing direct mapping ∗ Easier than the other two ∗ Maintains three pieces of information » Cache data – Actual data » Cache tag – Problem: More memory blocks than cache lines � Several memory blocks are mapped to a cache line – Tag stores the address of memory block in cache line » Valid bit – Indicates if cache line contains a valid block 2003  S. Dandamudi Chapter 17: Page 16 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  17. Mapping Function (cont’d) 2003  S. Dandamudi Chapter 17: Page 17 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  18. Mapping Function (cont’d) Direct mapping Reference pattern: 0, 4, 0, 8, 0, 8, 0, 4, 0, 4, 0, 4 Hit ratio = 0% 2003  S. Dandamudi Chapter 17: Page 18 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  19. Mapping Function (cont’d) Direct mapping Reference pattern: 0, 7, 9, 10, 0, 7, 9, 10, 0, 7, 9, 10 Hit ratio = 67% 2003  S. Dandamudi Chapter 17: Page 19 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  20. Mapping Function (cont’d) Associative mapping 2003  S. Dandamudi Chapter 17: Page 20 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  21. Mapping Function (cont’d) Associative mapping Reference pattern: 0, 4, 0, 8, 0, 8, 0, 4, 0, 4, 0, 4 Hit ratio = 75% 2003  S. Dandamudi Chapter 17: Page 21 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  22. Mapping Function (cont’d) Address match logic for associative mapping 2003  S. Dandamudi Chapter 17: Page 22 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  23. Mapping Function (cont’d) Associative cache with address match logic 2003  S. Dandamudi Chapter 17: Page 23 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  24. Mapping Function (cont’d) Set-associative mapping 2003  S. Dandamudi Chapter 17: Page 24 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  25. Mapping Function (cont’d) Address partition in set-associative mapping 2003  S. Dandamudi Chapter 17: Page 25 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  26. Mapping Function (cont’d) Set-associative mapping Reference pattern: 0, 4, 0, 8, 0, 8, 0, 4, 0, 4, 0, 4 Hit ratio = 67% 2003  S. Dandamudi Chapter 17: Page 26 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  27. Replacement Policies • We invoke the replacement policy ∗ When there is no place in cache to load the memory block • Depends on the actual placement policy in effect ∗ Direct mapping does not need a special replacement policy » Replace the mapped cache line ∗ Several policies for the other two mapping functions » Popular: LRU (least recently used) » Random replacement » Less interest (FIFO, LFU) 2003  S. Dandamudi Chapter 17: Page 27 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  28. Replacement Policies (cont’d) • LRU ∗ Expensive to implement » Particularly for set sizes more than four • Implementations resort to approximation ∗ Pseudo-LRU » Partitions sets into two groups – Maintains the group that has been accessed recently – Requires only one bit » Requires only ( W -1) bits ( W = degree of associativity) – PowerPC is an example � Details later 2003  S. Dandamudi Chapter 17: Page 28 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

  29. Replacement Policies (cont’d) Pseudo-LRU implementation 2003  S. Dandamudi Chapter 17: Page 29 To be used with S. Dandamudi, “Fundamentals of Computer Organization and Design,” Springer, 2003.

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache

What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache

What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache Performance Proc/Regs L1-Cache Bigger Faster L2-Cache Memory hierarchy concept, cache L3-Cache (optional) design fundamentals, set-associative

303 views • 4 slides
Chapter 4 Cache Memory Contents Computer memory system overview Characteristics of

Chapter 4 Cache Memory Contents Computer memory system overview Characteristics of

Chapter 4 Cache Memory Contents Computer memory system overview Characteristics of memory systems Memory hierarchy Cache memory principles Elements of cache design Cache size Mapping function Replacement

793 views • 64 slides
General Cache Mechanics CPU Block: unit of data in cache and memory. (a.k.a. line) Memory

General Cache Mechanics CPU Block: unit of data in cache and memory. (a.k.a. line) Memory

General Cache Mechanics CPU Block: unit of data in cache and memory. (a.k.a. line) Memory Hierarchy: Cache Smaller, faster, more expensive. Cache 8 9 14 3 Stores subset of memory blocks . (lines) Data is moved in block units Memory

467 views • 8 slides
Lecture 23: Cache, Memory, Virtual Memory Todays topics: Cache examples, caching

Lecture 23: Cache, Memory, Virtual Memory Todays topics: Cache examples, caching

Lecture 23: Cache, Memory, Virtual Memory Todays topics: Cache examples, caching policies Main memory system Virtual memory 1 Example 1 32 KB 4-way set-associative data cache array with 32 byte line sizes How many

436 views • 19 slides
Cache Memory Chapter 17 S. Dandamudi Outline Introduction Types of cache misses

Cache Memory Chapter 17 S. Dandamudi Outline Introduction Types of cache misses

Cache Memory Chapter 17 S. Dandamudi Outline Introduction Types of cache misses How cache memory works Types of caches Why cache memory works Example implementations Cache design basics Pentium PowerPC

813 views • 28 slides
Memory Hierarchy: Cache Memory hierarchy Cache basics Locality Cache organization Cache-aware

Memory Hierarchy: Cache Memory hierarchy Cache basics Locality Cache organization Cache-aware

Memory Hierarchy: Cache Memory hierarchy Cache basics Locality Cache organization Cache-aware programming How does execution time grow with SIZE? int[] array = new int[SIZE]; fillArrayRandomly(array); int s = 0; for (int i = 0; i &lt;

847 views • 48 slides
Cache Systems CPU Main Main CPU Memory Memory 400MHz 10MHz Cache 10MHz Memory Hierarchy

Cache Systems CPU Main Main CPU Memory Memory 400MHz 10MHz Cache 10MHz Memory Hierarchy

Cache Systems CPU Main Main CPU Memory Memory 400MHz 10MHz Cache 10MHz Memory Hierarchy Design Bus 66MHz Bus 66MHz Chapter 5 and Appendix C Data object Block transfer transfer Main CPU Cache Memory 1 4 Example: Two-level

353 views • 10 slides
1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

Today Cache memory organization and operation Performance impact of caches Cache Memories The memory mountain Rearranging loops to improve spatial locality Using blocking to improve temporal locality CSci 2021: Machine

102 views • 8 slides
Tutorial 9 : cache memory Why use a cache ? Main memory (VRAM/DRAM) is slow ! To deal with

Tutorial 9 : cache memory Why use a cache ? Main memory (VRAM/DRAM) is slow ! To deal with

Tutorial 9 : cache memory Why use a cache ? Main memory (VRAM/DRAM) is slow ! To deal with this, the -machine speed is reduced to match the memory read and write speed To make the machine faster, one can use a intermediate smaller

219 views • 8 slides
Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory

Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory

Cache Management Improving Memory Locality and Reducing Memory Latency Introduction Memory system performance is critical in modern architectures DO I = 1, M Accessing memory takes much longer than DO J = 1, N accessing cache

364 views • 19 slides
Virtual Memory 1 Memory Hierarchy Memory 4GB Cache 1M Registers 1K Question: What if

Virtual Memory 1 Memory Hierarchy Memory 4GB Cache 1M Registers 1K Question: What if

Virtual Memory 1 Memory Hierarchy Memory 4GB Cache 1M Registers 1K Question: What if we want to run a process that requires 10GB memory? 2 Memory Hierarchy Virtual Memory Memory Cache Registers Answer: Pretend we had something

737 views • 45 slides
1 Implementation Snoop Caches Implementing Snooping Caches Write Races: Multiple processors

1 Implementation Snoop Caches Implementing Snooping Caches Write Races: Multiple processors

An Example Snoopy Protocol Invalidation protocol, write-back cache Each block of memory is in one state: Cache Coherence and Memory Clean in all caches and up-to-date in memory (Shared) Consistency OR Dirty in exactly one cache

507 views • 3 slides
P age 1 Review: Cache perf ormance What are all the aspects of cache organization that impact

P age 1 Review: Cache perf ormance What are all the aspects of cache organization that impact

Who Cares About the Memory Hierarchy? CS252 Graduate Computer Architecture Proc 1000 Lecture 4 CPU 60%/yr. Moores Law Performance CPU- DRAM Gap Cache Design 100 Processor-Memory Performance Gap: (grows 50% / year) 10

380 views • 9 slides
LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to

LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to

LECTURE 12 Virtual Memory VIRTUAL MEMORY Just as a cache can provide fast, easy access to recently-used code and data, main memory acts as a cache for magnetic disk. The mechanism by which this is accomplished is known as virtual memory

974 views • 34 slides
1 Direct-mapped cache Tags and validation stickers Address (showing bit positions) 31 30 13 12

1 Direct-mapped cache Tags and validation stickers Address (showing bit positions) 31 30 13 12

Basic structure of memory hierarchy Exploiting memory hierarchy Speed Size Cost ($/bits) Technology Cache basics CPU &lt; ns 32 high CPU registers 0.5-5 ns KB-MB $4K-$10K SRAM Cache per GB 50-70 ns MB-GB $100-$200 DRAM Main

190 views • 8 slides
Lecture 24: Cache, Memory, Security Todays topics: Caching policies Main memory

Lecture 24: Cache, Memory, Security Todays topics: Caching policies Main memory

Lecture 24: Cache, Memory, Security Todays topics: Caching policies Main memory system Hardware security intro 1 Cache Misses On a write miss, you may either choose to bring the block into the cache (write-allocate) or

146 views • 10 slides
CSE 502: Computer Architecture Memory Hierarchy &amp; Caches Motivation 10000 Performance

CSE 502: Computer Architecture Memory Hierarchy &amp; Caches Motivation 10000 Performance

CSE 502: Computer Architecture Memory Hierarchy &amp; Caches Motivation 10000 Performance 1000 Processor 100 10 Memory 1 1985 1990 1995 2000 2005 2010 Want memory to appear: As fast as CPU As large as required by all of

587 views • 39 slides
Chapter 5 Large and Fast: Exploiting Memory Hierarchy 5.1 Introduction Principle of Locality

Chapter 5 Large and Fast: Exploiting Memory Hierarchy 5.1 Introduction Principle of Locality

COMPUTER ORGANIZATION AND DESIGN 5 th Edition The Hardware/Software Interface Chapter 5 Large and Fast: Exploiting Memory Hierarchy 5.1 Introduction Principle of Locality Programs access a small proportion of their address space at any

1.13k views • 85 slides
Caching 3 1 last time tag / index / ofgset lookup in associative caches replacement policies

Caching 3 1 last time tag / index / ofgset lookup in associative caches replacement policies

Caching 3 1 last time tag / index / ofgset lookup in associative caches replacement policies least recently used best miss rate assuming locality random simplest to implement write policies: write-through versus write-back

1.26k views • 72 slides
$$$ $$$ Cache Memory 2 $$$ 2 Schedule This week

$$$ $$$ Cache Memory 2 $$$ 2 Schedule This week

Computer Systems and Networks ECPE 170 Jeff Shafer University of the Pacific $$$ $$$ Cache Memory 2 $$$ 2 Schedule This week

599 views • 59 slides
Low-Level Memory Optimisations at the High-Level with Ownership-like Annotations Do you want

Low-Level Memory Optimisations at the High-Level with Ownership-like Annotations Do you want

Juliana Franco Martin Hagelin Tobias Wrigstad Sophia Drossopoulou The OHMM framework Low-Level Memory Optimisations at the High-Level with Ownership-like Annotations Do you want fast programs? More cores? More threads? Write better

330 views • 29 slides
CPUs Chapter 3.5 Caches. Memory management. Caches and CPUs address data cache

CPUs Chapter 3.5 Caches. Memory management. Caches and CPUs address data cache

CPUs Chapter 3.5 Caches. Memory management. Caches and CPUs address data cache controller cache main CPU memory address data data ARM Cortex-A9 Configurations ARM Cortex A9 Microarchitecture Main System Memory ARM Cortex-A9

839 views • 34 slides
Memory Virtualization: Swapping and Demand Paging Policies 1 University of New Mexico Beyond

Memory Virtualization: Swapping and Demand Paging Policies 1 University of New Mexico Beyond

University of New Mexico Memory Virtualization: Swapping and Demand Paging Policies 1 University of New Mexico Beyond Physical Memory: Policies Memory pressure forces the OS to start paging out pages to make room for actively-used pages.

510 views • 24 slides
Lecture 12: Memory hierarchy &amp; caches A modern memory subsystem combines fast small

Lecture 12: Memory hierarchy &amp; caches A modern memory subsystem combines fast small

Lecture 12: Memory hierarchy &amp; caches A modern memory subsystem combines fast small memory, slower larger memories This lecture looks at why and how Focus today mostly on electronic memories. Next lecture looks at supplementing

522 views • 17 slides

More recommend