cache memories
play

Cache Memories Lecture, Oct. 30, 2018 1 Bryant and OHallaron, - PowerPoint PPT Presentation

Oct 12, 2022 •185 likes •949 views

Cache Memories Lecture, Oct. 30, 2018 1 Bryant and OHallaron, Computer Systems: A Programmers Perspective, Third Edition General Cache Concept Smaller, faster, more expensive Cache 8 4 9 14 10 3 memory caches a subset of the

  1. Cache Memories Lecture, Oct. 30, 2018 1 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  2. General Cache Concept Smaller, faster, more expensive Cache 8 4 9 14 10 3 memory caches a subset of the blocks Data is copied in block-sized 10 4 transfer units Larger, slower, cheaper memory Memory 0 1 2 3 viewed as partitioned into “blocks” 4 4 5 6 7 8 9 10 10 11 12 13 14 15 2 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  3. 3 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  4. 4 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  5. 5 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  6. Structure Representation r struct rec { int a[4]; size_t i; a i next struct rec *next; 24 32 0 16 }; 6 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  7. I[0].A I[0].B I[0].BV[0] I[0].B[1] I[1].A I[1].B I[1].BV[0] I[1].B[1] 7 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  8. I[0].A I[0].B I[0].BV[0] I[0].B[1] Each block I[1].A I[1].B I[1].BV[0] I[1].B[1] associated the first half of the array has a unique spot in memory I[2].A I[2].B I[2].BV[0] I[2].B[1] I[3].A I[3].B I[3].BV[0] I[3].B[1] 2^9 8 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  9. Cache Optimization Techniques for (j = 0; j < 3: j = j+1){ for (i = 0; i < 3: i = i+1){ for( i = 0; i < 3; i = i + 1){ for( j = 0; j < 3; j = j + 1){ x[i][j] = 2*x[i][j]; x[i][j] = 2*x[i][j]; } } } } Inner loop analysis These two loops compute the same result Array in row major order X[0][0] X[0][1] X[0][2] X[1][0] X[1][1] X[1][2] X[2][0] X[2][1] X[2][2] 0x0 – 0x3 0x4 - 0x7 0x8-0x11 0x12 – 0x15 0x16 - 0x19 0x20-0x23 X[0][0] X[0][1] X[0][2] X[1][0] X[1][1] X[1][2] X[2][0] X[2][1] X[2][2] 9 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  10. Cache Optimization Techniques for (j = 0; j < 3: j = j+1){ for (i = 0; i < 3: i = i+1){ for( i = 0; i < 3; i = i + 1){ for( j = 0; j < 3; j = j + 1){ x[i][j] = 2*x[i][j]; x[i][j] = 2*x[i][j]; } } } } These two loops compute the same result int *x = malloc(N*N); Array in row major order for (i = 0; i < 3: i = i+1){ for( j = 0; j < 3; j = j + 1){ x[i*N +j] = 2*x[i*N + j]; X[0][0] X[0][1] X[0][2] } } X[1][0] X[1][1] X[1][2] X[2][0] X[2][1] X[2][2] 0x0 – 0x3 0x4 - 0x7 0x8-0x11 0x12 – 0x15 0x16 - 0x19 0x20-0x23 X[0][0] X[0][1] X[0][2] X[1][0] X[1][1] X[1][2] X[2][0] X[2][1] X[2][2] 10 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  11. Matrix Multiplication Refresher 11 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  12. Miss Rate Analysis for Matrix Multiply • Assume: • Block size = 32B (big enough for four doubles) • Matrix dimension (N) is very large • Cache is not even big enough to hold multiple rows • Analysis Method: • Look at access pattern of inner loop j j k = x i i k C A B 12 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  13. Layout of C Arrays in Memory (review) • C arrays allocated in row-major order • each row in contiguous memory locations • Stepping through columns in one row: • for (i = 0; i < N; i++) sum += a[0][i]; • accesses successive elements • if block size (B) > sizeof(a ij ) bytes, exploit spatial locality • miss rate = sizeof(a ij ) / B • Stepping through rows in one column: • for (i = 0; i < n; i++) sum += a[i][0]; • accesses distant elements • no spatial locality! • miss rate = 1 (i.e. 100%) 13 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  14. Matrix Multiplication (ijk) /* ijk */ Inner loop: for (i=0; i<n; i++) { (*,j) for (j=0; j<n; j++) { (i,j) sum = 0.0; (i,*) for (k=0; k<n; k++) A B C sum += a[i][k] * b[k][j]; c[i][j] = sum; } matmult/mm.c } Row-wise Column- Fixed wise Misses per inner loop iteration: A B C 0.25 1.0 0.0 14 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  15. Matrix Multiplication (jik) /* jik */ Inner loop: for (j=0; j<n; j++) { for (i=0; i<n; i++) { (*,j) sum = 0.0; (i,j) (i,*) for (k=0; k<n; k++) A B C sum += a[i][k] * b[k][j]; c[i][j] = sum } } matmult/mm.c Row-wise Column- Fixed wise Misses per inner loop iteration: A B C 0.25 1.0 0.0 15 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  16. Matrix Multiplication (kij) /* kij */ Inner loop: for (k=0; k<n; k++) { for (i=0; i<n; i++) { (i,k) (k,*) (i,*) r = a[i][k]; for (j=0; j<n; j++) A B C c[i][j] += r * b[k][j]; } Row-wise Row-wise } Fixed matmult/mm.c Misses per inner loop iteration: A B C 0.0 0.25 0.25 16 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  17. Matrix Multiplication (ikj) /* ikj */ Inner loop: for (i=0; i<n; i++) { for (k=0; k<n; k++) { (i,k) (k,*) (i,*) r = a[i][k]; for (j=0; j<n; j++) A B C c[i][j] += r * b[k][j]; } } matmult/mm.c Row-wise Row-wise Fixed Misses per inner loop iteration: A B C 0.0 0.25 0.25 17 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  18. Matrix Multiplication (jki) Inner loop: /* jki */ for (j=0; j<n; j++) { (*,k) (*,j) for (k=0; k<n; k++) { (k,j) r = b[k][j]; for (i=0; i<n; i++) A B C c[i][j] += a[i][k] * r; } } Column- Fixed Column- matmult/mm.c wise wise Misses per inner loop iteration: A B C 1.0 0.0 1.0 18 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  19. Matrix Multiplication (kji) /* kji */ Inner loop: for (k=0; k<n; k++) { (*,k) (*,j) for (j=0; j<n; j++) { (k,j) r = b[k][j]; for (i=0; i<n; i++) A B C c[i][j] += a[i][k] * r; } } matmult/mm.c Column- Fixed Column- wise wise Misses per inner loop iteration: A B C 1.0 0.0 1.0 19 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  20. Summary of Matrix Multiplication for (i=0; i<n; i++) { for (j=0; j<n; j++) { ijk (& jik): sum = 0.0; • 2 loads, 0 stores for (k=0; k<n; k++) { • misses/iter = 1.25 sum += a[i][k] * b[k][j];} c[i][j] = sum; } } for (k=0; k<n; k++) { kij (& ikj): for (i=0; i<n; i++) { • 2 loads, 1 store r = a[i][k]; for (j=0; j<n; j++){ • misses/iter = 0.5 c[i][j] += r * b[k][j];} } } for (j=0; j<n; j++) { jki (& kji): for (k=0; k<n; k++) { • 2 loads, 1 store r = b[k][j]; for (i=0; i<n; i++){ • misses/iter = 2.0 c[i][j] += a[i][k] * r;} } } 20 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  21. Core i7 Matrix Multiply Performance 100 jki / kji Cycles per inner loop iteration ijk / jik jki kji 10 ijk jik kij ikj kij / ikj 1 50 100 150 200 250 300 350 400 450 500 550 600 650 700 Array size (n) 21 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  22. Example: Matrix Multiplication c = (double *) calloc(sizeof(double), n*n); /* Multiply n x n matrices a and b */ void mmm(double *a, double *b, double *c, int n) { int i, j, k; for (i = 0; i < n; i++) for (j = 0; j < n; j++) for (k = 0; k < n; k++) c[i*n + j] += a[i*n + k] * b[k*n + j]; } j c a b = * i 22 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  23. Cache Miss Analysis • Assume: • Matrix elements are doubles • Assume the matrix is square • Cache block = 8 doubles • Cache size C << n (much smaller than n) • First iteration: • n/8 + n = 9n/8 misses n • Afterwards in cache: (schematic) = * = * 8 wide 23 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  24. Cache Miss Analysis • Assume: • Matrix elements are doubles • Cache block = 8 doubles • Cache size C << n (much smaller than n) • Second iteration: • Again: n/8 + n = 9n/8 misses n • Total misses: • 9n/8 * n 2 = (9/8) * n 3 = * 8 wide 24 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  25. Blocked Matrix Multiplication j1 c a b += * i1 Block size B x B 25 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  26. j1 c a b += * i1 Block size B x B 26 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  27. j1 c a b += * i1 Block size B x B 1 2 5 6 1 2 5 6 3 4 7 8 3 4 7 8 9 10 13 14 9 10 13 14 11 12 15 16 11 12 15 16 27 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

  28. j1 c a b += * i1 Block size B x B 1 2 5 6 1 2 5 6 3 4 7 8 3 4 7 8 9 10 13 14 9 10 13 14 11 12 15 16 11 12 15 16 1 2 1 2 5 6 9 10 * + * 3 4 3 4 7 8 11 12 28 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspective, Third Edition

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Plan Hierarchical memories and their impact on our programs 1 Cache Memories, Cache Complexity

Plan Hierarchical memories and their impact on our programs 1 Cache Memories, Cache Complexity

Plan Hierarchical memories and their impact on our programs 1 Cache Memories, Cache Complexity Cache Analysis in Practice 2 Marc Moreno Maza The Ideal-Cache Model 3 University of Western Ontario, London, Ontario (Canada) Cache Complexity

632 views • 25 slides
Cache Memories, Cache Complexity Marc Moreno Maza University of Western Ontario, London, Ontario

Cache Memories, Cache Complexity Marc Moreno Maza University of Western Ontario, London, Ontario

Cache Memories, Cache Complexity Marc Moreno Maza University of Western Ontario, London, Ontario (Canada) CS3101 and CS4402-9635 Plan Hierarchical memories and their impact on our programs Cache Analysis in Practice The Ideal-Cache Model

1.18k views • 100 slides
Cache Memories 15-213: Introduc;on to Computer Systems 12 th

Cache Memories 15-213: Introduc;on to Computer Systems 12 th

Carnegie Mellon Cache Memories 15-213: Introduc;on to Computer Systems 12 th Lecture, Oct. 8, 2015 Instructors: Randal E. Bryant and David R. OHallaron

646 views • 45 slides
Cache Memories Thanks to Randal E. Bryant and David R.

Cache Memories Thanks to Randal E. Bryant and David R.

Carnegie Mellon Cache Memories Thanks to Randal E. Bryant and David R. OHallaron from CMU Reading Assignment: Computer Systems: A Programmers Perspec4ve,

568 views • 45 slides
Cache Memories 15-213: Introduc0on to Computer Systems 10 th

Cache Memories 15-213: Introduc0on to Computer Systems 10 th

Carnegie Mellon Cache Memories 15-213: Introduc0on to Computer Systems 10 th Lecture, Sep. 23, 2010. Instructors: Randy Bryant and Dave OHallaron 1 Carnegie

752 views • 47 slides
1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

Today Cache memory organization and operation Performance impact of caches Cache Memories The memory mountain Rearranging loops to improve spatial locality Using blocking to improve temporal locality CSci 2021: Machine

102 views • 8 slides
Exploiting On-Chip Memories In Linux Applications Will Newton, Imagination Technologies What's

Exploiting On-Chip Memories In Linux Applications Will Newton, Imagination Technologies What's

Exploiting On-Chip Memories In Linux Applications Will Newton, Imagination Technologies What's wrong with SDRAM? 70 60 50 Cycles 40 30 20 10 0 L1 Cache Hit L1 Cache Miss 64 cycles is optimistic RAM clock often slower than core SoC

226 views • 20 slides
1 Classifying cache misses Cache Organization Classifying misses by causes (3Cs) Cache size,

1 Classifying cache misses Cache Organization Classifying misses by causes (3Cs) Cache size,

Cache Performance Metrics Cache miss rate: number of cache misses divided by number of accesses Lecture 14: Hardware Approaches for Cache Optimizations Cache hit time: the time between sending address and data returning from cache Cache

608 views • 4 slides
Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency

Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency

Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency Requirements of performance, availability, and disconnected operation require us to relax the goal of semantic transparency. - HTTP 1.1 specification

598 views • 13 slides
L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache, and all blocks initially marked as not valid , Given the main memory word addresses 0 1 2 3 4 3 4 15, calculate Cache hit rate. Cache Index

654 views • 9 slides
A A Comprehensiv ive R Revie iew o of the Chal alle lenges an and Opportunit itie ies

A A Comprehensiv ive R Revie iew o of the Chal alle lenges an and Opportunit itie ies

A A Comprehensiv ive R Revie iew o of the Chal alle lenges an and Opportunit itie ies Con onfronti ting C g Cache M Memor ory System Perfor ormance ce Photograph of Intel Xeon processor 7500 series die showing cache memories [ 1 ]

615 views • 24 slides
Memories Introduction Why do we need memory in an FPGA Device? Topics Types of FPGA

Memories Introduction Why do we need memory in an FPGA Device? Topics Types of FPGA

Memories Introduction Why do we need memory in an FPGA Device? Topics Types of FPGA Memories Available Resources Realizing memories in an HDL design Applications Examples Extra: Memory Controller Block Memories

692 views • 10 slides
Real Time Embedded Systems &quot; Memories Memories &quot; rene.beuchat@epfl.ch LAP/ISIM/IC/EPFL

Real Time Embedded Systems &quot; Memories Memories &quot; rene.beuchat@epfl.ch LAP/ISIM/IC/EPFL

Real Time Embedded Systems &quot; Memories Memories &quot; rene.beuchat@epfl.ch LAP/ISIM/IC/EPFL Charg de cours LSN/hepia Prof. HES 1998-2008 R.Beuchat / Memories 2 General classification of electronic memories Non-volatile

1.13k views • 86 slides
What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache

What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache

What Is Memory Hierarchy A typical memory hierarchy today: Lecture 13: Cache Basics and Cache Performance Proc/Regs L1-Cache Bigger Faster L2-Cache Memory hierarchy concept, cache L3-Cache (optional) design fundamentals, set-associative

303 views • 4 slides
A Suitcase full of memories: Exploring the meaning of tourism memories for people with dementia

A Suitcase full of memories: Exploring the meaning of tourism memories for people with dementia

A Suitcase full of memories: Exploring the meaning of tourism memories for people with dementia and their partners: reconnecting through the past. Jane Mullins (RGN, BN, PGCASR ) PhD Candidate School of Management Supervisors: Dr Diane

154 views • 14 slides
Cache Performance Associativity Replacement Samira Khan Cache Performance March 28,

Cache Performance Associativity Replacement Samira Khan Cache Performance March 28,

3/28/17 Agenda Review from last lecture Cache access Cache Performance Associativity Replacement Samira Khan Cache Performance March 28, 2017 Direct-Mapped Cache: Placement and Access Cache Abstraction and Metrics 00 | 000

518 views • 16 slides
Bounded Degree Spanning Tree using Iterative Relaxation Barna Saha March 11, 2015 Bounded

Bounded Degree Spanning Tree using Iterative Relaxation Barna Saha March 11, 2015 Bounded

Bounded Degree Spanning Tree using Iterative Relaxation Barna Saha March 11, 2015 Bounded Degree Spanning Tree Iterative Minimum Spanning Tree Algorithm Iterative Algorithm for Degree Bounded Spanning Tree Primal Dual Methods for Algorithm

873 views • 45 slides
Effective Word-Level Interpolation for Software Verification Alberto Griggio FBK-IRST

Effective Word-Level Interpolation for Software Verification Alberto Griggio FBK-IRST

FMCAD 2011 Effective Word-Level Interpolation for Software Verification Alberto Griggio FBK-IRST Motivations Craig interpolation applied succesfully for Formal Verification of both hardware and software Ongoing research (at least for

572 views • 41 slides
Fairness in ML 2: Equal opportunity and odds Privacy &amp; Fairness in Data Science CS848 Fall

Fairness in ML 2: Equal opportunity and odds Privacy &amp; Fairness in Data Science CS848 Fall

Fairness in ML 2: Equal opportunity and odds Privacy &amp; Fairness in Data Science CS848 Fall 2019 Slides adapted from https://fairmlclass.github.io/4.html 2 Outline Recap: Disparity impact Issues with Disparate Impact

5.56k views • 35 slides
BRST-BV treatment of Vasilievs four-dimensional higher-spin gravity P. Sundell (University of

BRST-BV treatment of Vasilievs four-dimensional higher-spin gravity P. Sundell (University of

BRST-BV treatment of Vasilievs four-dimensional higher-spin gravity P. Sundell (University of Mons, Belgique) Based on arXiv:1102.2219 with N. Boulanger arXiv:1103.2360 with E. Sezgin to-appear very soon with N. B. and N. Colombo. ESI

574 views • 29 slides
a r = v b c 1 Dot product (scalar

a r = v b c 1 Dot product (scalar

Appendix B: Linear Algebra Vectors x R n y v x a r = v b c 1 Dot product (scalar product) A B C C B A Cross product Vector product (cross product) 2

530 views • 8 slides
Functional Analysis Review Lorenzo Rosasco slides courtesy of Andre Wibisono 9.520:

Functional Analysis Review Lorenzo Rosasco slides courtesy of Andre Wibisono 9.520:

Outline Vector Spaces Hilbert Spaces Matrices Linear Operators Functional Analysis Review Lorenzo Rosasco slides courtesy of Andre Wibisono 9.520: Statistical Learning Theory and Applications February 13, 2012 L. Rosasco Functional

1.26k views • 53 slides
O N THE R ANK -O NE T HEOREM FOR BV FUNCTIONS Annalisa Massaccesi Warwick, 13 - 07 - 2017 Joint

O N THE R ANK -O NE T HEOREM FOR BV FUNCTIONS Annalisa Massaccesi Warwick, 13 - 07 - 2017 Joint

State-of-the-art Proof Generalization to Heisenberg groups O N THE R ANK -O NE T HEOREM FOR BV FUNCTIONS Annalisa Massaccesi Warwick, 13 - 07 - 2017 Joint works with Don Vittone Annalisa Massaccesi O N THE R ANK -O NE T HEOREM FOR BV

605 views • 32 slides
Speed and Scale: How to get there. Adrian Cockcroft @adrianco May 2014 # | Battery

Speed and Scale: How to get there. Adrian Cockcroft @adrianco May 2014 # | Battery

Speed and Scale: How to get there. Adrian Cockcroft @adrianco May 2014 # | Battery Ventures Typical reactions to my Netflix talks What Netflix is doing You guys are wont work crazy! Cant 2010 It only works for

809 views • 77 slides

More recommend