reex amining direct cache access to optimize i o
play

Reex amining Direct Cache Access to Optimize I/O Intensive - PowerPoint PPT Presentation

Nov 28, 2023 •371 likes •830 views

Reex amining Direct Cache Access to Optimize I/O Intensive Applications for Multi- hundred- gigabit Netw orks Alireza Farshin * , Amir Roozbeh *+ , Gerald Q. Maguire Jr. * , Dejan Kosti * KTH Royal Institute of Technology, School of Electrical

  1. Reex amining Direct Cache Access to Optimize I/O Intensive Applications for Multi- hundred- gigabit Netw orks Alireza Farshin * , Amir Roozbeh *+ , Gerald Q. Maguire Jr. * , Dejan Kostić * KTH Royal Institute of Technology, School of Electrical Engineering and Computer Science (EECS) + Ericsson Research

  2. Traditional I/O 1. I/O device DMAs* packets to main memory 2. CPU later fetches them to cache I/O Device * Direct Memory Access (DMA) 2020-07-02 2

  3. Traditional I/O 1. I/O device DMAs* packets to main memory 2. CPU later fetches them to cache Inefficient: Large number of accesses to main • memory High access latency (>60ns) • Unnecessary memory bandwidth usage • I/O Device * Direct Memory Access (DMA) 2020-07-02 3

  4. Direct Cache Access (DCA) 1. I/O device DMAs packets to main memory 2. DCA exploits TPH* to prefetch a portion of packets into cache 3. CPU later fetches them from cache Prefetch I/O Device * PCIe Transaction protocol Processing Hint (TPH) 2020-07-02 4

  5. Direct Cache Access (DCA) 1. I/O device DMAs packets to main memory 2. DCA exploits TPH* to prefetch a portion of packets into cache 3. CPU later fetches them from cache Prefetch Still inefficient in terms of memory bandwidth usage • Requires OS intervention and support from processor • I/O Device * PCIe Transaction protocol Processing Hint (TPH) 2020-07-02 5

  6. Intel Data Direct I/O (DDIO) DDIO in Xeon processors since • Xeon E5 DMA packets or descriptors • directly to/from Last Level Cache (LLC) I/O Device 2020-07-02 6

  7. Trends More in-network computing + offloading capabilities Push costly calculations into the network and perform state teful functions at the processor, which makes applications more I/O intensive. 2020-07-02 7

  8. Pressure from these trends Every 6.72 ns a new (64-B+20-B*) packet arrives at 100 Gbps More in-network computing + offloading capabilities Faster link speeds Multi-hundred-gigabit networks cannot tolerate memory access and interarrival time of packets continues to shrin ink * 7B preamble + 1B start-of-frame delimiter +12B inter-frame gap = 20B 2020-07-02 8

  9. DCA matters because Without DCA we are unable to process I/O at line rate, thus increasing packet loss or latency when utilizing multi-hundred-gigabit networks. 2020-07-02 9

  10. Forw arding Packets at 100 Gbps 100 G 100 Gbps Device under Test Packet Forwarding Packets Generator 1400 99 th Percentile Latency (µs) 1200 1000 800 Intel Xeon Gold 6140 600 400 Mellanox ConnectX-5 200 0 Each NIC is placed in 100 Gbps 200 Gbps a PCIe 3.0 16x slot* 100 Rate Gbps * A PCIe 3.0 16x slot is capable of providing ~125 Gbps effective full-duplex bandwidth. 2020-07-02 10

  11. What happens at 200 Gbps? When forwarding 2x100 G 00 Gbps at 200 Gbps, 30% higher latency for the NIC forwarding at 100 Gbps Device under Test Packet Forwarding Packets Generator 1400 99 th Percentile Latency (µs) 1200 1000 30% 800 Intel Xeon Gold 6140 600 400 Mellanox ConnectX-5 200 0 Each NIC is placed in 100 Gbps 200 Gbps a PCIe 3.0 16x slot* 100 100 Latency of the first NIC, when Gbps Gbps forwarding at indicated aggregate rate * A PCIe 3.0 16x slot is capable of providing ~125 Gbps effective full-duplex bandwidth. 2020-07-02 11

  12. How does DDIO w ork? CPU Socket Writing packets/descriptors: C C C C DDIO overwrites a cache line if if it is already C C C C present in any ny LLC ways ( ≡ write update or hit) C C C C Logical Write to the Same cache line LLC Sending/Receiving Already Packets via DDIO Present In LLC 2020-07-02 12

  13. How does DDIO w ork? CPU Socket Writing packets/descriptors: C C C C DDIO overwrites a cache line if if it is already C C C C present in any ny LLC ways ( ≡ write update or hit) C C C C Otherwise, DDIO allocates a cache line in a limited portion of LLC ( ≡ write allocate or miss) Logical Allocate a cache LLC line Sending/Receiving Not Packets via DDIO Present In LLC 2020-07-02 13

  14. How does DDIO w ork? CPU Socket Writing packets/descriptors: C C C C DDIO overwrites a cache line if if it is already C C C C present in any ny LLC ways ( ≡ write update or hit) C C C C Otherwise, DDIO allocates a cache line in a limited portion of LLC ( ≡ write allocate or miss) Logical Reading packets/descriptors: LLC NIC reads a cache line if it is already present in any LLC ways ( ≡ read hit) Sending/Receiving Packets via DDIO Otherwise, NIC reads it from main memory ( ≡ read miss) 2020-07-02 14

  15. How does DDIO w ork? CPU Socket Designed a set of micro-benchmarks to learn C C C C about DDIO: C C C C • Which ways are used for allocation? C C C C • How does DDIO interact with other applications? Logical • Does DMA via a remote CPU socket LLC pollute LLC? Sending/Receiving Packets via DDIO 2020-07-02 15

  16. LLC w ays used by DDIO I/O Application C0 Logical LLC 1 2 3 4 5 6 7 8 9 10 11 Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology 2020-07-02 16

  17. LLC w ays used by DDIO I/O Cache-sensitive Application Application + C0 C1 Logical LLC 1 2 3 4 5 6 7 8 9 10 11 Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 17

  18. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 18

  19. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 19

  20. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 20

  21. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 21

  22. LLC w ays used by DDIO 10 I/O Cache-sensitive Contention with code/data causes a rise in Application Application + Sum of Cache Misses (Million) the cache misses of the I/O application 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 22

  23. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 23

  24. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 24

  25. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 25

  26. LLC w ays used by DDIO 10 I/O Cache-sensitive Application Application + Sum of Cache Misses (Million) 8 C0 C1 6 4 Logical 2 LLC 0 1,2 2,3 3,4 4,5 5,6 6,7 7,8 8,9 9,10 10,11 1 2 3 4 5 6 7 8 9 10 11 Ways Allocated by CAT to the Cache-sensitive Application Sending/Receiving Use CAT* to Packets via DDIO limit code/data * Cache Allocation Technology + water_nsquared from Splash-3 benchmark 2020-07-02 26

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Cache Performance Associativity Replacement Samira Khan Cache Performance March 28,

Cache Performance Associativity Replacement Samira Khan Cache Performance March 28,

3/28/17 Agenda Review from last lecture Cache access Cache Performance Associativity Replacement Samira Khan Cache Performance March 28, 2017 Direct-Mapped Cache: Placement and Access Cache Abstraction and Metrics 00 | 000

518 views • 16 slides
L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache,

L09: Cache Name: ID: Question: Direct Mapping Cache Hit Rate Consider a 4-block empty Cache, and all blocks initially marked as not valid , Given the main memory word addresses 0 1 2 3 4 3 4 15, calculate Cache hit rate. Cache Index

654 views • 9 slides
Direct Map Cache and Set Associative Cache (Revision) Lecture 14 CDA 3103 07-07-2014 Example 1

Direct Map Cache and Set Associative Cache (Revision) Lecture 14 CDA 3103 07-07-2014 Example 1

Direct Map Cache and Set Associative Cache (Revision) Lecture 14 CDA 3103 07-07-2014 Example 1 A set-associative cache consists of 64 lines, or slots, divided into four-line sets. Main memory contains 4K blocks of 128 words each. Show the

357 views • 19 slides
Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and

Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and

Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers Norm Jouppi In Context At the time, CPU performance was really beginning to pull away from DRAM performance Increased

329 views • 22 slides
Improving Cache Performance AMAT: Average Memory Access Time AMAT = T hit + Miss Rate x Miss

Improving Cache Performance AMAT: Average Memory Access Time AMAT = T hit + Miss Rate x Miss

Improving Cache Performance AMAT: Average Memory Access Time AMAT = T hit + Miss Rate x Miss Penalty Small Hit Time: On the critical (common case) path Requires small, direct-mapped cache Small size and lack of associativity implies

286 views • 12 slides
1 Blocking Example Reducing Conflict Misses by Blocking /* After */ for (jj = 0; jj < N; jj

1 Blocking Example Reducing Conflict Misses by Blocking /* After */ for (jj = 0; jj < N; jj

Reducing Misses by Compiler Optimizing Memory Layout or Access Pattern McFarling [1989] reduced caches misses by 75% Lecture 15: Application-level Cache on 8KB direct mapped cache, 4 byte blocks in software Instructions Optimizations

191 views • 3 slides
1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

1 Locate set Cache Read Example: Direct Mapped Cache (E = 1) Check if any line in set

Today Cache memory organization and operation Performance impact of caches Cache Memories The memory mountain Rearranging loops to improve spatial locality Using blocking to improve temporal locality CSci 2021: Machine

102 views • 8 slides
Cache 10/27/16 The Memory Hierarchy Smaller On 1 cycle to access Chip Faster Registers CPU

Cache 10/27/16 The Memory Hierarchy Smaller On 1 cycle to access Chip Faster Registers CPU

Cache 10/27/16 The Memory Hierarchy Smaller On 1 cycle to access Chip Faster Registers CPU Storage instrs Costlier can per byte directly Cache(s) ~10s of cycles to access access (SRAM) Main memory ~100 cycles to access

1.07k views • 64 slides
1 Classifying cache misses Cache Organization Classifying misses by causes (3Cs) Cache size,

1 Classifying cache misses Cache Organization Classifying misses by causes (3Cs) Cache size,

Cache Performance Metrics Cache miss rate: number of cache misses divided by number of accesses Lecture 14: Hardware Approaches for Cache Optimizations Cache hit time: the time between sending address and data returning from cache Cache

608 views • 4 slides
Access Control Access Control 1 Access Control Access control : ensures that all direct

Access Control Access Control 1 Access Control Access control : ensures that all direct

Access Control Access Control 1 Access Control Access control : ensures that all direct accesses to object are authorized a scheme for mapping users to allowed actions Protection objects : system resources for which protection is

576 views • 21 slides
1 Direct-mapped cache Tags and validation stickers Address (showing bit positions) 31 30 13 12

1 Direct-mapped cache Tags and validation stickers Address (showing bit positions) 31 30 13 12

Basic structure of memory hierarchy Exploiting memory hierarchy Speed Size Cost ($/bits) Technology Cache basics CPU < ns 32 high CPU registers 0.5-5 ns KB-MB $4K-$10K SRAM Cache per GB 50-70 ns MB-GB $100-$200 DRAM Main

190 views • 8 slides
Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency

Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency

Web Cache Consistency Web Cache Consistency Web Cache Consistency Web Cache Consistency Requirements of performance, availability, and disconnected operation require us to relax the goal of semantic transparency. - HTTP 1.1 specification

598 views • 13 slides
Cache Refill/Access Decoupling for Vector Machines Christopher Batten, Ronny Krashinsky, Steve

Cache Refill/Access Decoupling for Vector Machines Christopher Batten, Ronny Krashinsky, Steve

Cache Refill/Access Decoupling for Vector Machines , Christopher Batten, Ronny Krashinsky, Steven Gerding, and Krste Asanovic, 3 7th International Symposium on Microarchitecture, Portland, Oregon, December 2004 Cache Refill/Access Decoupling for

1.33k views • 77 slides
Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL)

Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL)

Caches Electronic Computers M Caches 1 Cache LOCALITY PRINCIPLE (SPATIAL AND TEMPORAL) WORKING SET CPU Cache Registers Cache I lev. Cache II lev. Cache III lev. Memory Disk Tape The cache is a memory with an access time some

572 views • 55 slides
Is Is There a Ge Genetic Ba Basi sis t s to R Race? Exam amining human an var ariation

Is Is There a Ge Genetic Ba Basi sis t s to R Race? Exam amining human an var ariation

Is Is There a Ge Genetic Ba Basi sis t s to R Race? Exam amining human an var ariation an and an ancestry usi using ng mt mtDNA CCSF Mission Campus February 10, 2018 No Note t to t teache achers: : the following slides are

583 views • 57 slides
Direct Access Electricity Administration/Finance Issues Cte May 16, 2013 Approved Direct Access

Direct Access Electricity Administration/Finance Issues Cte May 16, 2013 Approved Direct Access

Direct Access Electricity Administration/Finance Issues Cte May 16, 2013 Approved Direct Access (DA) Service Providers Contracts in place for two 1. major DA Service Providers They will both offer pricing, 2. with the award going to the

186 views • 14 slides
UNIVERSITY OF CALIFORNIA Economics 134 DEPARTMENT OF ECONOMICS Spring 2018 Professor David

UNIVERSITY OF CALIFORNIA Economics 134 DEPARTMENT OF ECONOMICS Spring 2018 Professor David

UNIVERSITY OF CALIFORNIA Economics 134 DEPARTMENT OF ECONOMICS Spring 2018 Professor David Romer LECTURE 7 MONETARY FACTORS IN THE GREAT DEPRESSION? FEBRUARY 7, 2018 I. M ONETARY A RRANGEMENTS IN THE 1920 S A. Early Federal Reserve B.

454 views • 43 slides
(De)Stabilizing the ACAs Insurance Market Mark A. Hall Wake Forest University Brookings

(De)Stabilizing the ACAs Insurance Market Mark A. Hall Wake Forest University Brookings

(De)Stabilizing the ACAs Insurance Market Mark A. Hall Wake Forest University Brookings Institution, Nonresident Senior Fellow Funded by Robert Wood Johnson Foundation Assisted by AcademyHealth Policy Shifts Under the Trump Administration

736 views • 39 slides
Volusia County, Florida Volusia County recognizes the importance of supporting our local

Volusia County, Florida Volusia County recognizes the importance of supporting our local

Volusia County, Florida Volusia County recognizes the importance of supporting our local companies by encouraging them to seek out and develop new opportunities to sell their locally produced products and services. Increase your sales in

462 views • 15 slides
From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig

From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig

From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Yoav Goldberg, Hal Daume III T odays topics: Addressing compounding error Improving on gold parse oracle

687 views • 22 slides
Knapsack Problem Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by

Knapsack Problem Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by

CMPS 6610/4610 Fall 2016 Knapsack Problem Carola Wenk Slides courtesy of Charles Leiserson with changes and additions by Carola Wenk CMPS 6610/4610 Algorithms 1 Knapsack Problem Given a knapsack with weight capacity , and given

298 views • 8 slides
Creating dummies Nele Verbiest, Ph. D. Senior Data Scientist @ Python Predictions DataCamp

Creating dummies Nele Verbiest, Ph. D. Senior Data Scientist @ Python Predictions DataCamp

DataCamp Intermediate Predictive Analytics in Python INTERMEDIATE PREDICTIVE ANALYTICS IN PYTHON Creating dummies Nele Verbiest, Ph. D. Senior Data Scientist @ Python Predictions DataCamp Intermediate Predictive Analytics in Python Motivation

380 views • 34 slides
All that glitters is not gold: Zero-point energy in the Johnson noise of resistors L.B. Kish 1 , G.

All that glitters is not gold: Zero-point energy in the Johnson noise of resistors L.B. Kish 1 , G.

An error does not become truth by multiplied propagation. Mahatma Gandhi see more at: http://vixra.org/abs/1504.0183 http://vixra.org/abs/1506.0009 (still developing, not final) All that glitters is not gold: Zero-point energy in the Johnson

364 views • 33 slides
Slides for Lecture 30 ENEL 353: Digital Circuits Fall 2013 Term Steve Norman, PhD, PEng

Slides for Lecture 30 ENEL 353: Digital Circuits Fall 2013 Term Steve Norman, PhD, PEng

Slides for Lecture 30 ENEL 353: Digital Circuits Fall 2013 Term Steve Norman, PhD, PEng Electrical & Computer Engineering Schulich School of Engineering University of Calgary 20 November, 2013 slide 2/21 ENEL 353 F13 Section 02

418 views • 21 slides

More recommend