cost effective compiler directed memory prefetching and
play

Cost-Effective Compiler Directed Memory Prefetching and Bypassing - PowerPoint PPT Presentation

Oct 01, 2022 •272 likes •676 views

Cost-Effective Compiler Directed Memory Prefetching and Bypassing Daniel Ortega, , Eduard Ayguad e , Jean-Loup Baer and Mateo Valero Departamento de Arquitectura de Computadores, Department of

  1. ☎ � � ✁ � ✂ ✄ ✆ Cost-Effective Compiler Directed Memory Prefetching and Bypassing Daniel Ortega, , Eduard Ayguad´ e , Jean-Loup Baer and Mateo Valero Departamento de Arquitectura de Computadores, Department of Computer Science and Engineering, Universidad Polit´ ecnica de Catalu˜ na – Barcelona University of Washington – Seattle dortega,eduard,mateo @ac.upc.es baer@cs.washington.edu PACT 2002 – p.1/20

  2. Conventional Cache Hierarchy Regs 1 cycle L1 L1 10s cycles L2 >100 cycles Main Memory PACT 2002 – p.2/20

  3. Conventional Cache Hierarchy Regs 3-5 cycles (more?) L1 L1 10s cycles L2 >100 cycles Main Memory PACT 2002 – p.2/20

  4. Conventional Cache Hierarchy Regs 3-5 cycles (more?) (smaller?) L1 L1 10s cycles L2 >100 cycles Main Memory PACT 2002 – p.2/20

  5. ✝ ✝ ✞ ✞ ✞ Our approach Attack register-L1 gap with memory instruction bypassing Use hardware prefetcher for L1-L2 gap Directed by the software thus simple Hardware no recovery needed in case of misprediction PACT 2002 – p.3/20

  6. Limit Study 2.5 SpeedUp No Prefetching No Bypassing 2.0 No Prefetching Perfect Bypassing Perfect Prefetching No Bypassing Perfect Prefetching Perfect Bypassing 1.5 1.0 applu apsi hydro2d swim tomcatv Average Limit values for a 4-way machine PACT 2002 – p.4/20

  7. ✝ ✝ ✝ ✝ Index Motivation Memory Instruction Bypassing Compiler Directed Memory Prefetcher Comparison with APDP PACT 2002 – p.5/20

  8. ✠ ✟ ✡☛ MIB through Renaming I ... pref a[i] ... load r , a[i+1] load r , a[i] load r , a[i+3] ... loop branch PACT 2002 – p.6/20

  9. ✡☛ ✑ ✠ ✡ ✡ ✑ ✡ ✟ ✒ ✌ ✎ ✡ ✍ ✌ ☞ MIB through Renaming I ... ... ✡✏✎ ✡✏✎ pref a[i] pref r , a[i] ... ... load r , a[i+1] load r , a[i+1] load r , a[i] load r , a[i] load r , a[i+3] load r , a[i+3] ... ... loop branch loop branch PACT 2002 – p.6/20

  10. ✔ ✕ ✗ ✕ ✓ ✕✖ ✕ ✖ ✓ ✓ ✓ ✕ ✓ ✕ ✓ ✕✖ ✕ ✓ ✔ ✓ ✓ ✓ ✔ MIB through Renaming II Renaming Table . . . . . . r f r f r f . . . . . . Special Renaming Table . . . . . . r . . . r . . . r . . . . . . . . . PACT 2002 – p.7/20

  11. ✖ ✕ ✕ ✓ ✤ ✓ ✥ ✕✖ ✕ ✗ ✔ ✥✦ ✤ ✓ ✓ ✓ ✕ ✔ ✓ ✕ ✕ ✓ ✕ ✓ ✣ ✁ ✘ ✙ ✢ ✛ ✕✖ ✚ ✓ ✔ ✓ MIB through Renaming II ... Renaming Table ✚✜✛ ✚✜✛ pref r , a[i] . . . . . . r f r f r f . . . . . . Special Renaming Table . . . . . . r f r f r f . . . . . . PACT 2002 – p.7/20

  12. ✕ ✥✦ ✕ ✕ ✕ ✓ ✓ ✕✖ ✥ ✤ ✔ ✕ ✓ ✓ ✤ ✓ ✔ ✓ ✓ ✕ ✖ ✓ ✕✖ ✓ ✔ ✚ ✁ ✘ ✙ ✓ ✢ ✛ ✚ ✢ ✣ MIB through Renaming II ... Renaming Table ✚✜✛ ✚✜✛ pref r , a[i] . . . . . . r f ... r f load r , a[i+1] r f . . . . . . Special Renaming Table . . . . . . r f r . . . r f . . . . . . PACT 2002 – p.7/20

  13. ✓ ✤ ✕ ✕✖ ✓ ✕ ✕ ✓ ✓ ✓ ✕ ✓ ✥ ✤ ✔ ✕ ✕ ✓ ✓ ✖ ✔ ✔ ✓ ✢ ✘ ✛ ✚ ✣ ✙ ✢ ✚ ✥✦ ✁ ✘ ✓ MIB through Renaming II ... Renaming Table ✚✜✛ ✚✜✛ pref r , a[i] . . . . . . r f ... r f load r , a[i+1] r f . . . . . . load r , a[i] Special Renaming Table . . . . . . r . . . r . . . r f . . . . . . PACT 2002 – p.7/20

  14. ✕ ✓ ✓ ✦ ✚ ✚ ✥ ✤ ✥ ✔ ✕ ✔ ✓ ✤ ✕✖ ✓ ✓ ✘ ✓ ✓ ✕ ✓ ✁ ✓ ✘ ✕ ✙ ✓ ✢ ✢ ✛ ✚ ✣ ✖ ✕ ✚ ✕ MIB through Renaming II ... Renaming Table ✚✜✛ ✚✜✛ pref r , a[i] . . . . . . r f ... r f load r , a[i+1] r f . . . . . . load r , a[i] Special Renaming Table load r , a[i+3] . . . . . . r . . . r . . . r . . . . . . . . . PACT 2002 – p.7/20

  15. ✕ ✥ ✓ ✦ ✚ ✚ ✥ ✤ ✕ ✔ ✓ ✤ ✓ ✔ ✓ ✓ ✕✖ ✘ ✓ ✕ ✕ ✓ ✁ ✓ ✘ ✕ ✙ ✓ ✢ ✢ ✛ ✚ ✣ ✖ ✕ ✚ ✓ MIB through Renaming II ... Renaming Table ✚✜✛ ✚✜✛ pref r , a[i] . . . . . . r f ... r f load r , a[i+1] r f . . . . . . load r , a[i] Special Renaming Table load r , a[i+3] . . . . . . r . . . ... r . . . loop branch r . . . . . . . . . PACT 2002 – p.7/20

  16. ✝ ✝ ✝ ✝ Index Motivation Memory Instruction Bypassing Compiler Directed Memory Prefetcher Comparison with APDP PACT 2002 – p.8/20

  17. ✝ ✝ ✝ ✝ ✝ Decoupled Prefetcher Compiler inserts prefetching operations Instructions bring data closer to the processor Memory instruction bypassing takes care of bringing data to the register file from L1 Compiler also instructs prefetching hardware to prefetch ahead (to L1) No.s of prefetches are minimised by compiler control PACT 2002 – p.9/20

  18. Prefetching Mechanism pctag type last @ ... PACT 2002 – p.10/20

  19. ✧ Prefetching Mechanism Decode phase pctag type last @ pctag PC last @ ... PACT 2002 – p.10/20

  20. ✧ ✧ Prefetching Mechanism Decode phase pctag type last @ Address calc. pctag PC last @ ... eff @ - stride PACT 2002 – p.10/20

  21. ✧ ✧ ✧ Prefetching Mechanism Decode phase pctag type last @ Address calc. pctag PC last @ ... Generate new prefetch eff @ - stride *N+ PACT 2002 – p.10/20

  22. ✧ ✧ ★ ✧ Prefetching Mechanism Decode phase pctag type last @ Address calc. pctag PC last @ ... Generate new prefetch eff @ N depends on type - stride *N+ PACT 2002 – p.10/20

  23. ★ ✧ ★ ✧ ✧ Prefetching Mechanism Decode phase pctag type last @ Address calc. pctag PC last @ ... Generate new prefetch eff @ N depends on type - stride wait for free port *N+ PACT 2002 – p.10/20

  24. Performance Results 2.0 O3 LoadStore O3+SW pref LoadStore SpeedUp O3+SW pref only Bypassing O3+SW pref Proposal 16 1.5 O3+SW pref Proposal 32 O3+SW pref Proposal 64 O3+SW pref Proposal infinite 1.0 applu apsi hydro2d swim tomcatv Average Effect of number of streams in a 4-way machine PACT 2002 – p.11/20

  25. Performance Results II 2.0 O3 LoadStore SpeedUp O3+SW pref LoadStore O3+SW pref only Bypassing 1.5 O3+SW pref Proposal N=(pref 1,load 1) O3+SW pref Proposal N=(pref 1,load 2) O3+SW pref Proposal N=(pref 2,load 1) 1.0 applu apsi hydro2d swim tomcatv Average Effect of lookahead policy in a 4-way machine with 32 entries PACT 2002 – p.12/20

  26. ✝ ✝ ✝ ✝ Index Motivation Memory Instruction Bypassing Compiler Directed Memory Prefetcher Comparison with APDP PACT 2002 – p.13/20

  27. ✞ ✝ ✞ ✞ ✩ ✞ ✄ Comparison with APDP Address Prediction for Data Prefetching predicts addresses of memory operations makes prediction available as soon as load arrives to decoding needs recovery mechanism in case of misprediction dynamic mechanism J. González and A. González, ICS 97 PACT 2002 – p.14/20

  28. Prefetching Mechanism v address stride conf. val ... PACT 2002 – p.15/20

  29. ✧ Prefetching Mechanism Decode phase v address stride conf. val PC ... PACT 2002 – p.15/20

  30. ✧ ✧ Prefetching Mechanism Decode phase v address stride conf. val Is value correct? PC ... ? PACT 2002 – p.15/20

  31. ✧ ✧ ✪ ✧ Prefetching Mechanism Decode phase v address stride conf. val Is value correct? PC ... Yes bypass ? register file PACT 2002 – p.15/20

  32. ✧ ✧ ✧ ✪ ✧ Prefetching Mechanism Decode phase v address stride conf. val Is value correct? PC ... Yes bypass new address ? ? Address calc. register file PACT 2002 – p.15/20

  33. ✧ ✧ ✧ ✪ ✧ ✧ Prefetching Mechanism Decode phase v address stride conf. val Is value correct? PC ... Yes bypass new address ? Address calc. - Update table register file PACT 2002 – p.15/20

  34. ✧ ✧ ✧ ✧ ✧ ✪ ✧ Prefetching Mechanism Decode phase v address stride conf. val Is value correct? PC ... Yes bypass new address ? Address calc. - Update table register file ? + Generate prefetch? new prediction PACT 2002 – p.15/20

  35. Comparison Results 2.0 O3 LoadStore O3 APDP only prefetching SpeedUp O3 APDP (squash) O3 APDP (selective) 1.5 O3+SW pref LoadStore O3+SW pref APDP only prefetching O3+SW pref APDP (squash) O3+SW pref APDP (selective) 1.0 applu apsi hydro2d swim tomcatv Average Effect of only prefetching in APDP (4-way and 2 ports) PACT 2002 – p.16/20

  36. Comparison Results 1.2 O3 LoadStore SpeedUp O3 APDP (squash) O3 APDP (selective) 1.1 O3 only Bypassing O3 Proposal 16 entries O3 Proposal 32 entries 1.0 0.9 applu apsi hydro2d swim tomcatv Average 4 way machine comparison against APDP PACT 2002 – p.17/20

  37. Memory traffic applu apsi hydro2d swim tomcatv 1.00 1.00 1.00 1.00 1.00 LoadStore 1.14 1.14 1.24 1.21 1.21 APDP sel. 1.14 1.15 1.24 1.21 1.22 APDP sq. 0.90 0.74 0.82 0.72 0.89 Bypassing 0.98 0.85 0.95 0.83 0.94 Proposal PACT 2002 – p.18/20

  38. Any questions? PACT 2002 – p.19/20

  39. Thank you dortega@ac.upc.es PACT 2002 – p.20/20

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

1 Prefetching Implementations Recall Stream Buffer Diagram Sequential and stride prefetching

1 Prefetching Implementations Recall Stream Buffer Diagram Sequential and stride prefetching

Memory Wall 1000 Lecture 25: Advanced Data CPU Prefetching Techniques 100 Prefetching and data prefetching overview, Stride prefetching, 10 Markov prefetching, precomputation- DRAM based prefetching 1 1980 1981 1982 1983 1984 1985

298 views • 4 slides
Prefetching Hyperlinks Prefetching Methods Prefetching Uncacheable/Dynamic Data

Prefetching Hyperlinks Prefetching Methods Prefetching Uncacheable/Dynamic Data

Prefetching Hyperlinks Prefetching Methods Prefetching Uncacheable/Dynamic Data Effect on Network Load Prefetching and the Wireless World Wide Web Privacy Concerns Additional References February 22, 2000 1

436 views • 15 slides
Memory Prefetching Nima Honarmand Spring 2016 :: CSE 502 Computer Architecture The memory

Memory Prefetching Nima Honarmand Spring 2016 :: CSE 502 Computer Architecture The memory

Spring 2016 :: CSE 502 Computer Architecture Memory Prefetching Nima Honarmand Spring 2016 :: CSE 502 Computer Architecture The memory wall 10000 Performance 1000 Processor 100 10 Memory 1 1985 1990 1995 2000 2005 2010

312 views • 29 slides
Reducing Seek Overhead with Application-Directed Prefetching Steve VanDeBogart, Christopher

Reducing Seek Overhead with Application-Directed Prefetching Steve VanDeBogart, Christopher

Reducing Seek Overhead with Application-Directed Prefetching Steve VanDeBogart, Christopher Frost, Eddie Kohler University of California, Los Angeles http://libprefetch.cs.ucla.edu Disks are Relatively Slow Average Throughput Whetstone Seek

789 views • 58 slides
Effectively Prefetching Remote Memory with Leap Hasan Al Maruf and Mosharaf Chowdhury 1

Effectively Prefetching Remote Memory with Leap Hasan Al Maruf and Mosharaf Chowdhury 1

Effectively Prefetching Remote Memory with Leap Hasan Al Maruf and Mosharaf Chowdhury 1 Memory-Intensive Applications 2 Perform Great! 40 38.61 35 30 TPS (Thousands) 25 20 15 10 6.61 5 1.01 0 100% 75% 50% In-Memory Working Set

864 views • 39 slides
Memory Prefetching Instructor: Nima Honarmand Spring 2015 :: CSE 502 Computer Architecture

Memory Prefetching Instructor: Nima Honarmand Spring 2015 :: CSE 502 Computer Architecture

Spring 2015 :: CSE 502 Computer Architecture Memory Prefetching Instructor: Nima Honarmand Spring 2015 :: CSE 502 Computer Architecture The memory wall 10000 Performance 1000 Processor 100 10 Memory 1 1985 1990 1995 2000 2005

337 views • 30 slides
Memory Tagging: e m p o s r r F e p how it improves C/C++ memory safety. Compiler

Memory Tagging: e m p o s r r F e p how it improves C/C++ memory safety. Compiler

r e l i p e m v o i t c c Memory Tagging: e m p o s r r F e p how it improves C/C++ memory safety. Compiler perspective. Kostya Serebryany , Evgenii Stepanov, Vlad Tsyrklevich (Google) Oct 2018 1 Agenda ARM v8.5

504 views • 29 slides
Syntax Directed Analysis Chapter 5 1 Compiler Construction Syntax Directed Analysis

Syntax Directed Analysis Chapter 5 1 Compiler Construction Syntax Directed Analysis

Syntax Directed Analysis Chapter 5 1 Compiler Construction Syntax Directed Analysis Syntax-Directed Definitions Generalizes a context-free grammar Each grammar symbol has an associated set of attributes Synthesized attributes

3.06k views • 30 slides
Prefetching in Hybrid Main Memory Systems Subisha V , Varun Gohil , Nisarg Ujjainkar

Prefetching in Hybrid Main Memory Systems Subisha V , Varun Gohil , Nisarg Ujjainkar

Prefetching in Hybrid Main Memory Systems Subisha V , Varun Gohil , Nisarg Ujjainkar , Manu Awasthi * IIT Gandhinagar * Ashoka University HotStorage 2020 1 2 Outline of the Presentation Background Insights

859 views • 83 slides
1 Prefetching Prefetching or Streaming or Streaming Prediction Prediction Compiler-driven

1 Prefetching Prefetching or Streaming or Streaming Prediction Prediction Compiler-driven

I/O Buffering and Caching I/O Buffering and Caching I/O accesses are reads or writes (e.g., to files) Application access is arbitary (offset, len) Convert accesses to read/write of fixed-size blocks or pages I/O Buffering and Streaming I/O

362 views • 4 slides
Memory Management Memory Manager Requirements Minimize primary memory access time

Memory Management Memory Manager Requirements Minimize primary memory access time

Memory Management Memory Manager Requirements Minimize primary memory access time Maximize primary memory size Primary memory must be cost-effective Todays memory manager: Allocates primary memory to processes Maps

637 views • 27 slides
Linux solution for prefetching necessary data during application and system startup Krzysztof

Linux solution for prefetching necessary data during application and system startup Krzysztof

Linux solution for prefetching necessary data during application and system startup Krzysztof Lichota lichota@mimuw.edu.pl What is prefetching and why it is needed? The problem In modern computers CPUs fast Memory fast

893 views • 53 slides
Mnemo: Boosting Memory Cost Efficiency in Hybrid Memory Systems Thaleia Dimitra Doudali Ada

Mnemo: Boosting Memory Cost Efficiency in Hybrid Memory Systems Thaleia Dimitra Doudali Ada

Mnemo: Boosting Memory Cost Efficiency in Hybrid Memory Systems Thaleia Dimitra Doudali Ada Gavrilovska Problem Space Cost of memory dominates in the cloud. 60% - 85% of the hourly VM cost! Big Data Analytics Volatile Memory (DRAM)

535 views • 18 slides
0 Correct Programming Compiler Compiler Hardware Language P Syntax PL compile P P PL HW

0 Correct Programming Compiler Compiler Hardware Language P Syntax PL compile P P PL HW

Bridging the Gap between Programming Languages and Hardware Weak Memory Models Anton Podkopaev Ori Lahav Viktor Vafeiadis 0 Correct Programming Compiler Compiler Hardware Language P Syntax PL compile P P PL HW PL HW is Memory Model

980 views • 74 slides
1 Store Buffer Design Example Memory Dependence Any load instruction receives the memory Store

1 Store Buffer Design Example Memory Dependence Any load instruction receives the memory Store

Register and Memory Dependences Store: SW Rt, A(Rs) LW Rt, A(Rs) 1. 1. Calculate effective Calculate effective Lecture 10: Memory Dependence memory address memory address Detection and Speculation dependent on Rs dependent on Rs

479 views • 3 slides
Aestimo : A Feedback-Directed Optimization Evaluation Tool A Compiler Perspective on Input

Aestimo : A Feedback-Directed Optimization Evaluation Tool A Compiler Perspective on Input

Aestimo : A Feedback-Directed Optimization Evaluation Tool A Compiler Perspective on Input Characterization Paul Berube Compiler Design and Optimization Laboratory University of Alberta March 21, 2006 Paul Berube, ISPASS 2006 1 Outline

910 views • 37 slides
based Last Level Cache Kunal Korgaonkar, Ishwar Bhati, Huichu Liu, Jayesh Gaur, Sasikanth

based Last Level Cache Kunal Korgaonkar, Ishwar Bhati, Huichu Liu, Jayesh Gaur, Sasikanth

Density Tradeoffs of Non-Volatile Memory as a Replacement for SRAM based Last Level Cache Kunal Korgaonkar, Ishwar Bhati, Huichu Liu, Jayesh Gaur, Sasikanth Manipatruni, Sreenivas Subramoney, Tanay Karnik, Steven Swanson, Ian Young and Hong Wang

373 views • 18 slides
Boxed Out: Blocking Cellular Interconnect Bypass Fraud at the Network Edge Brad Reaves * , Ethan

Boxed Out: Blocking Cellular Interconnect Bypass Fraud at the Network Edge Brad Reaves * , Ethan

Boxed Out: Blocking Cellular Interconnect Bypass Fraud at the Network Edge Brad Reaves * , Ethan Shernan ** , Adam Bates * , Henry Carter ** , Patrick T raynor * *Florida Institute for Cyber Security University of Florida **Georgia Institute of T

421 views • 23 slides
AVPASS: Automatically Bypassing Android Malware Detection System Jinho Jung, Chanil Jeon, Max

AVPASS: Automatically Bypassing Android Malware Detection System Jinho Jung, Chanil Jeon, Max

AVPASS: Automatically Bypassing Android Malware Detection System Jinho Jung, Chanil Jeon, Max Wolotsky, Insu Yun, and Taesoo Kim Georgia Institute of Technology, July 27, 2017 Ab About t Us SSLab (@GT) Focusing on s ystem and security

1.25k views • 72 slides
PRAGUE-6 Trial Off-Pump Versus On-Pump Coronary Artery Bypass Graft Surgery in Patients With

PRAGUE-6 Trial Off-Pump Versus On-Pump Coronary Artery Bypass Graft Surgery in Patients With

PRAGUE-6 Trial Off-Pump Versus On-Pump Coronary Artery Bypass Graft Surgery in Patients With EuroSCORE 6 Jan Hlavicka, Zbynek Straka, Stepan Jelinek, Petr Budera, Tomas Vanek, Petr Widimsky Cardiocenter Royal Vineyards, Third

310 views • 17 slides
Exploi'ng Unpatched iOS Vulnerabili'es for Fun and Profit

Exploi'ng Unpatched iOS Vulnerabili'es for Fun and Profit

Exploi'ng Unpatched iOS Vulnerabili'es for Fun and Profit Yeongjin Jang, Tielei Wang, Byoungyoung Lee, and Billy Lau Georgia Tech Informa;on Security Center

989 views • 84 slides
Cloud Interconnections William B. Norton Console, Chief Scientist Last Updated: 9/19/16 4:04pm

Cloud Interconnections William B. Norton Console, Chief Scientist Last Updated: 9/19/16 4:04pm

Cloud Interconnect Models v1.6 Cloud Interconnections William B. Norton Console, Chief Scientist Last Updated: 9/19/16 4:04pm Comments to the author welcome: wbn@console.net Cloud Interconnections William B. Norton Console, Chief Scientist

300 views • 12 slides
Bypassing Microsoft JEA role capabilities for fun & profit whoami Cristhian Parrot -

Bypassing Microsoft JEA role capabilities for fun & profit whoami Cristhian Parrot -

Bypassing Microsoft JEA role capabilities for fun & profit whoami Cristhian Parrot - @elc0rr3Km1n0s Sr. Penetration Tester & Lead Auditor @Airbus Father, Bug Hunter, Tech-entrepreneur Plan Intro Install Prerequisites

376 views • 22 slides
2PAC 2Furious: Envisioning an iOS compromise in 2019 Marco Grassi - @marcograss Liang Chen -

2PAC 2Furious: Envisioning an iOS compromise in 2019 Marco Grassi - @marcograss Liang Chen -

2PAC 2Furious: Envisioning an iOS compromise in 2019 Marco Grassi - @marcograss Liang Chen - @chenliang0817 About Us Members of Tencent KEEN Security Lab (formerly known as KeenTeam) Marco (@marcograss): My main focus is

846 views • 61 slides

More recommend