comment on bitonic merging more cuda performance tuning
play

Comment on bitonic merging; more CUDA performance tuning CSE 6230: - PowerPoint PPT Presentation

Jan 30, 2024 •990 likes •1.88k views

Comment on bitonic merging; more CUDA performance tuning CSE 6230: HPC Tools & Apps Tu Sep 18, 2012 Tuesday, September 18, 12 Comment on bitonic merging , including ideas & hints for Lab 3 Note: Some figures taken from Grama et al.

  1. Comment on bitonic merging; more CUDA performance tuning CSE 6230: HPC Tools & Apps Tu Sep 18, 2012 Tuesday, September 18, 12

  2. ๏ Comment on bitonic merging , including ideas & hints for Lab 3 Note: Some figures taken from Grama et al. book (2003) http://www-users.cs.umn.edu/~karypis/parbook/ This book is also available online through the GT library – see our course website. Tuesday, September 18, 12

  3. Source: Grama et al. (2003) Tuesday, September 18, 12

  4. Summary so far: bitonicMerge (bitonic sequence) == sorted Q: How do we get a bitonic sequence? Tuesday, September 18, 12

  5. Source: Grama et al. (2003) Tuesday, September 18, 12

  6. “ ⊕ ” = (min, max) “ ⊖ ” = (max, min) Source: Grama et al. (2003) Tuesday, September 18, 12

  7. “ ⊕ ” = (min, max) “ ⊖ ” = (max, min) Source: Grama et al. (2003) Tuesday, September 18, 12

  8. “ ⊕ ” = (min, max) “ ⊖ ” = (max, min) Source: Grama et al. (2003) Tuesday, September 18, 12

  9. “ ⊕ ” = (min, max) “ ⊖ ” = (max, min) Source: Grama et al. (2003) Tuesday, September 18, 12

  10. Source: Grama et al. (2003) Tuesday, September 18, 12

  11. Source: Grama et al. (2003) Tuesday, September 18, 12

  12. Bitonic sort parallel complexity (work-depth)? Tuesday, September 18, 12

  13. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 7: 0111 8: 1000 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 Tuesday, September 18, 12

  14. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 7: 0111 8: 1000 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log p steps: Comm req’d log (n/p) steps: No comm Block Layout (p=4) Tuesday, September 18, 12

  15. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 rounds of communication = O (log n ) 5: 0101 6: 0110 number of pairwise exchanges per round = O ( P ) 7: 0111 words sent per exchange = O ( n / P ) 8: 1000 9: 1001 total words sent = O ( n log n ) 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log p steps: Comm req’d log (n/p) steps: No comm Block Layout (p=4) Tuesday, September 18, 12

  16. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 7: 0111 8: 1000 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log p steps: Comm req’d log (n/p) steps: No comm Block Layout (p=4) Tuesday, September 18, 12

  17. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 7: 0111 8: 1000 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log (n/p): No comm log (p): Comm req’d Cyclic Layout (p=4) Tuesday, September 18, 12

  18. These (block or cyclic) examples are binary exchange algorithms. Question: Can we get the “best” of these two schemes? Tuesday, September 18, 12

  19. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 … All-to-all 7: 0111 exchange 8: 1000 … 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log (p): No comm log (n/p): No comm “Transpose” (p=4) Tuesday, September 18, 12

  20. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 rounds of communication = 1 5: 0101 6: 0110 number of pairwise exchanges per round = O ( P 2 ) … All-to-all 7: 0111 exchange words sent per exchange = O ( n / P 2 ) 8: 1000 … 9: 1001 total words sent = O ( n ) 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log (p): No comm log (n/p): No comm “Transpose” (p=4) Tuesday, September 18, 12

  21. 0: 0000 1: 0001 2: 0010 3: 0011 4: 0100 5: 0101 6: 0110 … All-to-all 7: 0111 exchange 8: 1000 … 9: 1001 10: 1010 11: 1011 12: 1100 13: 1101 14: 1110 15: 1111 log (p): No comm log (n/p): No comm “Transpose” (p=4) Tuesday, September 18, 12

  22. Cyclic Block 0 1 2 3 0 4 8 12 All-to-all 4 5 6 7 1 5 9 13 exchange ≡ 8 9 10 11 2 6 10 14 Matrix transpose 12 13 14 15 3 7 11 15 Tuesday, September 18, 12

  23. “Binary exchange” algorithm (block or cyclic): rounds of communication = O (log n ) number of pairwise exchanges per round = O ( P ) total number of pairwise exchanges = O ( P log n ) words sent per exchange = O ( n / P ) total words sent = O ( n log n ) “Transpose” algorithm (cyclic → all-to-all → block): rounds of communication = 1 number of pairwise exchanges per round = O ( P 2 ) total number of pairwise exchanges = O ( P 2 ) words sent per exchange = O ( n / P 2 ) total words sent = O ( n ) Tuesday, September 18, 12

  24. ๏ More CUDA tuning: Occupancy and ILP References: http://developer.nvidia.com/cuda/get-started-cuda-cc http://developer.download.nvidia.com/CUDA/training/cuda_webinars_WarpsAndOccupancy.pdf http://www.cs.berkeley.edu/~volkov/volkov10-GTC.pdf http://www.cs.berkeley.edu/~volkov/volkov11-unrolling.pdf Tuesday, September 18, 12

  25. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  26. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  27. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  28. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  29. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  30. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  31. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  32. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  33. /opt/cuda-4.0/cuda/bin/nvcc -arch=sm_20 --ptxas-options=-v -O3 \ -o bitmerge-cuda.o -c bitmerge-cuda.cu ptxas info : Compiling entry function '_Z12bitonicSplitjPfj' for 'sm_20' ptxas info : Function properties for _Z12bitonicSplitjPfj 0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads ptxas info : Used 8 registers , 52 bytes cmem[0] icpc -O3 -g -o bitmerge timer.o bitmerge.o bitmerge-seq.o \ bitmerge-cilk.o bitmerge-cuda.o \ -L/opt/cuda-4.0/cuda/bin/../lib64 \ -Wl,-rpath /opt/cuda-4.0/cuda/bin/../lib64 -lcudart Tuesday, September 18, 12

  34. Occupancy Limiters: Registers Register usage: compile with --ptxas-options=-v Fermi has 32K registers per SM Example 1 Kernel uses 20 registers per thread (+1 implicit) Active threads = 32K/21 = 1560 threads > 1536 thus an occupancy of 1 Example 2 Kernel uses 63 registers per thread (+1 implicit) Active threads = 32K/64 = 512 threads 512/1536 = .3333 occupancy Can control register usage with the nvcc flag: --maxrregcount Occupancy = (Active warps) / (Max active warps) Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  35. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  36. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  37. https://piazza.com/class#fall2012/cse6230/52 Tuesday, September 18, 12

  38. Recall: Reduction example Tuesday, September 18, 12

  39. Recall: Reduction example Tuesday, September 18, 12

  40. Recall: Reduction example Tuesday, September 18, 12

  41. Recall: Reduction example b = 256 threads/block ⇒ shmem = 256 * (4 Bytes/ int ) = 1024 Bytes Tuesday, September 18, 12

  42. Occupancy Limiters: Shared Memory Shared memory usage: compile with --ptxas-options=-v Reports shared memory per block Fermi has either 16K or 48K shared memory Example 1, 48K shared memory Kernel uses 32 bytes of shared memory per thread 48K/32 = 1536 threads occupancy=1 Example 2, 16K shared memory Kernel uses 32 bytes of shared memory per thread 16K/32 = 512 threads occupancy=.3333 Don’t use too much shared memory Choose L1/Shared config appropriately. Occupancy = (Active warps) / (Max active warps) Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

  43. Occupancy Occupancy = Active Warps / Maximum Active Warps Remember: resources are allocated for the entire block Resources are finite Utilizing too many resources per thread may limit the occupancy Potential occupancy limiters: Register usage Shared memory usage Block size Jinx’s Fermi GPUs: 48 max active warps/SM, 32 threads/warp Tuesday, September 18, 12

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Introduction to CUDA C What is CUDA? CUDA Architecture Expose general-purpose GPU

Introduction to CUDA C What is CUDA? CUDA Architecture Expose general-purpose GPU

Introduction to CUDA C What is CUDA? CUDA Architecture Expose general-purpose GPU computing as first-class capability Retain traditional DirectX/OpenGL graphics performance CUDA C Based on industry-standard C A handful of

1.31k views • 62 slides
Guided Profiling for Auto-Tuning Array Layouts on GPUs Nicolas Weber, Sandra C. Amend and Michael

Guided Profiling for Auto-Tuning Array Layouts on GPUs Nicolas Weber, Sandra C. Amend and Michael

Guided Profiling for Auto-Tuning Array Layouts on GPUs Nicolas Weber, Sandra C. Amend and Michael Goesele TU Darmstadt Motivation Memory access is one of the most important performance factors in CUDA applications CUDA Programming Guide

644 views • 36 slides
PAC PACE AUT AUTO-WER WERKS KS Vehicle Tuning Services Performance tuning with fuel

PAC PACE AUT AUTO-WER WERKS KS Vehicle Tuning Services Performance tuning with fuel

PAC PACE AUT AUTO-WER WERKS KS Vehicle Tuning Services Performance tuning with fuel efficiency Com pany Profile Viezu specialize in electronic engine tuning, rolling roads, diagnostic and test equipment Dynocom EU launched 2010;

281 views • 15 slides
Performance Target Tesla M2090 512 CUDA processors @ 1.3 GHz each processor is capable

Performance Target Tesla M2090 512 CUDA processors @ 1.3 GHz each processor is capable

CUDA Deep Dive Performance CSE 6230 Jee Choi September 10, 2013 Performance Target Tesla M2090 512 CUDA processors @ 1.3 GHz each processor is capable of issuing 1 FMA (2 flops) instruction per cycle throughput = 512 1.3

756 views • 58 slides
Performance Tuning best pracitces and performance monitoring with Zabbix Andrew Nelson Senior

Performance Tuning best pracitces and performance monitoring with Zabbix Andrew Nelson Senior

Performance Tuning best pracitces and performance monitoring with Zabbix Andrew Nelson Senior Linux Consultant May 28, 2015 NLUUG Conf, Utrecht, Netherlands Overview Introduction Performance tuning is Science! A little Law and

595 views • 47 slides
Plan Optimizing Matrix Transpose with CUDA 1 CS4402-9535: High-Performance Computing with CUDA

Plan Optimizing Matrix Transpose with CUDA 1 CS4402-9535: High-Performance Computing with CUDA

Plan Optimizing Matrix Transpose with CUDA 1 CS4402-9535: High-Performance Computing with CUDA Performance Optimization 2 Marc Moreno Maza Parallel Reduction 3 University of Western Ontario, London, Ontario (Canada) Parallel Scan 4

572 views • 29 slides
Performance Factors and Tuning Tips for JavaScript RIA

Performance Factors and Tuning Tips for JavaScript RIA

PUG Challenge Americas 2011 Performance Factors and Tuning Tips for JavaScript RIA Frameworks Presented by Dustin Grau 6/5/11-6/8/11 Performance Tuning

701 views • 44 slides
CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement

CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement

CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement Learning Yan li, Kenneth Chang, Oceane Bel, Ethan L. Miller, Darrel D. E. Long Performance Tuning Tuning systems parameters for high

343 views • 15 slides
Sanjay Rajopadhye Colorado State University n Class objectives, goals, introduction n CUDA

Sanjay Rajopadhye Colorado State University n Class objectives, goals, introduction n CUDA

Sanjay Rajopadhye Colorado State University n Class objectives, goals, introduction n CUDA performance tuning (wrap up) n Equational Programming (intro) 2 n Parallel Programming is hard n End of the free lunch [Sut05] n

284 views • 16 slides
Automatically Tuning Performance and Power for GPUs Jeffrey K. Hollingsworth What is

Automatically Tuning Performance and Power for GPUs Jeffrey K. Hollingsworth What is

Automatically Tuning Performance and Power for GPUs Jeffrey K. Hollingsworth What is Auto-tuning? Making programs better based on empirical feedback from observed runs without programmers having to in the loop. What to Tune

363 views • 13 slides
Lecture 2.1 - Introduction to CUDA C CUDA C vs. Thrust vs. CUDA Libraries Objective To learn

Lecture 2.1 - Introduction to CUDA C CUDA C vs. Thrust vs. CUDA Libraries Objective To learn

GPU Teaching Kit Accelerated Computing Lecture 2.1 - Introduction to CUDA C CUDA C vs. Thrust vs. CUDA Libraries Objective To learn the main venues and developer resources for GPU computing Where CUDA C fits in the big picture 2 3

1.28k views • 12 slides
Bitonic st -orderings for Upward Planar Graphs Martin Gronemann University of Cologne, Germany

Bitonic st -orderings for Upward Planar Graphs Martin Gronemann University of Cologne, Germany

Bitonic st -orderings for Upward Planar Graphs Martin Gronemann University of Cologne, Germany September 20, 2016 Introduction Graph Drawing 2014 Bitonic st-orderings of biconnected planar graphs. A special st -ordering that is similar

729 views • 44 slides
Lecture 6.2 Performance Considerations Memory Coalescing in CUDA Objective To learn that

Lecture 6.2 Performance Considerations Memory Coalescing in CUDA Objective To learn that

GPU Teaching Kit Accelerated Computing Lecture 6.2 Performance Considerations Memory Coalescing in CUDA Objective To learn that memory coalescing is important for effectively utilizing memory bandwidth in CUDA Its origin in DRAM

623 views • 13 slides
GPGPU 2015: High Performance Tutorial contents for today [118 slides] Computing with CUDA

GPGPU 2015: High Performance Tutorial contents for today [118 slides] Computing with CUDA

GPGPU 2015: High Performance Tutorial contents for today [118 slides] Computing with CUDA Department of Computer Science. University of Cape Town April 20 th -24 th , 2015 1. Introduction. [17 slides] 2. Architecture. [21] 1. CUDA hardware

503 views • 38 slides
Gerrit Performance Tuning How to Properly Tune and Size your Gerrit Backend 1 Git Sizing /

Gerrit Performance Tuning How to Properly Tune and Size your Gerrit Backend 1 Git Sizing /

Gerrit Performance Tuning How to Properly Tune and Size your Gerrit Backend 1 Git Sizing / Performance Tuning - FAQ How many servers will I need? Which cloning protocols to offer? How to set those gazillion gerrit.config options?

731 views • 51 slides
CHAPTER 9: PID TUNING Process Solve the tuning Apply, is the reaction curve problem. Requires

CHAPTER 9: PID TUNING Process Solve the tuning Apply, is the reaction curve problem. Requires

CHAPTER 9: PID TUNING Process Solve the tuning Apply, is the reaction curve problem. Requires a performance computer program. good? 1.5 1 0.5 0 -0.5 0 5 101520253035404550 COMBINED DEFINITION OF TUNING 1 0.8 v 0.6 First

753 views • 22 slides
THE MOD METHOD with VESPERS MASTERING In this Module What mastering can do & what it

THE MOD METHOD with VESPERS MASTERING In this Module What mastering can do & what it

THE MOD METHOD with VESPERS MASTERING In this Module What mastering can do & what it cant Self-mastering vs. third-party mastering Picking a mastering engineer Mastering work fl ow Audio artifacts & fi delity:

520 views • 16 slides
Figure 2.14 from page 63 of Exploring the Heart of Ma2er

Figure 2.14 from page 63 of Exploring the Heart of Ma2er

Figure 2.14 from page 63 of Exploring the Heart of Ma2er FIGURE 2.14 The fusion probability of helium-3 (3He) and helium-4 (4He), an important reaction in stars affecting neutrino production in the

160 views • 12 slides
Design of an experimental set-up to analyse compliant mechanisms used for the deployment of a

Design of an experimental set-up to analyse compliant mechanisms used for the deployment of a

I NTRODUCTION S ET - UP FE MODEL I DENTIFICATION V ALIDATION C ONCLUSIONS Design of an experimental set-up to analyse compliant mechanisms used for the deployment of a panel Florence Dewalque, Olivier Brls Department of Aerospace and

383 views • 22 slides
Postdoc event Science, Engineering & Technology Group June 28, 2019 Thermotechnisch

Postdoc event Science, Engineering & Technology Group June 28, 2019 Thermotechnisch

Postdoc event Science, Engineering & Technology Group June 28, 2019 Thermotechnisch Instituut Ann Gevers & Stefan Wellens (HR dept) Agenda for this session Who are we as postdocs? An overview of the population and its categories

1.15k views • 58 slides
Positivity- -preserving Lagrangian schemes for preserving Lagrangian schemes for Positivity

Positivity- -preserving Lagrangian schemes for preserving Lagrangian schemes for Positivity

Sino-German Symposium on Advanced Numerical Methods for Compressible Fluid Mechanics and Related Problems, May 22-26, 2014 Beijing Positivity- -preserving Lagrangian schemes for preserving Lagrangian schemes for Positivity multi-

746 views • 53 slides
Spectroscopic applications for plasma-wall interaction observations in fusion devices Kalle

Spectroscopic applications for plasma-wall interaction observations in fusion devices Kalle

Spectroscopic applications for plasma-wall interaction observations in fusion devices Kalle Heinola Joint ICTP-IAEA School on Atomic and Molecular Spectroscopy in Plasmas 6 10 May, 2019, Trieste, Italy Outline 1. Introduction a)

628 views • 36 slides
at sub-25 ps precision: the PICOSEC detection concept F.J. Iguaz On behalf of PICOSEC

at sub-25 ps precision: the PICOSEC detection concept F.J. Iguaz On behalf of PICOSEC

Charged particle timing at sub-25 ps precision: the PICOSEC detection concept F.J. Iguaz On behalf of PICOSEC collaboration PM2018, 28 th May 2018 (*) iguaz@cea.fr PICOSEC detection concept 1 F.J. Iguaz, PM2018, 28th May 2018 PICOSEC

954 views • 39 slides
QuadVtx vertex reconstruction DUNE FD sim/reco meeting 2 Sep, 2019 Chris Backhouse University

QuadVtx vertex reconstruction DUNE FD sim/reco meeting 2 Sep, 2019 Chris Backhouse University

QuadVtx vertex reconstruction DUNE FD sim/reco meeting 2 Sep, 2019 Chris Backhouse University College London 1 / 16 Introduction Independent vertex can be useful as input to other algorithms Presenting a very simple hit-based approach

544 views • 22 slides

More recommend