1 n n 1 microbenchmark logp prediction 1 n n 1 benchmark
play

1:n n:1 Microbenchmark LogP Prediction 1:n n:1 Benchmark Results A - PowerPoint PPT Presentation

Feb 13, 2024 •159 likes •447 views

Architectural Specialities of InfiniBand TM A new Barrier Algorithm for InfiniBand TM Results and Conclusions Fast Barrier Synchronization for InfiniBand TM Torsten Hoefler Chair of Computer Architecture Technical University of Chemnitz

  1. Architectural Specialities of InfiniBand TM A new Barrier Algorithm for InfiniBand TM Results and Conclusions Fast Barrier Synchronization for InfiniBand TM Torsten Hoefler Chair of Computer Architecture Technical University of Chemnitz IPDPS’06 - CAC’06 Workshop Rhodes Island, Greece 25th April 2006 Torsten Hoefler n-way Dissemination

  2. Architectural Specialities of InfiniBand TM A new Barrier Algorithm for InfiniBand TM Results and Conclusions Architectural Specialities of InfiniBand TM 1 1:n n:1 Microbenchmark LogP Prediction 1:n n:1 Benchmark Results A new Barrier Algorithm for InfiniBand TM 2 The Dissemination Algorithm The n-way Dissemination Algorithm Results and Conclusions 3 Comparison with other MPI_Barrier Implementations Conclusions and Future Work Torsten Hoefler n-way Dissemination

  3. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results Architectural Specialities of InfiniBand TM 1 1:n n:1 Microbenchmark LogP Prediction 1:n n:1 Benchmark Results A new Barrier Algorithm for InfiniBand TM 2 The Dissemination Algorithm The n-way Dissemination Algorithm Results and Conclusions 3 Comparison with other MPI_Barrier Implementations Conclusions and Future Work Torsten Hoefler n-way Dissemination

  4. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results 1:n n:1 Microbenchmark Developed to analyze the InfiniBand TM network Especially for collective communication Measures single message performance (RDTSC) MPI based Supports (nearly) all transport types Torsten Hoefler n-way Dissemination

  5. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results 1:n n:1 Microbenchmark - principle 1 1 ping pong 2 2 0 0 . . . . . . P P (0): Take time 1 (1..n-1): Send a single message to n-1 hosts 2 (1..n-1): Hosts respond immediately 3 (0): Wait for message receiption from all hosts 4 (0): Take time 5 Torsten Hoefler n-way Dissemination

  6. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results Architectural Specialities of InfiniBand TM 1 1:n n:1 Microbenchmark LogP Prediction 1:n n:1 Benchmark Results A new Barrier Algorithm for InfiniBand TM 2 The Dissemination Algorithm The n-way Dissemination Algorithm Results and Conclusions 3 Comparison with other MPI_Barrier Implementations Conclusions and Future Work Torsten Hoefler n-way Dissemination

  7. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results The LogP Model LogP model by Culler et.al. 1993 LogP Parameters L - Latency g - Bandwidth-limiting Gap between consecutive messages ( g ≈ 1 / BW ) o - Send-/Receive Overhead P - Number of involved Processors Torsten Hoefler n-way Dissemination

  8. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results LogP Prediction RTT(P) P max{g,o} 1 P RTT ( P ) / P = ( 4 o + 2 L + ( P − 1 ) · max { g , o } ) / P Torsten Hoefler n-way Dissemination

  9. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results Architectural Specialities of InfiniBand TM 1 1:n n:1 Microbenchmark LogP Prediction 1:n n:1 Benchmark Results A new Barrier Algorithm for InfiniBand TM 2 The Dissemination Algorithm The n-way Dissemination Algorithm Results and Conclusions 3 Comparison with other MPI_Barrier Implementations Conclusions and Future Work Torsten Hoefler n-way Dissemination

  10. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results 1:n n:1 Benchmark Results Test Environment 8 Nodes Dual Xeon 2.066 GHz Red Hat Linux release 9 (Shrike) Kernel: 2.4.27 SMP HCA: Mellanox ”Cougar” (MTPB 23108) Torsten Hoefler n-way Dissemination

  11. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results 1:n n:1 Benchmark Results 16 RDMA Write RDMA Write inline Minimal RTT in Microseconds 14 12 10 8 6 4 0 10 20 30 40 50 60 70 # Processors (P) RDMA/W - fastest transport type in our tests Graph shows minimal values We benefit from sending to multiple hosts simultaneously Atomic was not available on our HCAs Torsten Hoefler n-way Dissemination

  12. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results 1:n n:1 Benchmark Results 24 RDMA Write RDMA Write inline 22 Average RTT in Microseconds 20 18 16 14 12 10 8 6 5 10 15 20 25 30 35 40 45 50 # Processors (P) Average Graph has many outliers Still same ”shape” Torsten Hoefler n-way Dissemination

  13. Architectural Specialities of InfiniBand TM 1:n n:1 Microbenchmark A new Barrier Algorithm for InfiniBand TM LogP Prediction Results and Conclusions 1:n n:1 Benchmark Results A possible Explanation - The LoP Model RTT(P) P t saturated t processing P pipeline processing saturation Pipeline startup function - hardware pipe, caches Minimal processing time - hardware Network saturation - network hardware / transceiver Torsten Hoefler n-way Dissemination

  14. Architectural Specialities of InfiniBand TM The Dissemination Algorithm A new Barrier Algorithm for InfiniBand TM The n-way Dissemination Algorithm Results and Conclusions Architectural Specialities of InfiniBand TM 1 1:n n:1 Microbenchmark LogP Prediction 1:n n:1 Benchmark Results A new Barrier Algorithm for InfiniBand TM 2 The Dissemination Algorithm The n-way Dissemination Algorithm Results and Conclusions 3 Comparison with other MPI_Barrier Implementations Conclusions and Future Work Torsten Hoefler n-way Dissemination

  15. Architectural Specialities of InfiniBand TM The Dissemination Algorithm A new Barrier Algorithm for InfiniBand TM The n-way Dissemination Algorithm Results and Conclusions The Dissemination Algorithm Round 0 Round 1 Round 2 Logarithmic running time ( O ( log 2 P ) ) Works with non-power of two P Torsten Hoefler n-way Dissemination

  16. Architectural Specialities of InfiniBand TM The Dissemination Algorithm A new Barrier Algorithm for InfiniBand TM The n-way Dissemination Algorithm Results and Conclusions Dissemination - Peer selection Round 0 Round 1 Round 2 speer = ( p + 2 r ) mod P rpeer = ( p − 2 r ) mod P Torsten Hoefler n-way Dissemination

  17. Architectural Specialities of InfiniBand TM The Dissemination Algorithm A new Barrier Algorithm for InfiniBand TM The n-way Dissemination Algorithm Results and Conclusions Architectural Specialities of InfiniBand TM 1 1:n n:1 Microbenchmark LogP Prediction 1:n n:1 Benchmark Results A new Barrier Algorithm for InfiniBand TM 2 The Dissemination Algorithm The n-way Dissemination Algorithm Results and Conclusions 3 Comparison with other MPI_Barrier Implementations Conclusions and Future Work Torsten Hoefler n-way Dissemination

  18. Architectural Specialities of InfiniBand TM The Dissemination Algorithm A new Barrier Algorithm for InfiniBand TM The n-way Dissemination Algorithm Results and Conclusions The n-way Dissemination Algorithm Round 0 Round 1 Logarithmic running time ( O ( log 2 P ) − O ( log n P )? ) Works with non-power of n P Torsten Hoefler n-way Dissemination

  19. Architectural Specialities of InfiniBand TM The Dissemination Algorithm A new Barrier Algorithm for InfiniBand TM The n-way Dissemination Algorithm Results and Conclusions n-way Dissemination - Peer selection Round 0 Round 1 speer i = ( p + i · ( n + 1 ) r ) mod P rpeer i = ( p − i · ( n + 1 ) r ) mod P Torsten Hoefler n-way Dissemination

  20. Architectural Specialities of InfiniBand TM Comparison with other MPI_Barrier Implementations A new Barrier Algorithm for InfiniBand TM Conclusions and Future Work Results and Conclusions Architectural Specialities of InfiniBand TM 1 1:n n:1 Microbenchmark LogP Prediction 1:n n:1 Benchmark Results A new Barrier Algorithm for InfiniBand TM 2 The Dissemination Algorithm The n-way Dissemination Algorithm Results and Conclusions 3 Comparison with other MPI_Barrier Implementations Conclusions and Future Work Torsten Hoefler n-way Dissemination

  21. Architectural Specialities of InfiniBand TM Comparison with other MPI_Barrier Implementations A new Barrier Algorithm for InfiniBand TM Conclusions and Future Work Results and Conclusions The n-way Dissemination Algorithm Implementation Details Implementation as collv1 component in Open MPI Communication peers are precomputed Benchmark Details LAM/MPI 7.1.1 TUC SHIBA 1.0 MVAPICH 0.9.4 Torsten Hoefler n-way Dissemination

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Embedded Databases MicroBenchmark Team CodeBlooded Refinement of goals Try and understand

Embedded Databases MicroBenchmark Team CodeBlooded Refinement of goals Try and understand

Embedded Databases MicroBenchmark Team CodeBlooded Refinement of goals Try and understand the usefulness of a DB in terms of a particular application domain Goal : Build a DB calculator that works with the micro-benchmark Embedded

349 views • 33 slides
Early Prediction of Product Performance and Yield Via Technology Benchmark Choongyeun Cho 1 ,

Early Prediction of Product Performance and Yield Via Technology Benchmark Choongyeun Cho 1 ,

Early Prediction of Product Performance and Yield Via Technology Benchmark Choongyeun Cho 1 , Daeik D. Kim 1 , Jonghae Kim 2 , Daihyun Lim 1 , Sangyeun Cho 3 1 IBM, 2 Qualcomm, 3 U. Pittsburgh Background Process variation as grave concern:

276 views • 14 slides
A practical Approach to the Rating of Barrier Algorithms using the LogP Model and Open MPI

A practical Approach to the Rating of Barrier Algorithms using the LogP Model and Open MPI

LogP Predictions Implementation Motivation Conclusions A practical Approach to the Rating of Barrier Algorithms using the LogP Model and Open MPI Torsten Hfler, Wolfgang Rehm TU Chemnitz, Germany 24.05.2005 Torsten Hfler, Wolfgang Rehm

571 views • 22 slides
One-Step Ahead Prediction of the Wiener-Hammerstein Benchmark with Process Noise using Kernel

One-Step Ahead Prediction of the Wiener-Hammerstein Benchmark with Process Noise using Kernel

One-Step Ahead Prediction of the Wiener-Hammerstein Benchmark with Process Noise using Kernel Adaptive Learning Rishi Relan Dieter Verbecke Koen Tiels Department ELEC 12.05.2017 Idea of this workshop: Nonlinear System 2 Idea of this

356 views • 21 slides
PCERE: Fine-grained Parallel Benchmark Decomposition for Scalability Prediction Mihail Popov,

PCERE: Fine-grained Parallel Benchmark Decomposition for Scalability Prediction Mihail Popov,

PCERE: Fine-grained Parallel Benchmark Decomposition for Scalability Prediction Mihail Popov, Chadi Akel, Florent Conti, William Jalby, Pablo de Oliveira Castro UVSQ - PRiSM - ECR Mai 28, 2015 Introduction Evaluate strong scalability

450 views • 27 slides
KEEPING IT REAL WHY HPC DATA SERVICES DONT ACHIEVE MICROBENCHMARK PERFORMANCE e rh tjh

KEEPING IT REAL WHY HPC DATA SERVICES DONT ACHIEVE MICROBENCHMARK PERFORMANCE e rh tjh

5 TH INTERNATIONAL PARALLEL DATA SYSTEMS WORKSHOP KEEPING IT REAL WHY HPC DATA SERVICES DONT ACHIEVE MICROBENCHMARK PERFORMANCE e rh tjh tyh y PHIL CARNS 1 , KEVIN HARMS 1 , BRAD SETTLEMYER 2 , BRIAN ATKINSON 2 , AND ROB ROSS 1

957 views • 27 slides
LogP: Towards a Realistic Model of Parallel Computation y David Culler, Richard Karp ,

LogP: Towards a Realistic Model of Parallel Computation y David Culler, Richard Karp ,

LogP: Towards a Realistic Model of Parallel Computation y David Culler, Richard Karp , David Patterson, Abhijit Sahay, Klaus Erik Schauser, Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken Computer Science Division, University

476 views • 24 slides
Beyond Macrobenchmarks Microbenchmark-based Graph Database Evaluation Matteo Lissandrini, Martin

Beyond Macrobenchmarks Microbenchmark-based Graph Database Evaluation Matteo Lissandrini, Martin

Beyond Macrobenchmarks Microbenchmark-based Graph Database Evaluation Matteo Lissandrini, Martin Brugnara, Yannis Velegrakis Universiteit Utrecth Knowledge Graph Protein Interaction Road Network Network Graphs are Everywhere Social

1.05k views • 28 slides
Establishing Realistic Investment Earnings Benchmarks What is a Benchmark? A benchmark is a

Establishing Realistic Investment Earnings Benchmarks What is a Benchmark? A benchmark is a

Establishing Realistic Investment Earnings Benchmarks What is a Benchmark? A benchmark is a standard reference point that is used to compare the results of similar investments over time. Institutional investors use benchmarks to analyze

543 views • 39 slides
The HPC Challenge Benchmark The HPC Challenge Benchmark http://icl.cs.utk.edu/hpcc/ Jack

The HPC Challenge Benchmark The HPC Challenge Benchmark http://icl.cs.utk.edu/hpcc/ Jack

The HPC Challenge Benchmark The HPC Challenge Benchmark http://icl.cs.utk.edu/hpcc/ Jack Dongarra Innovative Computing Laboratory University of Tennessee and Computer Science and Mathematics Division Oak Ridge National Laboratory 1 Phases

119 views • 10 slides
The HPC Challenge Benchmark: The HPC Challenge Benchmark: A Candidate for Replacing A Candidate

The HPC Challenge Benchmark: The HPC Challenge Benchmark: A Candidate for Replacing A Candidate

2007 SPEC Benchmark Workshop January 21, 2007 Radisson Hotel Austin North The HPC Challenge Benchmark: The HPC Challenge Benchmark: A Candidate for Replacing A Candidate for Replacing LINPACK in the TOP500? LINPACK in the TOP500? Jack

1.02k views • 47 slides
Using Stata 16s lasso features for prediction and inference Di Liu StataCorp 1 / 50

Using Stata 16s lasso features for prediction and inference Di Liu StataCorp 1 / 50

Using Stata 16s lasso features for prediction and inference Di Liu StataCorp 1 / 50 Motivation I: Prediction What is a prediction? Prediction is to predict an outcome variable on new (unseen) data Good prediction minimizes mean-squared

762 views • 53 slides
Medicaid Benchmark Options Analysis Stakeholder Advisory Committee July 23, 2012 Overview

Medicaid Benchmark Options Analysis Stakeholder Advisory Committee July 23, 2012 Overview

Medicaid Benchmark Options Analysis Stakeholder Advisory Committee July 23, 2012 Overview Legal Requirements for Medicaid Benchmark Open Policy Questions Considerations for Designing Medicaid Benchmark 2 What Benchmark are We

338 views • 29 slides
Using lasso and related estimators for prediction Di Liu StataCorp July 12, 2019 1 / 20

Using lasso and related estimators for prediction Di Liu StataCorp July 12, 2019 1 / 20

Using lasso and related estimators for prediction Di Liu StataCorp July 12, 2019 1 / 20 Prediction What is a prediction? Prediction is to predict an outcome variable on new (unseen) data Good prediction minimizes mean-squared error (or other

217 views • 20 slides
Joint Joint Doctrine Doctrine Ontology as Ontology as Benchmark fo Benchmark for Military r

Joint Joint Doctrine Doctrine Ontology as Ontology as Benchmark fo Benchmark for Military r

Joint Joint Doctrine Doctrine Ontology as Ontology as Benchmark fo Benchmark for Military r Military Informat Information ion Systems Interoper Systems Interoperability ability Barry Smith National Center for Ontological Research

448 views • 44 slides
The HPC Challenge Benchmark: The HPC Challenge Benchmark: A Candidate for Replacing A Candidate

The HPC Challenge Benchmark: The HPC Challenge Benchmark: A Candidate for Replacing A Candidate

The HPC Challenge Benchmark: The HPC Challenge Benchmark: A Candidate for Replacing A Candidate for Replacing LINPACK in the TOP500? LINPACK in the TOP500? Jack Dongarra University of Tennessee and Oak Ridge National Laboratory 1 Outline

785 views • 22 slides
The Promise of Sensor Networks to Revolutionize our Environment: Applications and Research Aris

The Promise of Sensor Networks to Revolutionize our Environment: Applications and Research Aris

The Promise of Sensor Networks to Revolutionize our Environment: Applications and Research Aris M. Ouksel The University of Illinois at Chicago aris@uic.edu ICPS-06 The IEEE International Conference on Pervasive Services Outline

1.05k views • 80 slides
Programming Wireless Sensor Networks Francisco Martins University of Lisbon MiNEMA Winter

Programming Wireless Sensor Networks Francisco Martins University of Lisbon MiNEMA Winter

Programming Wireless Sensor Networks Francisco Martins University of Lisbon MiNEMA Winter School, Gteborg, Sweden,March 23-26, 2009. 1 Summary Sensor networks overview Hardware and Operating System Program Languages and Systems

796 views • 57 slides
The Lakota Program A brief history of the company and the UH-72A (Lakota). Lloyd Reaves 14 July

The Lakota Program A brief history of the company and the UH-72A (Lakota). Lloyd Reaves 14 July

The Aerospace & Defense Forum Dallas-Ft. Dallas Chapter July 13, 2017 The Lakota Program A brief history of the company and the UH-72A (Lakota). Lloyd Reaves 14 July 2017 A Leader in Aerospace, Defense and Related Services

245 views • 9 slides
Summary of PNNL Visit Project-X and Nuclear Energy Shekhar Mishra Steve Holmes Bob Tschirhart

Summary of PNNL Visit Project-X and Nuclear Energy Shekhar Mishra Steve Holmes Bob Tschirhart

Summary of PNNL Visit Project-X and Nuclear Energy Shekhar Mishra Steve Holmes Bob Tschirhart Pat Hurh Pacific Northwest National Laboratory Participants Mike Thompson, Deputy Director, Fundamental & Computation Sciences Mary Peterson,

491 views • 14 slides
COMP26120: Algorithms and Imperative Programming Lecture C2: C - Simple Data Structures Pete

COMP26120: Algorithms and Imperative Programming Lecture C2: C - Simple Data Structures Pete

COMP26120: Algorithms and Imperative Programming Lecture C2: C - Simple Data Structures Pete Jinks School of Computer Science, University of Manchester Autumn 2011 COMP26120 Lecture C2 1/28 Review What have you learnt about C in the lab?

878 views • 54 slides
Humans Impact on Earth Ecological Footprint Negative Human Impacts Positive Human

Humans Impact on Earth Ecological Footprint Negative Human Impacts Positive Human

Slide 1 / 143 Slide 2 / 143 5th Grade Human Impacts on Earth 2015-08-28 www.njctl.org Slide 3 / 143 Slide 4 / 143 Table of Contents: Human Impacts on Earth Click on the topic to go to that section Human Impacts and Importance of the

581 views • 24 slides
Biosens II Background Biosens II Subproject 2.3 Optimal replacement policies for dairy

Biosens II Background Biosens II Subproject 2.3 Optimal replacement policies for dairy

Biosens II Background Biosens II Subproject 2.3 Optimal replacement policies for dairy cows based on Overview daily yield measurements MDP intro Dairy HMDP Model Case example: Markov decision processes State space model Embedding

67 views • 6 slides
CS 744: MAPREDUCE Shivaram Venkataraman Fall 2019 ANNOUNCEMENTS Assignment 1 out

CS 744: MAPREDUCE Shivaram Venkataraman Fall 2019 ANNOUNCEMENTS Assignment 1 out

CS 744: MAPREDUCE Shivaram Venkataraman Fall 2019 ANNOUNCEMENTS Assignment 1 out CloudLab notes on Piazza No teams yet? Applications Machine Learning SQL Streaming Graph Computational Engines Scalable Storage Systems Resource

825 views • 24 slides

More recommend