with proprietary interconnect and
play

with Proprietary Interconnect and Its Programming and Applications - PowerPoint PPT Presentation

Apr 07, 2024 •309 likes •912 views

Tightly Coupled Accelerators with Proprietary Interconnect and Its Programming and Applications Toshihiro Hanawa Information Technology Center, The University of Tokyo Taisuke Boku Center for Computational Sciences, University of Tsukuba

  1. Tightly Coupled Accelerators with Proprietary Interconnect and Its Programming and Applications Toshihiro Hanawa Information Technology Center, The University of Tokyo Taisuke Boku Center for Computational Sciences, University of Tsukuba Collaboration with Yuetsu Kodama, Mitsuhisa Sato, Masayuki Umemura @ CCS, Univ. of Tsukuba Hitoshi Murai @ Riken AICS, Hideharu Amano @ Keio Univ. GPU Technology Conference 2015 1 Mar. 19, 2015

  2. Agenda  Background  Application Examples  HA-PACS / AC-Crest Project  QUDA (QCD)  FFTE (FFT)  Introduction of HA-PACS / TCA  Introduction of XcalableACC  Organization of TCA  Concept  PEACH2 Board designed for TCA  Code Examples  Evaluation of Basic  Evaluations Performance  Summary  Collective Communications  Implementation Examples  Performance Evaluation GPU Technology Conference 2015 2 Mar. 19, 2015

  3. Current Trend of HPC using GPU Computing  Advantageous Features High peak performance / cost ratio  High peak performance / power ratio   Examples of HPC System:  GPU Clusters in Green500 (Nov.  GPU Clusters and MPPs in TOP500 2014) (“Greenest” Supercomputers (Nov. 2014) ranked in Top500)  2nd: Titan (NVIDIA K20X, 27 PF)  3rd: TSUBAME-KFC (NVIDIA K20X, 4.4  6th: Piz Daint (NVIDIA K20X, 7.8 PF) GF/W)  10th: Cray CS-Storm (NVIDIA K40, 6.1  4th: Cray Storm1 (NVIDIA K40, 3.9 GF / PF) W)  15th: TSUBAME2.5 (NVIDIA K20X, 5.6  7th: HA-PACS/TCA (NVIDIA K20X, 3.5 PF) GF/W)  48 systems use NVIDIA GPUs.  8 systems of Top10 use NVIDIA GPUs. GPU Technology Conference 2015 3 Mar. 19, 2015

  4. Issues of GPU Computing  Data I/O performance limitation Ex) K20X: PCIe gen2 x16  Peak Performance: 8GB/s (I/O) ⇔ 1.3 TFLOPS (Computation) Communication bottleneck becomes significant on multi GPU application   Strong-scaling on GPU cluster Important to shorten Turn-Around Time of production-run  Heavy impact of communication latency   Ultra- low latency between GPUs is important for next generation’s HPC Our target is developing a direct communication system between external GPUs for a feasibility study for future accelerated computing. ⇒ “Tightly Coupled Accelerators (TCA)” architecture GPU Technology Conference 2015 4 Mar. 19, 2015

  5. HA-PACS Project HA- PACS is not only a “commodity GPU HA-PACS (Highly Accelerated Parallel  cluster ” but also experiment platform Advanced system for Computational Sciences) HA-PACS base cluster  for development of GPU-accelerated code 8 th generation of PAX/PACS series   for target fields, and performing product-run supercomputer in University of Tsukuba Now in operation since Feb. 2012  HA-PACS/TCA (TCA = Tightly Coupled FY2011-2013, operation until   Accelerators) FY2016(?) for elementary research on direct Promotion of computational science   communication technology for accelerated applications in key areas in CCS- computing Tsukuba Our original communication chip named  Target field: QCD, astrophysics, “PEACH2” was installed in each node.  QM/MM (quantum mechanics / Now in operation since Nov. 2013  molecular mechanics, bioscience) GPU Technology Conference 2015 5 Mar. 19, 2015

  6. AC-CREST project  Project “ Research and Development on Unified Environment of Accelerated Computing and Interconnection for Post-Petascale Era ” ( AC-CREST )  Objectives Supported by JST-CREST  Realization of high-performance (direct)  “ Development of System communication among accelerators Software Technologies for Development of system software supporting  post-Peta Scale High communication system among accelerators Performance Computing ” Development of parallel language and  program compilers  Higher productivity  Highly optimized (offload, communication) Development of practical applications  GPU Technology Conference 2015 6 Mar. 19, 2015

  7. What is “Tightly Coupled Accelerators (TCA)” ? Concept:  Direct connection between accelerators (GPUs) over the nodes without CPU assistance  Eliminate extra memory copies to the host  Reduce latency, improve strong scaling with small data size  Enable hardware support for complicated communication patterns GPU Technology Conference 2015 7 Mar. 19, 2015

  8. Communication on TCA Architecture  Using PCIe as a communication link between accelerators over the nodes  Direct device P2P communication is available thru PCIe.  PEACH2: Node Node PCI Express Adaptive CPU Memory CPU Memory Communication Hub ver. 2 CPU CPU PCIe Switch PCIe Switch Implementation of the  PCIe PCIe PCe interface and data PCe GPU GPU transfer engine for TCA PEACH2 PEACH2 PCIe GPU Memory GPU Memory GPU Technology Conference 2015 8 Mar. 19, 2015

  9. GPU Communication with traditional MPI  Traditional MPI using InfiniBand requires data copy 3 times  Data copy between CPU and GPU (1 and 3) have to perform manually CPU CPU 2: Data transfer over IB Mem Mem PCIe SW PCIe SW Mem GPU IB IB GPU Mem 1: Copy from GPU mem 3: Copy from CPU mem to CPU mem through PCI Express (PCIe) to GPU mem through PCIe GPU Technology Conference 2015 9 Mar. 19, 2015

  10. GPU Communication with IB/GDR  The InfiniBand controller read and write GPU memory directly (with GDR)  Temporal data copy is eliminated  Lower latency than the previous method  Protocol conversion is still needed 1: Direct data transfer CPU CPU (PCIe -> IB -> PCIe) Mem Mem PCIe SW PCIe SW Mem GPU IB IB GPU Mem GPU Technology Conference 2015 10 Mar. 19, 2015

  11. GPU Communication with TCA (PEACH2)  TCA does not need protocol conversion  direct data copy using GDR  much lower latency than InfiniBand 1: Direct data transfer CPU CPU (PCIe -> PCIe -> PCIe) Mem Mem PCIe SW PCIe SW Mem GPU TCA TCA GPU Mem GPU Technology Conference 2015 11 Mar. 19, 2015

  12. TCA node structure example  Similar to ordinary GPU cluster  PEACH2 can access all configuration except PEACH2 GPUs  80 PCIe lanes are required  NVIDIA Kepler architecture CPU CPU + “ GPUDirect Support for (Xeon (Xeon RDMA” are required. QPI E5 v2) E5 v2) G3 Single PCI address space G2  Connect among 3 nodes G2 x8 G2 G2 G2 PCIe x8 x16 x16 x16 x16 using remaining PEACH2 IB GPU GPU GPU GPU PEA G2 HC 0 1 2 3 CH2 port A x8 G2 G2 QDR x8 x8 GPU: NVIDIA K20X 2port GPU Technology Conference 2015 12 Mar. 19, 2015

  13. TCA node structure example  Similar to ordinary GPU cluster Actually, configuration except PEACH2  Performance over QPI is  80 PCIe lanes are required miserable. CPU CPU (Xeon (Xeon  PEACH2 is available for GPU0, QPI E5 v2) E5 v2) G3 GPU1. G2 G2 x8 G2 G2 G2 PCIe x8 x16 x16 x16 x16  Note that InfiniBand with GPU IB GPU GPU GPU GPU PEA G2 HC Direct for RDMA is available 0 1 2 3 CH2 A x8 only for GPU2, GPU3. G2 G2 x8 x8 GPU: NVIDIA K20X GPU Technology Conference 2015 13 Mar. 19, 2015

  14. Design of PEACH2  Implement by FPGA with four  Latency reduction PCIe Gen.2 IPs  Hardwired logic  Altera Stratix IV GX  Low-overhead routing mechanism  Prototyping, flexible enhancement  Efficient address mapping in PCIe address area using unused  Sufficient communication bits bandwidth  Simple comparator for decision  PCI Express Gen2 x8 for each of output port port (40Gbps = IB QDR) It is not only a proof-of-concept  Sophisticated DMA controller implementation, but it will also be available for product-run in GPU  Chaining DMA, Block-stride transfer function cluster. GPU Technology Conference 2015 14 Mar. 19, 2015

  15. PEACH2 board (Production version for HA-PACS/TCA) FPGA Main board (Altera Stratix IV Most part operates at 250 MHz + sub board (PCIe Gen2 logic runs at 250MHz) 530GX) DDR3- Power supply SDRAM for various voltage PCI Express x8 card edge PCIe x8 cable connecter PCIe x16 cable connecter GPU Technology Conference 2015 15 Mar. 19, 2015

  16. HA-PACS/TCA Compute Node PEACH2 Board is installed here! Front View (8 node / rack ） Rear View 3U height GPU Technology Conference 2015 16 Mar. 19, 2015

  17. Inside of HA-PACS/TCA Compute Node GPU Technology Conference 2015 17 Mar. 19, 2015

  18. Spec. of HA-PACS base cluster & HA-PACS/TCA Base cluster (Feb. 2012) TCA (Nov. 2013) Node CRAY GreenBlade 8204 CRAY 3623G4-SM MotherBoard Intel Washington Pass SuperMicro X9DRG-QF CPU Intel Xeon E5-2670 x 2 socket Intel Xeon E5-2680 v2 x 2 socket (SandyBridge-EP, 2.6GHz 8 core) x2 (IvyBridge-EP, 2.8GHz 10 core) x2 Memory DDR3-1600 128 GB DDR3-1866 128 GB GPU NVIDIA M2090 x4 NVIDIA K20X x 4 # of Nodes 268 (26) 64 (10) (Racks) Interconnect Mellanox InfiniBand QDR x2 (Connect X-3) Mellanox InfiniBand QDR x2 + PEACH2 Peak Perf. 802 TFlops 364 TFlops Power 408 kW 99.3 kW Totally, HA-PACS is over 1PFlops system ! GPU Technology Conference 2015 18 Mar. 19, 2015

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

QCD$Library$for$GPU$Cluster$with$ Proprietary$Interconnect$for$ GPU$Direct$Communica<on

QCD$Library$for$GPU$Cluster$with$ Proprietary$Interconnect$for$ GPU$Direct$Communica<on

QCD$Library$for$GPU$Cluster$with$ Proprietary$Interconnect$for$ GPU$Direct$Communica<on 1 , Hisafumi Fujii 1 , Toshihiro Hanawa 2 , Yuetsu Kodama 3 , ! Norihisa Fujita 4 1,3 , Yoshinobu Kuramashi 3 , Mike Clark Taisuke

690 views • 49 slides
with Proprietary Interconnect TCA for GPU Direct Communication Kazuya MATSUMOTO 1 , Toshihiro

with Proprietary Interconnect TCA for GPU Direct Communication Kazuya MATSUMOTO 1 , Toshihiro

Implementation of CG Method on GPU Cluster with Proprietary Interconnect TCA for GPU Direct Communication Kazuya MATSUMOTO 1 , Toshihiro HANAWA 2 , Yuetsu KODAMA 4 , Hisafumi FUJII 5 , Taisuke BOKU 1,3 1 Center for Computational

600 views • 32 slides
RF-Interconnect RF-Interconnect and its Applications to and its Applications to NoC Design NoC

RF-Interconnect RF-Interconnect and its Applications to and its Applications to NoC Design NoC

RF-Interconnect RF-Interconnect and its Applications to and its Applications to NoC Design NoC Design NOCS Tutorial Course, NOC Tutorial Course, May 10 2009 y 10 2009 San Diego, California San Diego, Californ Frank Chang, Jason Cong and

426 views • 41 slides
Interconnect Technologies for Clusters Interconnect approaches Cluster Computing WAN

Interconnect Technologies for Clusters Interconnect approaches Cluster Computing WAN

Cluster Computing Interconnect Technologies for Clusters Interconnect approaches Cluster Computing WAN infinite distance LAN Few kilometers SAN Few meters Backplane Not scalable Physical Cluster

759 views • 38 slides
Capturing the opportunity in the SMB segment Proprietary + Confidential Proprietary +

Capturing the opportunity in the SMB segment Proprietary + Confidential Proprietary +

Capturing the opportunity in the SMB segment Proprietary + Confidential Proprietary + Confidential John Lojek Partnership Development at Google for UK & Ireland Proprietary + Confidential Proprietary + Confidential Proprietary +

407 views • 16 slides
COOL Interconnect COOL Interconnect Low Power Interconnection Technology Low Power

COOL Interconnect COOL Interconnect Low Power Interconnection Technology Low Power

COOL Interconnect COOL Interconnect Low Power Interconnection Technology Low Power Interconnection Technology for Scalable 3D LSI Design for Scalable 3D LSI Design Marco Chacin, Hiroyuki Uchida, Michiya Hagimoto, Takashi Miyazaki, Takeshi

267 views • 15 slides
MacroPoint Proprietary Cargo Theft is a Global Issue MacroPoint Proprietary MacroPoint

MacroPoint Proprietary Cargo Theft is a Global Issue MacroPoint Proprietary MacroPoint

MacroPoint Proprietary Cargo Theft is a Global Issue MacroPoint Proprietary MacroPoint Proprietary Whats Needed, I n a Nutshell In-transit vi si b il i ty for every load MacroPoint Proprietary MacroPoint Proprietary Shipper Broker 3PL

534 views • 9 slides
Robust Interconnect Robust Interconnect Communication Capacity Algorithm Communication Capacity

Robust Interconnect Robust Interconnect Communication Capacity Algorithm Communication Capacity

Robust Interconnect Robust Interconnect Communication Capacity Algorithm Communication Capacity Algorithm by Geometric Programming by Geometric Programming Jifeng Chen, Jin Sun and Janet Wang Department of Electrical and Computer Engineering

413 views • 25 slides
Molded Interconnect Device Molded Interconnect Device (MID) Device (MID) A 3

Molded Interconnect Device Molded Interconnect Device (MID) Device (MID) A 3

MID Manufacturing: Advances in Metallization Plating Technologies Metallization Plating Technologies Leading to Improved Yields 9 th International Congress Molded Interconnect Devices MID 2010 September 30, 2010 Richard C. Retallick Senior

471 views • 25 slides
& Co Cogn gnition System em Proprietary & Confidential | 1 Proprietary &

& Co Cogn gnition System em Proprietary & Confidential | 1 Proprietary &

Int nteg egrated Sen ensin ing & Co Cogn gnition System em Proprietary & Confidential | 1 Proprietary & Confidential | 1 The last generation of drivers Proprietary & Confidential | 2 Proprietary &

292 views • 28 slides
CS184c: Computer Architecture [Parallel and Multithreaded] Day 15: May 29, 2001 Interconnect

CS184c: Computer Architecture [Parallel and Multithreaded] Day 15: May 29, 2001 Interconnect

CS184c: Computer Architecture [Parallel and Multithreaded] Day 15: May 29, 2001 Interconnect CALTECH cs184c Spring2001 -- DeHon Previously CS184a: Day 11--14 interconnect needs and requirements basic topology

606 views • 24 slides
The Interconnect Verification Challenge Franois Cerisier and Mike Bartley Test and

The Interconnect Verification Challenge Franois Cerisier and Mike Bartley Test and

The Interconnect Verification Challenge Franois Cerisier and Mike Bartley Test and Verification Solutions IP-SOC 2012 Grenoble, 5 Dec 2012 The Interconnect Verification Challenge Whats an interconnect Interconnects

182 views • 16 slides
Connectors: HiLo Interconnect Wednesday, October 04, 2017 Company Overview March 12, 2015

Connectors: HiLo Interconnect Wednesday, October 04, 2017 Company Overview March 12, 2015

Next Level Integration Next Level Integration Connectors: HiLo Interconnect Wednesday, October 04, 2017 Company Overview March 12, 2015 HiLo INTERCONNECT OVERVIEW BGA/LGA Socketing Module Interconnect Highs Lows Flexibility High

254 views • 23 slides
Interconnects Outline Interconnect scaling issues Aluminum technology Copper

Interconnects Outline Interconnect scaling issues Aluminum technology Copper

Interconnects Outline Interconnect scaling issues Aluminum technology Copper technology araswat tanford University EE311 / Al Interconnects 1 Properties of Interconnect Materials Material Thin film Melting resistivity point

438 views • 15 slides
Outline Interconnect scaling issues Aluminum technology Copper technology araswat

Outline Interconnect scaling issues Aluminum technology Copper technology araswat

Outline Interconnect scaling issues Aluminum technology Copper technology araswat tanford University 1 EE311/ Cu Interconnect Wire Half Pitch vs Technology Node ITRS 2002 Narrow line effects Ref: J. Gambino, IEDM, 2003 araswat

417 views • 18 slides
Power/Performance Issues on Interconnection Network (IN) Is interconnect power problem?

Power/Performance Issues on Interconnection Network (IN) Is interconnect power problem?

NsimPower: Interconnect Simulator for Power and Performance Prediction Koji Inoue Kyushu University, Japan 1 Power/Performance Issues on Interconnection Network (IN) Is interconnect power problem? Roughly 10 to 30 % of

381 views • 10 slides
1 Core new business Normalised operating profit Normalised headline earnings +16 16% % +19

1 Core new business Normalised operating profit Normalised headline earnings +16 16% % +19

1 Core new business Normalised operating profit Normalised headline earnings +16 16% % +19 19% % +3 +30% 0% to R9 303m to R4 059m to R2 829m +10% to R9 705m incl. DH take-on of new closed schemes and gross revenue for the Vitality

1.16k views • 83 slides
Challenges urban climate (change): Netherlands High resolution decision maps for urban planning:

Challenges urban climate (change): Netherlands High resolution decision maps for urban planning:

Challenges urban climate (change): Netherlands High resolution decision maps for urban planning: a combined analysis of urban flooding and thermal stress potential in Asia and Europe Floris Boogaard 1,2 , Zoran Vojinovic 4 , Yu-Cheng Chen 3 ,

530 views • 5 slides
under Climate Change Department of Water Resources 28 January 2015 1.Water Security 2.Strategic

under Climate Change Department of Water Resources 28 January 2015 1.Water Security 2.Strategic

Strategic Plans for Flood Mr.Chanawat Arrunrata Plan and policy analyst and Drought Management under Climate Change Department of Water Resources 28 January 2015 1.Water Security 2.Strategic Plans for Flood & Drought Management 3.Water

604 views • 25 slides
Flood Risk Management Project Karen Thomas Project Manager Water Management Alliance On behalf

Flood Risk Management Project Karen Thomas Project Manager Water Management Alliance On behalf

Benacre & Kessingland Flood Risk Management Project Karen Thomas Project Manager Water Management Alliance On behalf of Waveney, Lower Yare and Lothingland Internal Drainage Board Lowestoft N Courtesy of Mike Page 2016 Courtesy of Mike

468 views • 24 slides
AESA Member Annual Report 2015 PING, Inc. Leadership Characteristics Conservation and Pollution

AESA Member Annual Report 2015 PING, Inc. Leadership Characteristics Conservation and Pollution

AESA Member Annual Report 2015 PING, Inc. Leadership Characteristics Conservation and Pollution Prevention Objective: Reduce non-transportation energy use by 25% in 5 years (Year 2) Assembly Remodel - LED ~31,000 kWh annual savings

504 views • 31 slides
Building Capability for Hydrographic Products and Industry Contribution to Capacity Building

Building Capability for Hydrographic Products and Industry Contribution to Capacity Building

ROPME Sea Area Hydrographic Commission Riyadh 4th - 6th March 2013 Building Capability for Hydrographic Products and Industry Contribution to Capacity Building Commander Richard Bungy Williams RN (Rtd) Kongsberg Maritime / 1 / 6-Mar-13

713 views • 39 slides
Suggested Discussion Focus Paradigm shift in open space planning Multiple functional approach

Suggested Discussion Focus Paradigm shift in open space planning Multiple functional approach

Suggested Discussion Focus Paradigm shift in open space planning Multiple functional approach From Town Park to Town Plaza Integrating nature conservation / ecology revitalization / flood prevention with recreation How design and

280 views • 17 slides
New Urban Agenda Dr Graham Alabaster Chief of Sanitation & Waste Management, UNHABITAT 1

New Urban Agenda Dr Graham Alabaster Chief of Sanitation & Waste Management, UNHABITAT 1

Role of Water Utilities in SDGs and The New Urban Agenda Dr Graham Alabaster Chief of Sanitation & Waste Management, UNHABITAT 1 Water and Urbanization Issues and Solutions 2 Water and Cities Global Commitments Universal access to safe

241 views • 22 slides

More recommend