uni lu high performance computing ulhpc facility
play

Uni.lu High Performance Computing (ULHPC) Facility User Guide, 2020 - PowerPoint PPT Presentation

Jun 10, 2023 •323 likes •1.02k views

Uni.lu High Performance Computing (ULHPC) Facility User Guide, 2020 UL HPC Team https://hpc.uni.lu S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 1 / 48 Summary 1 High

  1. Uni.lu High Performance Computing (ULHPC) Facility User Guide, 2020 UL HPC Team https://hpc.uni.lu S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 1 / 48 �

  2. Summary 1 High Performance Computing (HPC) @ UL 2 Batch Scheduling Configuration 3 User [Software] Environment 4 Usage Policy 5 Appendix: Impact of Slurm 2.0 configuration on ULHPC Users S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 2 / 48 �

  3. High Performance Computing (HPC) @ UL Summary 1 High Performance Computing (HPC) @ UL 2 Batch Scheduling Configuration 3 User [Software] Environment 4 Usage Policy 5 Appendix: Impact of Slurm 2.0 configuration on ULHPC Users S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 3 / 48 �

  4. High Performance Computing (HPC) @ UL High Performance Computing @ UL Started in 2007 under resp. of Prof P. Bouvry & Dr. S. Varrette → 2nd Largest HPC facility in Luxembourg. . . ֒ � after EuroHPC MeluXina ( ≥ 15 PFlops) system Rectorate https://hpc.uni.lu/ HPC/Computing Capacity High Performance Procurement IT Office Computing Department 2794.23 TFlops @ Uni.lu (incl. 748.8 GPU TFlops) Shared Storage Capacity Logistics & Infrastructure 10713.4 TB storage Department S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 4 / 48 �

  5. High Performance Computing (HPC) @ UL High Performance Computing @ UL S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 5 / 48 �

  6. High Performance Computing (HPC) @ UL High Performance Computing @ UL 3 types of computing resources across 2 clusters ( aion , iris ) S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 6 / 48 �

  7. High Performance Computing (HPC) @ UL High Performance Computing @ UL 4 File Systems commons across the 2 clusters ( aion , iris ) S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 7 / 48 �

  8. High Performance Computing (HPC) @ UL Accelerating UL Research - User Software Sets Over 230 software packages available for researchers → software environment generated using Easybuild / LMod ֒ → containerized applications delivered with Singularity system ֒ Theorize Model Domain 2019 Software environment Develop Compiler Toolchains FOSS (GCC), Intel, PGI MPI suites OpenMPI, Intel MPI Machine Learning PyTorch, TensorFlow, Keras, Horovod, Apache Spark. . . Compute Math & Optimization Matlab, Mathematica, R, CPLEX, Gurobi. . . Simulate Physics & Chemistry GROMACS, QuantumESPRESSO, ABINIT, NAMD, VASP. . . Experiment Bioinformatics SAMtools, BLAST+, ABySS, mpiBLAST, TopHat, Bowtie2. . . Computer aided engineering ANSYS, ABAQUS, OpenFOAM. . . General purpose ARM Forge & Perf Reports, Python, Go, Rust, Julia. . . Container systems Singularity Visualisation ParaView, OpenCV, VMD, VisIT Supporting libraries numerical (arpack-ng, cuDNN), data (HDF5, netCDF). . . Analyze . . . https://hpc.uni.lu/users/software/ S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 8 / 48 �

  9. High Performance Computing (HPC) @ UL UL HPC Supercomputers: General Architecture Other Clusters Uni.lu cluster Local Institution network Network 10/40/100 GbE 10/25/40 GbE [Redundant] Load balancer Redundant Site routers [Redundant] Site access server(s) [Redundant] Adminfront(s) Site Computing Nodes dhcp brightmanager slurm dns puppet monitoring etc... Fast local interconnect (Infiniband EDR/HDR) 100-200 Gb/s SpectrumScale/GPFS Lustre Isilon Disk Enclosures Site Shared Storage Area S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 9 / 48 �

  10. High Performance Computing (HPC) @ UL UL HPC Supercomputers: iris cluster Uni.lu @ Internet Internal Network Dell/Intel supercomputer, Air-flow cooling Uni.lu (Belval) @ Restena UL internal → 196 compute nodes Iris cluster ֒ UL external (Local) � 5824 compute cores ULHPC Site router 2x 10 GbE 2x 40 GbE QSFP+ adminfront1 adminfront2 2 4 2 4 � Total 52224 GB RAM 10 GbE SFP+ slurm1 puppet1 dns1 dns2 puppet2 slurm2 lb1,lb2… brightmanager1 … … brightmanager2 Fast local interconnect 2x Dell R630 (2U) (Fat-Tree Infiniband EDR) Load Balancer(s) 2*16c Intel Xeon E5-2697A v4 (2,6GHz) → R peak : 1,072 PetaFLOP/s 100 Gb/s ֒ (SSH ballast, HAProxy, Apache ReverseProxy…) storage1 storage2 access1 access2 Fast InfiniBand (IB) EDR network Dell R730 (2U) (2*14c Intel Xeon E5-2660 v4@2GHz) 2 CRSI 1ES0094 (4U, 600TB) 2x Dell R630 (2U) RAM: 128GB, 2 SSD 120GB (RAID1) 60 disks 12Gb/s SAS JBOD (10 TB) (2*12c Intel Xeon E5-2650 v4 (2,2GHz) 5 SAS 1.2TB (RAID5) User Cluster Frontend Access sftp/ftp/pxelinux, node images, Container image gateways → Fat-Tree Topology Yum package mirror etc. ֒ blocking factor 1:1.5 CDC S-02 Belval - 196 computing nodes (5824 cores) DDN / GPFS Storage (2284 TB) EMC ISILON Storage (3188TB) Rack ID Purpose Description D02 Network Interconnect equipment D04 Management Management servers, Interconnect DDN GridScaler 7K (24U) 1xGS7K base + 4 SS8460 expansion D05 Compute iris-[001-056] , interconnect 380 disks (6 TB SAS SED, 37 RAID6 pools) 10 disks SSD (400 GB) D07 Compute iris-[057-112] , interconnect DDN / Lustre Storage (1300 TB) mds1 mds2 Dell R630, 2x[8c] Intel E5-2667v4@3.2GHz Dell R630XL, 2x[10c] Intel E5-2640v4@2.4GHz D09 Compute iris-[113-168] , interconnect oss1 RAM:128GB oss2 (Internal Lustre) Infiniband FDR D11 Compute iris-[169-177,191-193] (gpu), iris-[187-188] (bigmem) DDN ExaScaler7K(24U) D12 Compute iris-[178-186,194-196] (gpu), iris-[189-190] (bigmem) 2x SS7700 base + SS8460 expansion 42 Dell C6300 encl. - 168 Dell C6320 nodes [4704 cores] OSTs: 167 (83+84) disks (8 TB SAS, 16 RAID6 pools) 108 x (2 *14c Intel Xeon Intel Xeon E5-2680 v4 @2.4GHz), RAM: 128GB / 116,12 TFlops MDTs: 19 (10+9) disks (1.8 TB SAS, 8 RAID1 pools) 60 x (2 *14c Intel Xeon Intel Xeon Gold 6132 @ 2.6 GHz), RAM: 128GB / 139,78 TFlops 24 Dell C4140 GPU nodes [672 cores] iris cluster characteristics 24 x (2 *14c Intel Xeon Intel Xeon Gold 6132 @ 2.6 GHz), RAM: 768GB / 55.91 TFlops 24 x (4 NVidia Tesla V100 SXM2 16 or 32GB) = 96 GPUs / 748,8 TFlops Computing : 196 nodes, 5824 cores; 96 GPU Accelerators - Rpeak ≈ 1082,47 TFlops 4 Dell PE R840 bigmem nodes [448 cores] 4 x (4 *28c Intel Xeon Platinum 8180M @ 2.5 GHz), RAM: 3072GB / 35,84 TFlops Storage : 2284 TB (GPFS) + 1300 TB (Lustre) + 3188TB (Isilon/backup) + 600TB (backup) S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 10 / 48 �

  11. High Performance Computing (HPC) @ UL UL HPC Supercomputers: aion cluster Atos/AMD supercomputer, DLC cooling → 4 BullSequana XH2000 adjacent racks ֒ → 318 compute nodes ֒ � 40704 compute cores � Total 81408 GB RAM → R peak : 1,693 PetaFLOP/s ֒ Fast InfiniBand (IB) HDR network → Fat-Tree Topology ֒ blocking factor 1:2 Rack 1 Rack 2 Rack 3 Rack 4 TOTAL Weight [kg] 1872,4 1830,2 1830,2 1824,2 7357 kg #X2410 Rome Blade 28 26 26 26 106 #Compute Nodes 84 78 78 78 318 #Compute Cores 10752 9984 9984 9984 40704 R peak [TFlops] 447,28 TF 415,33 TF 415,33 TF 415,33 TF 1693.29 TF S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 11 / 48 �

  12. High Performance Computing (HPC) @ UL UL HPC Software Stack Operating System : Linux CentOS/Redhat User Single Sign-on : Redhat IdM/IPA Remote connection & data transfer : SSH/SFTP → User Portal : Open OnDemand ֒ Scheduler/Resource management : Slurm (Automatic) Server / Compute Node Deployment : → BlueBanquise, Bright Cluster Manager, Ansible, Puppet and Kadeploy ֒ Virtualization and Container Framework : KVM, Singularity Platform Monitoring (User level): Ganglia, SlurmWeb, OpenOndemand. . . ISV software : → ABAQUS, ANSYS, MATLAB, Mathematica, Gurobi Optimizer, Intel Cluster Studio XE, ֒ ARM Forge & Perf. Report, Stata, . . . S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 12 / 48 �

  13. Batch Scheduling Configuration Summary 1 High Performance Computing (HPC) @ UL 2 Batch Scheduling Configuration 3 User [Software] Environment 4 Usage Policy 5 Appendix: Impact of Slurm 2.0 configuration on ULHPC Users S. Varrette & UL HPC Team (University of Luxembourg) Uni.lu High Performance Computing (ULHPC) Facility 13 / 48 �

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Getting the Performance Out Of Getting the Performance Out Of High Performance Computing High

Getting the Performance Out Of Getting the Performance Out Of High Performance Computing High

Getting the Performance Out Of Getting the Performance Out Of High Performance Computing High Performance Computing Jack Dongarra I nnovative Computing Lab University of Tennessee and Computer Science and Math Division Oak Ridge Nat ional

380 views • 13 slides
High Performance Computing in Web Browsers CE Seminar WT14/15 Henning Lohse High Performance

High Performance Computing in Web Browsers CE Seminar WT14/15 Henning Lohse High Performance

High Performance Computing in Web Browsers CE Seminar WT14/15 Henning Lohse High Performance Computing voltagegate.scientopia.org ks.uiuc.edu High Performance Computing Processor Intra node Inter node GPU (coprocessor) High

580 views • 33 slides
Uni.lu HPC Facility Overview & Challenges at the EuroHPC Horizon Uni.lu High Performance

Uni.lu HPC Facility Overview & Challenges at the EuroHPC Horizon Uni.lu High Performance

Uni.lu HPC Facility Overview & Challenges at the EuroHPC Horizon Uni.lu High Performance Computing Team University of Luxembourg (UL), Luxembourg https://hpc.uni.lu S. Varrette & al. (HPC @ University of Luxembourg) Uni.lu HPC Facility

887 views • 41 slides
Trends in High Performance Trends in High Performance Computing and the Grid Computing and the

Trends in High Performance Trends in High Performance Computing and the Grid Computing and the

Trends in High Performance Trends in High Performance Computing and the Grid Computing and the Grid Jack Dongarra University of Tennessee and Oak Ridge National Laboratory 1 Technology Trends: Technology Trends: Microprocessor Capacity

465 views • 23 slides
NSF Future of High Performance Computing Bill Kramer NSF Workshop on the Future of High

NSF Future of High Performance Computing Bill Kramer NSF Workshop on the Future of High

NSF Future of High Performance Computing Bill Kramer NSF Workshop on the Future of High Performance Computing Washington DC December 4, 2009 Why Sustained Performance is the Critical Focus Rela#onship between Peak,

251 views • 9 slides
High Performance Computing, High Performance Computing, Computational Grid, and Numerical

High Performance Computing, High Performance Computing, Computational Grid, and Numerical

T h e 1 4 t h S ym p os iu m on C om p u t e r A r c h it e c t u r e a n d H ig h P e r f or m a n c e C om p u t in g V it o r ia / E S - B r a z il - O c t ob e r 2 8 -3 0 , 2 0 0 2 High Performance Computing, High Performance

785 views • 23 slides
Introduction to High Performance Computing at ZIH Architecture of the PC Farm (Deimos)

Introduction to High Performance Computing at ZIH Architecture of the PC Farm (Deimos)

Center for Information Services and High Performance Computing (ZIH) Introduction to High Performance Computing at ZIH Architecture of the PC Farm (Deimos) Zellescher Weg 12 Trefftz-Bau/HRSK 151 Phone +49 351 - 463 - 39871 Guido Juckeland

634 views • 19 slides
Introduction to High Performance Computing Pierre Aubert High Performance Computing (HPC)

Introduction to High Performance Computing Pierre Aubert High Performance Computing (HPC)

Introduction to High Performance Computing Pierre Aubert High Performance Computing (HPC) Part of the computer science Get the best performances by using the right algorithms on the right architectures Pierre Aubert, Introduction to HPC

506 views • 13 slides
High Performance Computing What is it used for and why? Overview What is it used for?

High Performance Computing What is it used for and why? Overview What is it used for?

High Performance Computing What is it used for and why? Overview What is it used for? Drivers for HPC Examples of usage Why do you need to learn the basics? Hardware layout and structure matters Serial computing is

585 views • 19 slides
Future Directions in High Future Directions in High P Performance Computing Performance

Future Directions in High Future Directions in High P Performance Computing Performance

Future Directions in High Future Directions in High P Performance Computing Performance Computing P f f C C ti ti Jack Dongarra k INNOVATIVE COMP ING LABORATORY U i University of Tennessee i f T Oak Ridge National Laboratory

653 views • 46 slides
Finding Performance-Optimal Configurations for High-Performance Computing Alexander Grebhahn,

Finding Performance-Optimal Configurations for High-Performance Computing Alexander Grebhahn,

Finding Performance-Optimal Configurations for High-Performance Computing Alexander Grebhahn, Norbert Siegmund, Sven Apel University of Passau FOSD Meeting 2014, Dagstuhl High-Performance Computing and ExaStencils Alexander Grebhahn Finding

949 views • 55 slides
Tux Faces the Rigors of Terascale Computation: High-Performance Computing and the Linux Kernel

Tux Faces the Rigors of Terascale Computation: High-Performance Computing and the Linux Kernel

Tux Faces the Rigors of Terascale Computation: High-Performance Computing and the Linux Kernel David Cowley Pacific Northwest National Laboratory Information Release #PNNL-SA-65780 EMSL is a national scientific user facility at the Pacific

675 views • 31 slides
Mercury: RPC for High-Performance Computing Jerome Soumagne The HDF Group June 23, 2017 RPC and

Mercury: RPC for High-Performance Computing Jerome Soumagne The HDF Group June 23, 2017 RPC and

Mercury: RPC for High-Performance Computing Jerome Soumagne The HDF Group June 23, 2017 RPC and High-Performance Computing 2 June 23, 2017 CS/NERSC Data Seminar RPC and High-Performance Computing 2 Remote Procedure Call (RPC) Allow

1.38k views • 50 slides
An Overview of High Performance An Overview of High Performance Computing, Clusters, and the Grid

An Overview of High Performance An Overview of High Performance Computing, Clusters, and the Grid

An Overview of High Performance An Overview of High Performance Computing, Clusters, and the Grid Computing, Clusters, and the Grid Jack Dongarra University of Tennessee and Oak Ridge National Laboratory 1 Technology Trends: Technology

500 views • 22 slides
15th Symposium on Computer Architecture and High Performance Computing 1/12 November 10 to 12 -

15th Symposium on Computer Architecture and High Performance Computing 1/12 November 10 to 12 -

15th Symposium on Computer Architecture and High Performance Computing 1/12 November 10 to 12 - S ao Paulo, SP Comparison of Genomes using High-Performance Parallel Computing N. F. Almeida Jr Universidade Federal de Mato Grosso do

406 views • 12 slides
Building a High Performance Team to Achieve Excellence in Facility Management . June 3,

Building a High Performance Team to Achieve Excellence in Facility Management . June 3,

Building a High Performance Team to Achieve Excellence in Facility Management . June 3, 2015 The Journey Begins High-Performance FM Organizations What does HP/world class mean to you? How do you get there? What role

669 views • 42 slides
Development for the iPhone/ iPad from a J ava Perspective The Tools, The Language, The

Development for the iPhone/ iPad from a J ava Perspective The Tools, The Language, The

Development for the iPhone/ iPad from a J ava Perspective The Tools, The Language, The Experience Ognen Ivanovski Netcetera 2 Myself > Software Architect; @Netcetera since 2001 > 2000 - present: Java (enterprise, web, swing, RCP)

592 views • 43 slides
!"#"$%&'()#"*'+",*-"$.' The Jeopardy Circuit ' '

!"#"$%&'()#"*'+",*-"$.' The Jeopardy Circuit ' '

!"#$%%&' ()**)+"',+--.#.' !"#"$%&'()#"*'+",*-"$.' The Jeopardy Circuit ' ' /0$,)1-*$")02' /"'01)*'-23'4+5'6)--'35)-7'2'8)$85)0'0120'7.0.$9)".*'61)81'0.29':-24.$':5*1.*'1)*;1.$'

685 views • 9 slides
Microprocessors & Interfacing Concepts Standards USART in AVR Serial

Microprocessors & Interfacing Concepts Standards USART in AVR Serial

Lecture Overview Serial Communication Microprocessors & Interfacing Concepts Standards USART in AVR Serial Input/Output Lecturer : Dr. Annie Guo S2, 2008 COMP9032 Week10 1 S2, 2008 COMP9032 Week10 2 Serial

335 views • 17 slides
an Some results of Zolt Esik on regular languages Jean- Eric Pin 1 1 IRIF, CNRS and

an Some results of Zolt Esik on regular languages Jean- Eric Pin 1 1 IRIF, CNRS and

an Some results of Zolt Esik on regular languages Jean- Eric Pin 1 1 IRIF, CNRS and University Paris Diderot FCT, September 2017, Bordeaux IRIF, CNRS and University Paris Diderot an Zolt Esik an Zolt Esik passed away in

828 views • 45 slides
George Chen Amit Dhuria Intel Cadence The TAU 2017 Contest Path Reporting Contest George Xi

George Chen Amit Dhuria Intel Cadence The TAU 2017 Contest Path Reporting Contest George Xi

George Chen Amit Dhuria Intel Cadence The TAU 2017 Contest Path Reporting Contest George Xi Amit Chen Chen Dhuria Intel Synopsys Cadence [Speaker] Sponsors: Programmable Solutions Group 2 Why Path Reporting? Path reporting occupies a

618 views • 25 slides
Centrality determination in MPD at NICA: Glauber model application Petr Parfenov 1,2 , Dim Idrisov

Centrality determination in MPD at NICA: Glauber model application Petr Parfenov 1,2 , Dim Idrisov

Centrality determination in MPD at NICA: Glauber model application Petr Parfenov 1,2 , Dim Idrisov 1 , Ilya Segal 1 , Vinh Luong 1 , Ilya Selyuzhenkov 1,3 , Arkadiy Taranenko 1 , Aleksandr Ivashkin 2 1 NRNU MEPhI, 2 INR RAS, 3 GSI for MPD

916 views • 68 slides
CLEF: 15 Years of IR Evaluation in Europe Nicola Ferro University of Padua, Italy Forum

CLEF: 15 Years of IR Evaluation in Europe Nicola Ferro University of Padua, Italy Forum

The CLEF Initiative Conference and Labs of the E valuation Forum I N I T I A T I V E http:/ /www.clef - initiative.eu/ CLEF: 15 Years of IR Evaluation in Europe Nicola Ferro University of Padua, Italy Forum for Information Retrieval

591 views • 22 slides
ALICE Experiment Highlights and Status Rene Bellwied (ALICE-USA Coordinator) University of

ALICE Experiment Highlights and Status Rene Bellwied (ALICE-USA Coordinator) University of

ALICE Experiment Highlights and Status Rene Bellwied (ALICE-USA Coordinator) University of Houston (bellwied@uh.edu) USLUA Meeting, Chicago, November 2015 R. BELLWIED * 1 Topics to be discussed Status of ALICE Run-1 analysis LS1

944 views • 50 slides

More recommend