GPU Clusters for HPC Edward Seidel Director Bill Kramer Blue - - PowerPoint PPT Presentation

▶
gpu clusters for hpc
SMART_READER_LITE
LIVE PREVIEW

GPU Clusters for HPC Edward Seidel Director Bill Kramer Blue - - PowerPoint PPT Presentation

Feb 22, 2024 268 likes •466 views

GPU Clusters for HPC Edward Seidel Director Bill Kramer Blue Waters Project Director National Center for Supercomputing Applications University of Illinois at Urbana- Champaign National Center for Supercomputing Applications University of

slide-1
SLIDE 1

National Center for Supercomputing Applications University of Illinois at Urbana-Champaign

GPU Clusters for HPC

Edward Seidel Director Bill Kramer Blue Waters Project Director National Center for Supercomputing Applications University of Illinois at Urbana- Champaign

slide-2
SLIDE 2

WHAT IS NCSA

National Center for Supercomputing Applications

Imaginations unbound

slide-3
SLIDE 3

National Center for Supercomputing Applications: 30 years of leadership

  • NCSA
  • R&D unit of the University of Illinois at Urbana-Champaign
  • One of original five NSF-funded supercomputing centers
  • Mission: Provide state-of-the-art computing capabilities (hardware, software, hpc

expertise) to nation’s scientists and engineers

  • The Numbers
  • Approximately 200 staff (160+ technical/professional staff)
  • Approximately 15 graduate students (+ new SPIN program), 15 undergrad students
  • Two major facilities (NCSA Building, NPCF)
  • Operating NSF’s most powerful computing system: Blue Waters
  • Managing NSF’s national cyberinfrastructure: XSEDE

Source: Thom Dunning

slide-4
SLIDE 4

Petas cale Computing Facility: Home to Blue Waters

  • Modern Data Center
  • 90,000+ ft2 total
  • 30,000 ft2 raised floor

20,000 ft2 machine room gallery

  • Energy Efficiency
  • LEED certified Gold
  • Power Utilization Efficiency

= 1.1–1.2

  • Blue Waters
  • 13PF, 1500TB,

300PB

  • >1PF On real apps
  • NAMD, MILC,

WRF, PPM, NWChem, etc

Source: Thom Dunning

slide-5
SLIDE 5

Data Intensive Computing

Source: Thom Dunning

LSST, DES Personalized Medicine w/ Mayo

slide-6
SLIDE 6

NCSA’s Industrial Partners

Source: Thom Dunning

slide-7
SLIDE 7

PARTNERSHIP FOR INNOVATION

NCSA and NVIDIA

Imaginations unbound

slide-8
SLIDE 8

NCSA, NVIDIA and GPUs

  • NCSA and NVIDIA have been partners for over a

decade, building the expertise, experience and technology.

  • The efforts were at first exploratory and small scale, but

have now blossomed into providing the largest GPU production resource in the US academic cyber- infrastructure

  • Today, we are focusing on helping world class science

and engineering teams decrease their time to insight for some of the world’s most important and challenging computational and data analytical problems

Imaginations unbound

slide-9
SLIDE 9

Innovative Systems Lab 1.0 Research

  • Evaluation of emerging

computing architectures

  • Reconfigurable computing
  • Many-core (GPU) architecture
  • Heterogeneous clusters
  • Systems software research

and development

  • Run-time systems
  • GPU accelerator cluster

management

  • Tools and utilities: GPU

memory test, power profiling, etc.

  • Power Measurement and

efficiency

Imaginations unbound

QP: first GPU cluster at NCSA 16 HP xw9400 workstations 32 Quadro Plex Computing Servers

slide-10
SLIDE 10

How might this be used for Science??

  • Focus on Application

development for emerging computing architectures – not benchmarking

  • Cuda Center of Excellence in

the Coordinated Systems Laboratory

  • Computer Science efforts in

GPUs

  • A variety of discipline

departments

  • Computational chemistry

(electronic structure, MD)

  • Computational physics (QCD)
  • Cosmology and astronomy
  • Data mining

Imaginations unbound

MD Simulations and Computational Microscope; see John Stone! Turbulence simulations; Paul Woodward

slide-11
SLIDE 11

FROM PIONEERING TO PRODUCTION

Imaginations unbound

slide-12
SLIDE 12

Blue Waters Computing S ys tem

Sonexion: 26 usable PB

>1 TB/sec 100 GB/sec

10/40/100 Gb Ethernet Switch

Spectra Logic: 300 usable PB

120+ Gb/sec

100-300 Gbps WAN

IB Switch External Servers

Aggregate Memory – 1.6 PB

Imaginations unbound

slide-13
SLIDE 13

Original Blue Waters Goals

  • Deploy a computing system capable of sustaining more than one

petaflops or more for a broad range of applications

  • Cray system achieves this goal using a well defined metrics
  • Enable the Science Teams to take full advantage of the sustained

petascale computing system

  • Blue Waters Team has established strong partnership with Science Teams, helping them to

improve the performance and scalability of their applications

  • Enhance the operation and use of the sustained petascale system
  • Blue Waters Team is developing tools, libraries and other system software to aid in operation of

the system and to help scientists and engineers make effective use of the system

  • Provide a world-class computing environment for the petascale

computing system

  • The NPCF is a modern, energy-efficient data center with a rich WAN environment (100-400

Gbps) and data archive (>300 PB)

  • Exploit advances in innovative computing technology
  • Proposal anticipated the rise of heterogeneous computing and planned to help the computational

community transition to new modes for computational and data-driven science and engineering

Imaginations unbound

slide-14
SLIDE 14

Details of Blue Waters

Imaginations unbound

slide-15
SLIDE 15

Production Computation Science with XK nodes

  • The Computational Microscope
  • PI – Klaus Schulten
  • Simulated flexibility of ribosome trigger factor complex at

full length and obtained better starting configuration of trigger factor model (simulated to 80ns)

  • 100ns simulation of cylindrical HIV 'capsule’ of CA proteins

revealed it is stabilized by hydrophobic interactions between CA hexamers; maturation involves detailed remodeling rather than disassembly/re-assembly of CA lattice, as had been proposed.

  • 200ns simulation of CA pentamer surrounded by CA

hexamers suggested interfaces in hexamer-hexamer and hexamer-pentamer pairings involve different patterns of interactions

  • Simulated photosynthetic membrane of a chromatophore in

bacterium Rps. photometricum for 20 ns -- simulation of a few hundred nanoseconds will be needed

Images from Klaus Schulten and John Stone, University of Illinois at Urbana-Champaign Imaginations unbound

slide-16
SLIDE 16

XK7 For Visualization on Blue Waters

  • Many visualization utilities rely on the OpenGL API for

hardware-accelerated rendering

  • Unsupported by default XK7 system software
  • Enabling NVIDIA’s OpenGL required that we:
  • Change operating mode of the XK7 GPU firmware
  • Develop a custom X11 stack
  • Work with Cray to acquire alternate driver package from NVIDIA
  • Blue Waters is the first Cray to offer this functionality

which has been distributed to other systems now

Imaginations unbound

slide-17
SLIDE 17

Impact: VMD

  • Molecular dynamics analysis and

visualization tool used by “The Computational Microscope” science team (PI Klaus Schulten)

  • 10X to 50X rendering speedup in

VMD

  • Interactive rate visualization
  • Drastic reduction in required time to

fine tune parameters for production visualization

Imaginations unbound

slide-18
SLIDE 18

Summary

  • NVIDIA GPUs are critical to the

nation’s most powerful academic supercomputer

  • Partnership with NCSA, Cray,

NVIDIA

  • Science, engineering, private sector

partner development all depend on it

  • Enable research that cannot be done

in any other way, on any other system

  • See John Stone presentation, Bill

Kramer presentation for more details

Imaginations unbound