www.bsc.es Interactive HPC: Large Scale In-Situ Visualization Using NVIDIA Index in ALYA MultiPhysics Christopher Lux (NV), Vishal Mehta (BSC) and Marc Nienhaus (NV) May 8 th 2017
Barcelona Supercomputing Center Marenostrum 4 • 13.7 PetaFlop/s • General Purpose Computing ▪ 3400 nodes of Xeon, 11 PF/s • Emerging Technologies ▪ Power 9 + Pascal 1.5 PF/s ▪ Knights Landing and Knights Hill 0.5 PF/s ▪ 64bit ARMv8 0.5 PF/s 2
Research at BSC EARTH SCIENCES COMPUTER SCIENCES To influence the way To develop and implement machines are built, global and regional state- programmed and used: of-the-art models for short- programming models, performance tools, Big Data, term air quality forecast computer architecture, energy and long-term climate efficiency applications LIFE SCIENCES CASE To develop scientific and To understand living engineering software to organisms by means of efficiently exploit super- theoretical and computing capabilities computational methods (biomedical, geophysics, atmospheric, energy, (molecular modeling, social and economic genomics, proteomics) simulations) 3
ALYA System: Large Scale Computational Mechanics 4
5
ALYA HPC Context 6
ALYA HPC Context 7
ALYA RED 8
Computational Cardiac Model Applications ▪ Pacemaker applications ▪ Computational analysis of malfunctioning tissue patches ▪ Computational Drug testing on cardiac tissue. 9
Computational Cardiac Model 10
Pacemaker Application 11
Pacemaker Application 12
Computational Drug Testing 13
INTERACTIVE HPC: LARGE SCALE IN-SITU VISUALIZATION USING NVIDIA INDEX IN ALYA MULTIPHYSICS Christopher Lux (NV), Vishal Mehta (BSC) and Marc Nienhaus (NV) May 8 th 2017
NVIDIA INDEX Scalable, Interactive Visual Computing GPU-cluster aware solution High-quality and scalable visualization of large-scale datasets In-situ visualization Commercial software Available and deployed in production 15
SCALABILITY AS ENABLER NVIDIA HPC Clusters NVIDIA Quadro VCA or DGX-1 NVIDIA Quadro Workstation Performance, dataset size, number of pixels, visual quality … 16
Scientific Data Visualization 17
Time-Varying Data Visualization 18
Time-Varying Data Visualization Simulation data source: A Numerical Study of High-Pressure Oxygen/Methane Mixing and Combustion of a Shear Coaxial Injector , 19 Nan Zong & Vigor Yang, AIAA 2005
In-Trans and In-Situ Visualization 20
Computational Heart 21
BACKGROUND 22
DISTRIBUTED PARALLEL RENDERING Sort-Last Rendering (multi-GPU) [..] [..] * image compositing 23
DISTRIBUTED PARALLEL RENDERING Sort-Last Rendering (Cluster of multi-GPU Nodes) Cluster of VCAs [..] [..] * image compositing 24
DISTRIBUTED DATATYPES Various Application Domains Volume datatypes Regular ▪ Sparse ▪ ▪ Unstructured/Irregular Surface-geometry datatypes ▪ Height field Triangle mesh ▪ 25
IN-TRANS AND IN-SITU VISUALIZATION 26
TRADITIONAL VISUALIZATION PIPELINE Simulation Cluster NVIDIA IndeX Visualization Data Storage Cluster e.g. Unstructured Data 5/15/2 27 017
TIME-SERIES DATA VISUALIZATION Visualize Pre-calculated ALYA Simulation Results Visualize and Animate Stream Interact and Explore Terabyte time-varying simulation data of nasal system 28
IN-SITU (IN-TRANS) VISUALIZATION PIPELINE Unstructured Data Unstructured Data Unstructured Data Simulation Cluster NVIDIA IndeX Visualization Network Cluster 29
IN-SITU VISUALIZATION PIPELINE Combined Simulation and Visualization Cluster 5/15/2 30 017
IN-SITU/IN-TRANS SUPPORT Compute Result Integration Parallel jobs executed locally or remotely Direct access to local host and device data Fast RDMA memory transfers User-defined affinity and spatial subdivision Application-driven updates Push updated data when ready ▪ Rendering-driven updates ▪ Request computation updates for active data 31 Clustered neuron activity
IN-TRANS RESULT TRANSFERS Fast Data Transfers to Rendering Nodes/GPUs Page-Locked System Page-Locked System Memory Memory RDMA (over InfiniBand) CUDA GPU CUDA GPU Memory Memory GPUDirect RDMA NVLink high-speed interconnect between system memory and GPU (IBM and NVIDIA) 32
COMPUTATIONAL HEART In-Situ Simulation and Visualization Simulate/Compute Visualize (ALYA) Interact and Parameterize and Explore Steer 33
NVIDIA INDEX AVAILABILITY 34
NVIDIA INDEX 1.4 In-Situ/In-Trans Visualization Support Support of 32bit, 16bit, 8bit fixed point, 32bit floating point and RGBA (8bit) regular volumes Dynamic streaming and GPU caching of time-varying volume data Irregular volumes and sparse volumes Built-in volume shading capabilities Multi-view capabilities NVIDIA IndeX 1.4 Zero-copy RDMA/GPUDirect compute integration infrastructure (released 07/2016) User-defined affinity and spatial subdivision support Architecture and API for in-situ/in-trans visualization (compute integration) Dynamic workload balancing Advanced CUDA memory management, error handling and logging MPI/NVIDIA IndeX interprocess coupling (CUDA IPC and shared memory) 35
NVIDIA INDEX 1.5 OUTLOOK User-defined Rendering Kernel Components 36
NVIDIA INDEX 1.5 OUTLOOK User-defined Rendering Kernel Components 37
IN-SITU VISUALIZATIONS 38
Challenges Simulation Index Rendering MPI MPI • Parallel Operations • Maintain frame rates • Steering Simulation MPI MPI • Data Affinity . . • Render @compute Cubical Scene region . . Unstructured Mesh • In-trans approach Even spatial partitions Uneven Spatial partitions Balanced rendering load Balanced computations MPI MPI 39
Multi-code Coupling in ALYA • All spatial interpolation on spatial domain in structured and unstructured meshes • Allows setting send and receive frequencies to synchronize simulation times. • Allows coupling with third party codes • Parallel and Asynchronous MPI coupling 40
Ingredients of Coupling in ALYA mpirun -np 8 Alya.x fluid : -np 4 Test.x : -np 4 Alya.x solidz • WHAT The underlying variables • WHERE Surface, Volume, etc. • WHEN Time step, iteration step • HOW Algorithmic interpolation 41
Coupling for IN-SITU Visualizations • Allows optimizing resources for compute Simulation Index Rendering and render MPI MPI • Application Driven Updates, push simulation data. • Allows inter-operability between coarser MPI MPI and finer meshes, adjusting data updates. . . . . • Maintains high frame rates and allows interaction with the volume. MPI MPI • Can couple multiple physics apps to a single rendering app. 42
Steering Simulations Coupling • Steering is Application Specific Simulation Rendering • Steering simulations requires handling interrupts. Time Interrupt handler Time S1 S1 • Interrupt communicated through backward coupling. Time Time • A general approach by S2 S2 Function scalar/vector interrupts, and user applied to defined function to handle the fields variables of simulation. Time Time S3 S3 . . . . 43
Summary: In-situ Visualization • Index enables better insights into simulation data through professional visualization techniques • Scalability is the enabler for HPC in-situ visualization. • Multiphysics coupling is the key to scalability, and resource management for in-situ. 44
SELF-PACED LABS Interactive HPC Volume Visualization in ParaView NVIDIA IndeX for ParaView plugin hands-on Location: Self-paced lab area on lower level ▪ Dates: Monday 1:00 – 5:00pm ▪ Tuesday 9:30 – 11.30am ▪ ▪ Wednesday 1:00 – 5:00pm 45
INTERACTIVE DEMO Interactiver HPC: Large Scale In-Situ Visualization using NVIDIA IndeX in ALYA MultiPhysics Live demonstration of in-situ visualization Interactive steering of simulation parameters Location: NVIDIA demo-booth in exhibit hall 1 ▪ 46
Christopher Lux, NVIDIA Vishal Mehta, BSC Marc Nienhaus, NVIDA
Recommend
More recommend
