Data-Intensive Applications on Numerically-Intensive Supercomputers David Daniel / James Ahrens Los Alamos National Laboratory July 2009 Operated by the Los Alamos National Security, LLC for the DOE/NNSA
Interactive visualization of a billion-cell plasma physics simulation Operated by the Los Alamos National Security, LLC for the DOE/NNSA
VPIC – A case study of visualization on a supercomputing platform • Goal: Running simulation on 4096 ASC Roadrunner processors – Computing an 8192x512x512 problem = 2 billion cell problem – VPIC per processor dump files = 100s of GB per time step • Data reduction, prioritization and visualization on the supercomputer combines will help this science team be successful Operated by the Los Alamos National Security, LLC for the DOE/NNSA
The RoadRunner Universe project • First Petascale Cosmology • The RoadRunner Universe Simulations Data Challenge – New scalable hybrid code – Individual trillion particle runs designed for heterogeneous generate 100s of TB of raw architectures data – New algorithmic ideas for high • Must carry out “on the fly” performance analysis • Domain overloading with – KD tree-based halo finder particle caches parallelized with particle • Digital filtering to reduce overloading communication across Opteron/Cell layer • >50 times speed-up over conventional codes Operated by the Los Alamos National Security, LLC for the DOE/NNSA June 2009-4
Visualization tools provide a high-level data-oriented programming model • Visualization tools are • Optimize access to programmable numerically-intensive architecture – Uses a data-flow program graph… – Multi-resolution out-of-core data visualization • Visualization tools provide their own run-time system Operated by the Los Alamos National Security, LLC for the DOE/NNSA
Multi-resolution out-of-core visualization Operated by the Los Alamos National Security, LLC for the DOE/NNSA
Multi-resolution out-of-core visualization Operated by the Los Alamos National Security, LLC for the DOE/NNSA
Impact: Using a data-oriented programming model we can directly address the fundamental massive scale visualization challenge • Supports an approach for the massive data visualization problem • Data reduction, prioritization, multi-resolution and out-of-core processing Performance comparison Time vs. image quality 14.00 1 normalized RMS CIELUV error 0.9 0.8 12.00 0.7 0.6 10.00 0.5 7.80 0.4 seconds 8.00 0.3 render 0.2 6.00 one time mesh 0.1 0 one time read 4.00 0 5 10 15 20 25 30 3.72 seconds 2.00 0.82 0.20 1.14 0.84 0.14 0.00 0.04 0.00 full data (1 low res data multi-res data GB) (1 MB) (38 MB) Operated by the Los Alamos National Security, LLC for the DOE/NNSA
There is a “Middle Way” between numerically- intensive and data-intensive supercomputing • Numerically-intensive supercomputing approach – Massive FLOPS • Data-intensive supercomputing (DISC) approach – Massive data • We are exploring this “Middle Way” by necessity for interactive scientific analysis and visualization of massive data • DISC using a traditional HPC platform (compare Bryant) – 1. Data as first-class citizen • In-situ analysis for Roadrunner Universe application – 2. High-level data oriented programming model • Programmable visualization tools • Multi-resolution out-of-core visualization – 3. Interactive access – human in the loop • Visualization on the supercomputing platform – 4. Reliability Operated by the Los Alamos National Security, LLC for the DOE/NNSA
Los Alamos Computer Science Symposium (LACSS) October 13, 14, 2009 La Fonda Hotel, Santa Fe, NM http://www.lanl.gov/conferences/lacss/2009/ Non-Traditional Programming Models for High-Performance Computing This year's LACSS will focus on Data-Intensive Architectures and Applications. The similarities and differences with "traditional" HPC will be explored. Other non-traditional HPC themes may be added. Attendance open to all. Speakers likely by invitation only but contact Al McPherson <mcpherson@lanl.gov> if you are interested. Operated by the Los Alamos National Security, LLC for the DOE/NNSA
Recommend
More recommend
