Fourteen years of Cactus Community Frank L offler Center for - - PowerPoint PPT Presentation

▶
fourteen years of cactus community
SMART_READER_LITE
LIVE PREVIEW

Fourteen years of Cactus Community Frank L offler Center for - - PowerPoint PPT Presentation

Nov 30, 2023 129 likes •414 views

Fourteen years of Cactus Community Frank L offler Center for Computation and Technology Louisiana State University, Baton Rouge, LA September 6th 2012 Frank L offler Fourteen years of Cactus Community 2012-09-06 Outline Motivation

slide-1
SLIDE 1

Fourteen years of Cactus Community

Frank L¨

  • ffler

Center for Computation and Technology Louisiana State University, Baton Rouge, LA

September 6th 2012

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-2
SLIDE 2

Outline

Motivation scenario from Astrophysics Cactus structure technically Cactus structure socially Future directions

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-3
SLIDE 3

Challenging Astrophysics Problems

Black Holes and Neutron Stars Supernovae Cosmology Gravitational Wave Data Analysis

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-4
SLIDE 4

Gravitational Wave Physics

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-5
SLIDE 5

Solving Einstein’s Equations

Einstein equations: Gµν = 8πTµν

12 fully 2nd order PDE evolution equations 4 coupled constraint equations 4 gauge conditions GR hydrodynamics, MHD, radiation transport

Fully numerical 3D models needed

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8
slide-9
SLIDE 9

Computational Requirements

Unigrid scales to hundreds of thousands of cores Productions runs use ≈ 10 levels of mesh refinement, nested grids of size ≈ 60x60x60 Current mesh refinement runs scale up to ≈ 10k cores Runtime from weeks to few months

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-10
SLIDE 10

Challenge

Many scientific/engineering components

Physics Mathematics CFD

Many numerical algorithm components

Finite difference, finite volume, spectral methods Structured or unstructured meshes, mesh refinements Multipatch and multimodel

Many different computational components

Parallelism (MPI, OpenMP, ...) Parallel I/O (e.g. Checkpointing) Visualization

Challenge

Defining good abstractions to bring these together in a unified, scalable framework, enabling science

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-11
SLIDE 11
slide-12
SLIDE 12

Cactus Framework Structure

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-13
SLIDE 13

Cactus Core: The Flesh

ANSI-C and Perl, C++ Independent of all other components Unified error handling Build system Parameter parsing/steering Global variable management Rule-based scheduler Extensible APIs

Parallel Operations Input/Output Reduction Interpolation Timers

Functionality provided by (swappable) components

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-14
SLIDE 14

Cactus Components: Thorns

C, C++, Fortran 77, Fortran 90 Typically not implementing

grid setup and memory allocation input/output interpolation, reductions

Encapsulating some functionality

initial data boundary conditions evolution systems equations of state remote steering (e.g. https server)

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-15
SLIDE 15

Basis Module Overview

Basis for scalable algorithm development Most used: finite differences on structured meshes Parallel driver components

Simple Unigrid Carpet: Multipatch, Mesh-refinement

Method of lines Interfaces to external Libraries/Tools Interface to elliptic solvers (e.g. PETSc, Lorene) Input/Output: HDF5 Visualization: VisIt, OpenDX, Vish Other: PAPI, Hypre, Saga, Flickr, Twitter

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-16
SLIDE 16

Convenience Tools

GetComponents Simfactory Formaline

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-17
SLIDE 17

Cactus as growing project

Cactus

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-18
SLIDE 18

Cactus as growing project

Cactus

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-19
SLIDE 19

Cactus as growing project

Cactus

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-20
SLIDE 20

Cactus as growing project

Cactus

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-21
SLIDE 21

Cactus as growing project

Cactus

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-22
SLIDE 22

Social structure

Few Core Members: Gabrielle Allen, Steven R. Brandt, Frank L¨

  • ffler,

Erik Schnetter, ... Developers: about 50 worldwide Many more users Cactus Community Open Source Yearly releases Mailing lists Issue tracker IRC support channel Tutorials Web, HPC allocations, Repositories, ...

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-23
SLIDE 23

Guiding Principles

Open, community-driven software development Separation of physics software and computational infrastructure Stable interfaces, allowing extensions Simplify usage where possible:

Doing science >> Running a simulation Students need to know a lot about physics (meaningful initial conditions, numerical stability, accuracy/resolution, have patience, have curiosity, develop a “gut feeling” for what is right ...) Cactus Toolkit cannot give that, however: Open codes that are easy to use allow to concentrate on these things!

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-24
SLIDE 24

Credits, Citations

In academics: citations, citations, citations! In Cactus: Open and free source No requirement to cite anything However: requested to cite a few publications Which publications:

Few for the Cactus framework Some components list a few as well List published on website and manage through publication database

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-25
SLIDE 25

Future

From certain to more speculative: Multiblock techniques GPU support Requirement-based scheduling Discontinuous Galerkin instead of finite differences Support for unstructured grids Completely requirement-based programming (MPI→ParalleX?)

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-26
SLIDE 26

Tools: GetComponents

Task: Collect software from various repositories at different sites Example simulation assembly: Cactus Flesh and Toolkit (svn.cactuscode.org) Core Einstein Toolkit (svn.einsteintoolkit.org) Carpet AMR (carpetcode.org, hg) Tools, Parameter Files and Data (svn.einsteintoolkit.org) Group Modules (x.groupthorns.org) Individual Modules (x.mythorns.org) x: cvs, svn, darcs, git, hg, http

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-27
SLIDE 27

Tools: Simulation Factory

http://www.simfactory.org/

Task: Provide support for common, repetitive steps: Access remote systems, synchronize source code trees Configure and build on different systems semi-automatically Provide maintained list of supercomputer configurations Manage simulations (follow “best practices”, avoid human errors)

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06

slide-28
SLIDE 28

Tools: Formaline

Task: Ensure that simulations are and remain repeatable, remember exactly how they were performed Take snapshots of source code, system configuration; store it in executable and/or git repository Tag all output files

Frank L¨

  • ffler

Fourteen years of Cactus Community 2012-09-06