yt: Astrophysically-Aware Analysis and Viz Matthew Turk (Columbia) - - PowerPoint PPT Presentation

▶
yt astrophysically aware analysis and viz
SMART_READER_LITE
LIVE PREVIEW

yt: Astrophysically-Aware Analysis and Viz Matthew Turk (Columbia) - - PowerPoint PPT Presentation

Mar 29, 2023 296 likes •1.33k views

yt: Astrophysically-Aware Analysis and Viz Matthew Turk (Columbia) Britton Smith (Michigan State) and the yt collaboration First, some motivation. the stars how to make a star 76% 24% hydrodynamics (example) There is only one sky.

slide-1
SLIDE 1

yt: Astrophysically-Aware Analysis and Viz

Matthew Turk (Columbia) Britton Smith (Michigan State) and the yt collaboration

slide-2
SLIDE 2

First, some motivation.

slide-3
SLIDE 3

the stars

slide-4
SLIDE 4
slide-5
SLIDE 5

how to make a star

slide-6
SLIDE 6
slide-7
SLIDE 7
slide-8
SLIDE 8

76% 24%

slide-9
SLIDE 9

hydrodynamics

slide-10
SLIDE 10

(example)

slide-11
SLIDE 11

There is only one sky.

(but there are many simulation codes)

slide-12
SLIDE 12
slide-13
SLIDE 13

Different methods, data structures, assumptions, IO methods, units, variable names, ...

slide-14
SLIDE 14
slide-15
SLIDE 15
slide-16
SLIDE 16

Analysis.

slide-17
SLIDE 17

astro-ph/1011.3514 yt-project.org

slide-18
SLIDE 18
slide-19
SLIDE 19
slide-20
SLIDE 20
slide-21
SLIDE 21

install script: Full dependency stack Source code Development environment GUI Sample data

slide-22
SLIDE 22

yt has been designed to address physical, not computational, entities.

slide-23
SLIDE 23
slide-24
SLIDE 24

The Universe is full of gas, dark matter and stars. yt makes it easy to access that material.

slide-25
SLIDE 25

Transparent IO, masking of overlapping data, load-on-demand, geometric and non- geometric selection, field generation, and common interfaces to different datatypes.

slide-26
SLIDE 26

Enzo, Orion, CASTRO, FLASH Chombo, Tiger, Athena, ART, RAMSES

yt is designed to be the lingua franca

  • f astrophysical codes.
slide-27
SLIDE 27

Objects (conceptual, uniformly accessible NumPy stores)

slide-28
SLIDE 28

Objects

Orthogonal Rays Non-orthogonal Rays 1D Slices Oblique Slices Projections 2D Spheres Rectangular Prisms Disks/Cylinders Inclined Boxes Clumps Extracted Regions Boolean combinations 3D

slide-29
SLIDE 29

Objects All respect unified interface: from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) ray = pf.h.ray([0.1, 0.2, 0.5], [0.4, 0.9, 0.1]) print ray[“Density”]

slide-30
SLIDE 30

Objects All respect unified interface: from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) sl = pf.h.slice(0, 0.5) print sl[“Density”]

slide-31
SLIDE 31

Objects All respect unified interface: from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) sp = pf.h.sphere(100.0/pf[‘au’], ‘max’) print sp[“Density”]

slide-32
SLIDE 32

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) v, c = pf.h.find_max(“Density”)

slide-33
SLIDE 33

Adding new fields should be easy.

slide-34
SLIDE 34

from yt.mods import * @derived_field(“Pressure”) def Pressure(field, data): return (data.pf["Gamma"] - 1.0) * \ data["Density"]*data["ThermalEnergy"]

slide-35
SLIDE 35

Scripts should be simple and clear.

slide-36
SLIDE 36

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) pc = PlotCollection(pf) pc.add_phase_sphere(1000.0, ‘au’, [“Density”, “Temperature”, “H2I_Fraction”]) pc.save()

slide-37
SLIDE 37

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) pc = PlotCollection(pf) pc.add_phase_sphere(1000.0, ‘au’, [“Density”, “Temperature”, “CellMassMsun”], weight = None) pc.save()

slide-38
SLIDE 38

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) pc = PlotCollection(pf) pc.add_slice(“Density”, 0) pc.set_width(0.5, ‘unitary’) pc.save()

slide-39
SLIDE 39

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) pc = PlotCollection(pf) pc.add_slice(“Density”, 0) pc.set_width(1.0, ‘pc’) pc.save()

slide-40
SLIDE 40

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) pc = PlotCollection(pf) pc.add_slice(“Density”, 0) pc.set_width(1000.0, ‘au’) pc.save()

slide-41
SLIDE 41

Some Highlights!

slide-42
SLIDE 42

(px, py, pdx, pdy, z)

Projections

slide-43
SLIDE 43
slide-44
SLIDE 44

(px, py, pdx, pdy, z)

Image Buffer

slide-45
SLIDE 45

(px, py, pdx, pdy, z)

Image Buffer

slide-46
SLIDE 46

Project once, pixelize many

slide-47
SLIDE 47

Slices

slide-48
SLIDE 48
slide-49
SLIDE 49
slide-50
SLIDE 50

Oblique Slices

slide-51
SLIDE 51

Isocontour Extraction

slide-52
SLIDE 52

Parallelism

slide-53
SLIDE 53

Parallelism

Embarassingly Parallel Spatial Decomposition Decomposed by load or IO characteristics Helper functions to decompose the domain

slide-54
SLIDE 54

Parallelism

Embarassingly Parallel Spatial Decomposition Quantities Profiles Slices Projections Volume Rendering Halo Finding

slide-55
SLIDE 55
slide-56
SLIDE 56
slide-57
SLIDE 57

Multi-level parallelism: dynamic workgroups, communicators, subgroups and task queues

slide-58
SLIDE 58

Volume Rendering

slide-59
SLIDE 59

dIν ds = jν − ανIν

Designed around integrating through a volume: visualization is a side effect.

slide-60
SLIDE 60

Credit: Sam Skillman, 3rd Place at SciDAV Viz Night

slide-61
SLIDE 61
slide-62
SLIDE 62
slide-63
SLIDE 63

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) v, c = pf.h.find_max("Density") L = [1.0, 1.0, 1.0] W = 1000.0/pf['au'] tf = vr.ColorTransferFunction((-14.0, -4)) tf.add_layers(8) cam = pf.h.camera(c, L, W, 1024, tf) cam.snapshot()

slide-64
SLIDE 64
slide-65
SLIDE 65

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) v, c = pf.h.find_max("Density") sp = pf.h.sphere(c, 1000.0/pf['au']) L = sp.quantities["AngularMomentumVector"]() W = 1000.0/pf['au'] tf = vr.ColorTransferFunction((-14.0, -4)) tf.add_layers(8) cam = pf.h.camera(c, L, W, 1024, tf) cam.snapshot()

slide-66
SLIDE 66
slide-67
SLIDE 67

cam.zoom(100.0) cam.snapshot()

slide-68
SLIDE 68
slide-69
SLIDE 69
slide-70
SLIDE 70

from yt.mods import * pf = load(“DD1701/DD1701”) v, c = pf.h.find_max("Density") L = [1.0, 1.0, 1.0] W = 100.0/pf['mpc'] tf = vr.PlanckTransferFunction() cam = pf.h.camera(c, L, W, 1024, tf) cam.snapshot()

slide-71
SLIDE 71
slide-72
SLIDE 72
slide-73
SLIDE 73

Off-axis Projection

slide-74
SLIDE 74
slide-75
SLIDE 75

Canned Analysis Tasks

slide-76
SLIDE 76

Absorption Spectrum Coordinate Transformations Halo Finding Mass Functions Merger Trees Halo Profiling Level Sets Light Cones Light Rays Time Series Star Analysis Two-Point Analysis

slide-77
SLIDE 77

Level Sets

slide-78
SLIDE 78

Synthetic Spectra

slide-79
SLIDE 79

Two-Point Functions

slide-80
SLIDE 80

Three Halo Finders: Standard HOP Friends of Friends Parallel HOP Rockstar (beta)

slide-81
SLIDE 81
slide-82
SLIDE 82

co-scheduled & in situ viz

slide-83
SLIDE 83

Simulation yt

Process

Thin NumPy wrappers and stop-n-go

slide-84
SLIDE 84

On Disc → In-situ

slide-85
SLIDE 85

Simulation yt

Inter-comm

Fire and forget, no embedded interpreter

slide-86
SLIDE 86

Developing as a Team

slide-87
SLIDE 87

Code review: eyes on (nearly) every changeset

slide-88
SLIDE 88

Forky development: very low barrier to entry; everything comes in the box.

slide-89
SLIDE 89

Testing: answer as well as integration tests get run every 30 minutes.

slide-90
SLIDE 90

80,000 lines of code Python, Cython, C 20 contributors (60+ users) Contributors from 10+ institutions

8AM 6PM

slide-91
SLIDE 91
slide-92
SLIDE 92

Challenges

slide-93
SLIDE 93

Low bus factor.

slide-94
SLIDE 94

Chores nobody wants to do.

slide-95
SLIDE 95

The import problem!

slide-96
SLIDE 96

What’s next?

slide-97
SLIDE 97

initialization

slide-98
SLIDE 98

better astrophysical

  • bject selection
slide-99
SLIDE 99

solvers

slide-100
SLIDE 100

non-astrophysical data

slide-101
SLIDE 101

Thank you.

Tom Abel David Collins Oliver Hahn Cameron Hummels Ji-hoon Kim Steffen Klemer Kacper Kowalik Eve Lee Chris Malone Christopher Moody Andrew Myers Michael Norman Brian O'Shea Stella Offner Jeff Oishi Devin Silvia Sam Skillman Stephen Skory Britton Smith Casey Stark Elizabeth Tasker Matthew Turk John Wise John ZuHone

yt-project.org