Analyzing Simulated Data Matthew Turk There is only one sky. (but - - PowerPoint PPT Presentation

▶
analyzing simulated data
SMART_READER_LITE
LIVE PREVIEW

Analyzing Simulated Data Matthew Turk There is only one sky. (but - - PowerPoint PPT Presentation

Jun 27, 2023 244 likes •1.09k views

Analyzing Simulated Data Matthew Turk There is only one sky. (but there are many simulation codes) Different methods, data structures, assumptions, IO methods, units, variable names, ... Analysis. astro-ph/1011.3514 yt-project.org

slide-1
SLIDE 1

Analyzing Simulated Data

Matthew Turk
slide-2
SLIDE 2

There is only one sky.

(but there are many simulation codes)

slide-3
SLIDE 3
slide-4
SLIDE 4

Different methods, data structures, assumptions, IO methods, units, variable names, ...

slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7

Analysis.

slide-8
SLIDE 8

astro-ph/1011.3514 yt-project.org

slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12

install script: Full dependency stack Source code Development environment GUI Sample data

slide-13
SLIDE 13

yt has been designed to address physical, not computational, entities.

slide-14
SLIDE 14

Process

◮ Read ◮ Correlate ◮ Process ◮ Visualize
slide-15
SLIDE 15
slide-16
SLIDE 16

The Universe is full of gas, dark matter and stars. yt makes it easy to access that material.

slide-17
SLIDE 17

Transparent IO, masking of overlapping data, load-on-demand, geometric and non- geometric selection, field generation, and common interfaces to different datatypes.

slide-18
SLIDE 18

Enzo, Orion, CASTRO, FLASH Chombo, Tiger, Athena, ART, RAMSES

yt is designed to be the lingua franca

  • f astrophysical codes.
slide-19
SLIDE 19

Objects (conceptual, uniformly accessible NumPy stores)

slide-20
SLIDE 20

Objects

Orthogonal Rays Non-orthogonal Rays 1D Slices Oblique Slices Projections 2D Spheres Rectangular Prisms Disks/Cylinders Inclined Boxes Clumps Extracted Regions Boolean combinations 3D

slide-21
SLIDE 21

Objects All respect unified interface: from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) ray = pf.h.ray([0.1, 0.2, 0.5], [0.4, 0.9, 0.1]) print ray[“Density”]

slide-22
SLIDE 22

Objects All respect unified interface: from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) sl = pf.h.slice(0, 0.5) print sl[“Density”]

slide-23
SLIDE 23

Objects All respect unified interface: from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) sp = pf.h.sphere(100.0/pf[‘au’], ‘max’) print sp[“Density”]

slide-24
SLIDE 24

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) v, c = pf.h.find_max(“Density”)

slide-25
SLIDE 25

Adding new fields should be easy.

slide-26
SLIDE 26

from yt.mods import * @derived_field(“Pressure”) def Pressure(field, data): return (data.pf["Gamma"] - 1.0) * \ data["Density"]*data["ThermalEnergy"]

slide-27
SLIDE 27

Scripts should be simple and clear.

slide-28
SLIDE 28

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) pc = PlotCollection(pf) pc.add_phase_sphere(1000.0, ‘au’, [“Density”, “Temperature”, “H2I_Fraction”]) pc.save()

slide-29
SLIDE 29

from yt.mods import * pf = load(“DataDump0155.dir/DataDump0155”) pc = PlotCollection(pf) pc.add_phase_sphere(1000.0, ‘au’, [“Density”, “Temperature”, “CellMassMsun”], weight = None) pc.save()

slide-30
SLIDE 30

Slices

slide-31
SLIDE 31
slide-32
SLIDE 32
slide-33
SLIDE 33 from yt.mods import * pf = load(’galaxy0030/galaxy0030’) p = SlicePlot(pf, 2, ’Density’, ’c’, (200,’kpc’)) p.save(’Galaxy’)
slide-34
SLIDE 34
slide-35
SLIDE 35 from yt.mods import * pf = load(’galaxy0030/galaxy0030’) p = SlicePlot(pf, 2, ’Density’, ’c’, (200,’kpc’)) p.set_width(20, ’kpc’) p.save("GalaxyZoom")
slide-36
SLIDE 36
slide-37
SLIDE 37

(px, py, pdx, pdy, z)

Projections

slide-38
SLIDE 38
slide-39
SLIDE 39 from yt.mods import * pf = load(’galaxy0030/galaxy0030’) p = ProjectionPlot(pf, 2, ’Density’, ’c’, (20,’kpc’)) p.save(’Galaxy’)
slide-40
SLIDE 40
slide-41
SLIDE 41

Oblique Slices

slide-42
SLIDE 42

Off-axis Projection

slide-43
SLIDE 43

Project once, pixelize many

slide-44
SLIDE 44

(px, py, pdx, pdy, z)

Image Buffer

slide-45
SLIDE 45

(px, py, pdx, pdy, z)

Image Buffer

slide-46
SLIDE 46

Parallelism

slide-47
SLIDE 47

Parallelism

Embarassingly Parallel Spatial Decomposition Decomposed by load or IO characteristics Helper functions to decompose the domain

slide-48
SLIDE 48

Parallelism

Embarassingly Parallel Spatial Decomposition Quantities Profiles Slices Projections Volume Rendering Halo Finding

slide-49
SLIDE 49
slide-50
SLIDE 50
slide-51
SLIDE 51

Multi-level parallelism: dynamic workgroups, communicators, subgroups and task queues

slide-52
SLIDE 52

Volume Rendering

slide-53
SLIDE 53

dIν ds = jν − ανIν

Designed around integrating through a volume: visualization is a side effect.

slide-54
SLIDE 54
slide-55
SLIDE 55

from yt.mods import * pf = load(“DD1701/DD1701”) v, c = pf.h.find_max("Density") L = [1.0, 1.0, 1.0] W = 100.0/pf['mpc'] tf = vr.PlanckTransferFunction() cam = pf.h.camera(c, L, W, 1024, tf) cam.snapshot()

slide-56
SLIDE 56
slide-57
SLIDE 57
slide-58
SLIDE 58
slide-59
SLIDE 59

Canned Analysis Tasks

slide-60
SLIDE 60

Absorption Spectrum Coordinate Transformations Halo Finding Mass Functions Merger Trees Halo Profiling Level Sets Light Cones Light Rays Time Series Star Analysis Two-Point Analysis

slide-61
SLIDE 61

Level Sets

slide-62
SLIDE 62

Synthetic Spectra

slide-63
SLIDE 63

Two-Point Functions

slide-64
SLIDE 64

Three Halo Finders: Standard HOP Friends of Friends Parallel HOP Rockstar (beta)

slide-65
SLIDE 65
slide-66
SLIDE 66

co-scheduled & in situ viz

slide-67
SLIDE 67

Simulation yt

Process

Thin NumPy wrappers and stop-n-go

slide-68
SLIDE 68

On Disc → In-situ

slide-69
SLIDE 69

Simulation yt

Inter-comm

Fire and forget, no embedded interpreter

slide-70
SLIDE 70

Developing as a Team

slide-71
SLIDE 71

Code review: eyes on (nearly) every changeset

slide-72
SLIDE 72

Forky development: very low barrier to entry; everything comes in the box.

slide-73
SLIDE 73

Testing: answer as well as integration tests get run every 30 minutes.

slide-74
SLIDE 74

80,000 lines of code Python, Cython, C 20 contributors (60+ users) Contributors from 10+ institutions

8AM 6PM

slide-75
SLIDE 75
slide-76
SLIDE 76
slide-77
SLIDE 77
slide-78
SLIDE 78
slide-79
SLIDE 79
slide-80
SLIDE 80
slide-81
SLIDE 81
slide-82
SLIDE 82
slide-83
SLIDE 83

Contributors

Tom Abel David Collins Andrew Cunningham Nathan Goldbaum Cameron Hummels Ji-hoon Kim Steffen Klemer Kacper Kowalik Michael Kuhlen Eve Lee Chris Malone Chris Moody Andrew Myers Jeff Oishi Jean-Claude Passy Thomass Robitaille Anna Rosen Anthony Scopatz Devin Silvia Sam Skillman Stephen Skory Britton Smith Geoffrey So Casey Stark Elizabeth Tasker Rick Wagner John Wise John ZuHone
slide-84
SLIDE 84

How to get started

http://yt-project.org/ http://yt-project.org/issac2012/