V ISUALIZING HDF5 DATA WITH O PEN DX Ireneusz Szcze sniak John - - PowerPoint PPT Presentation

▶
v isualizing hdf5 data with o pen dx
SMART_READER_LITE
LIVE PREVIEW

V ISUALIZING HDF5 DATA WITH O PEN DX Ireneusz Szcze sniak John - - PowerPoint PPT Presentation

Jun 01, 2023 203 likes •440 views

V ISUALIZING HDF5 DATA WITH O PEN DX Ireneusz Szcze sniak John Cary Center for Integrated Plasma Studies University of Colorado at Boulder September 25, 2002 P RESENTATION S PLAN 2 P LAN OF PRESENTATION Introduction to HDF5 and

slide-1
SLIDE 1

VISUALIZING HDF5 DATA WITH OPENDX

Ireneusz Szcze´ sniak John Cary

Center for Integrated Plasma Studies University of Colorado at Boulder

September 25, 2002

slide-2
SLIDE 2

PRESENTATION’S PLAN 2

PLAN OF PRESENTATION

  • Introduction to HDF5 and OpenDX
  • CIPS modules for OpenDX
  • Vorpal uses of the CIPS modules
  • Basics of OpenDX module design
  • Summary and conclusions
slide-3
SLIDE 3

INTRODUCTION TO HDF5 AND OPENDX 3

HDF5

HIERARCHICAL DATA FILE V. 5

  • An improved version of the HDF4 file format
  • Organizes data in a hierarchy
  • Faster and less memory consuming than netCDF
  • Available on many platforms
  • Makes data portable
slide-4
SLIDE 4

INTRODUCTION TO HDF5 AND OPENDX 4

Figure: Sample file

OBJECTS OF HDF5

HDF5 offers:

  • datasets - multidimensional arrays,
  • groups - collections of objects,
  • compound datatypes,
  • links to objects,
  • attributes.
slide-5
SLIDE 5

INTRODUCTION TO HDF5 AND OPENDX 5

OPENDX THE OPEN SOURCE PROJECT BASED ON IBM’S VISUALIZATION DATA EXPLORER

  • Open source software
  • Good technical support
  • Used worldwide
  • Easy to visualize 2-D and 3-D data
  • Easy to learn
slide-6
SLIDE 6

INTRODUCTION TO HDF5 AND OPENDX 6

Figure: Sample module

OPENDX MODULE

  • Provides a function
  • Enhances the OpenDX capabilities
  • Can be standard or user given
  • Usually requires inputs
  • Usually produces outputs
slide-7
SLIDE 7

INTRODUCTION TO HDF5 AND OPENDX 7

PROGRAMMING IN OPENDX

  • Visual programming
  • You work as a plumber
  • Your program is data-driven
  • No new scripting language to learn
slide-8
SLIDE 8

INTRODUCTION TO HDF5 AND OPENDX 8

HDF5 AND OPENDX

  • OpenDX does not import HDF5 data
  • OpenDX has large memory requirements
  • HDF5 can contain large datasets
  • OpenDX would run out of memory when reading a large HDF5 file
  • Alternative: filter and convert HDF5 data before importing
slide-9
SLIDE 9

CIPS MODULES FOR OPENDX 9

CIPS MODULES FOR OPENDX

  • The dxhdf5 package gives two OpenDX modules
  • ImportHDF5Field imports fields from HDF5 files
  • ImportHDF5Species imports particles from HDF5 files
  • They let reduce memory requirements
  • Developed at CIPS this summer
  • Announced by OpenDX.org
  • Available at www-beams.colorado.edu/dxhdf5
slide-10
SLIDE 10

CIPS MODULES FOR OPENDX 10

IMPORTHDF5FIELD

  • Imports a slab of a field from an HDF5 file
  • The field must be based on a regular grid
  • The HDF5 file must be of a special structure
  • The user can specify:

– the lower corner, – thickness, – stride.

slide-11
SLIDE 11

CIPS MODULES FOR OPENDX 11

IMPORTHDF5FIELD IN ACTION

Complete field with every third grid point chosen.

slide-12
SLIDE 12

CIPS MODULES FOR OPENDX 12

IMPORTHDF5FIELD IN ACTION - CONTINUED

A slab of the filed with every grid point.

slide-13
SLIDE 13

CIPS MODULES FOR OPENDX 13

IMPORTHDF5SPECIES

  • Selects and imports particles from an HDF5 file
  • Each particle has its position and data elements
  • The HDF5 file must be of a special structure
  • The user can specify:

– minimal and maximal values for the position and data, – stride, – random.

slide-14
SLIDE 14

CIPS MODULES FOR OPENDX 14

IMPORTHDF5SPECIES IN ACTION

Complete set of particles.

slide-15
SLIDE 15

CIPS MODULES FOR OPENDX 15

IMPORTHDF5SPECIES IN ACTION - CONTINUED

Particles with u0 > 107 plus roughly 750 random particles.

slide-16
SLIDE 16

VORPAL USE OF THE MODULES 16

VORPAL, HDF5 AND OPENDX

  • Vorpal files are very large and numerous
  • Vorpal saves its results in HDF5 files
  • We want to use OpenDX to visualize Vorpal data
slide-17
SLIDE 17

VORPAL USE OF THE MODULES 17

OBSOLETE WAY OF IMPORTING VORPAL PARTICLES INTO OPENDX

HDF5 files Import ASCII data in OpenDX Select data with "selector" Convert selected data to ASCII format with "h5ptcls2dx.sh"

The scheme of the data processing.

slide-18
SLIDE 18

VORPAL USE OF THE MODULES 18

VORPAL AND THE DXHDF5 MODULES

  • Vorpal uses both ImportHDF5Field and ImportHDF5Species
  • The data is selected and imported without preprocessing
  • Using the dxhdf5 modules saves time and work
  • The modules do not supersede the selector program
slide-19
SLIDE 19

BASICS OF OPENDX MODULE DESIGN 19

BASICS OF OPENDX MODULE DESIGN

  • Three module types:

– inboard – outboard – runtime-loadable

  • Module’s definition must be provided (MDF file)
  • OpenDX Module Builder may help
  • Module implementation in C, C++ or Java
slide-20
SLIDE 20

BASICS OF OPENDX MODULE DESIGN 20

.

MDF FILE - MODULE DEFINITION FILE

The MDF file defines:

  • module’s name, category, description and executable file,
  • inputs, outputs, and their types, default values and names.

Sample MDF file:

MODULE SampleModule CATEGORY Import and Export DESCRIPTION sample module LOADABLE "/our_path/SampleModule"; INPUT number; integer; 100 ; number we need INPUT debugflag; flag; 0; enable debug messages OPTIONS false ; true OUTPUT ouroutput; field; some output

slide-21
SLIDE 21

SUMMARY AND CONCLUSIONS 21

SUMMARY AND CONCLUSIONS

  • HDF5 and OpenDX are useful and powerful tools
  • The dxhdf5 package helps one import HDF5 data in OpenDX
  • The modules are documented, and released
  • The modules satisfy their users
slide-22
SLIDE 22

THANK YOU!