[PPT] - The Scientific Data Management The Scientific Data Management PowerPoint Presentation

SLIDE 1

Arie Shoshani

The Scientific Data Management The Scientific Data Management Center Center

Arie Shoshani (PI) Lawrence Berkeley National Laboratory Lawrence Berkeley National Laboratory

DOE Laboratories

ANL: Rob Ross LBNL: Doron Rotem LLNL: Chandrika Kamath ORNL: Nagiza Samatova PNNL: Terence Critchlow Jarek Nieplocha

Universities

NCSU: Mladen Vouk NWU: Alok Choudhary UCD: Bertram Ludaescher SDSC: Ilkay Altintas UUtah: Claudio Silva

Co-Principal Investigators

Centers/Institutes meeting, October 24-25, 2008

SLIDE 2

Arie Shoshani

Problems and Mandate Problems and Mandate

Why is Managing Scientific Data Important for Scientific

Why is Managing Scientific Data Important for Scientific Inv Investig stigations? ations?

Sheer volume and increasin

Sheer volume and increasing complexity of data be g complexity of data being collected are already ing collected are already interferi interfering with the scient g with the scientif ific ic inve investigat stigation process ion process

Managing the data by scie

Managing the data by scientists greatly wastes scientists e ntists greatly wastes scientists effective t ctive time in me in performin rforming th their applicatio eir applications work work

Data collection, storage, tr

Data collection, storage, transfer, and archival often ansfer, and archival often co confli nflict wi ct with effec th effectivel ely us using com ing computati utational nal resources resources

Effectively managing, and analyzing th

Effectively managing, and analyzing this data and associated metadata is data and associated metadata requires a comprehensive, end-to-end requires a comprehensive, end-to-end approach that enco approach that encompasses all of t mpasses all of the e stages from the initial da stages from the initial data acquisition to the fi ta acquisition to the final analysis of nal analysis of the data the data

Enable s

Enable scientists to most effect ientists to most effectively discover new knowledge by ively discover new knowledge by removing data manag removing data management bottlenecks, and enabling effective ment bottlenecks, and enabling effective data analysi data analysis

Improve productivity of data

Improve productivity of data management i management infrastructure frastructure

Taking away the burd

Taking away the burden f en from scie

m scientists

ntists

Engaging Scientists, education

Engaging Scientists, education

SLIDE 3

Arie Shoshani

Focus of SDM center Focus of SDM center

high performance

high performance

fast

fast, scala , scalable le

Paralle

Parallel I/O, para I/O, paralle llel fi file le sy systems stems

Inde

Indexing, ng, data m data movement vement

Usability and effectiveness

Usability and effectiveness

Easy-to-use tools and interfaces

Easy-to-use tools and interfaces

Use of w

Use of workfl flow

w, dashboa

dashboards ds

end-to-end use (data and metadata)

end-to-end use (data and metadata)

En

Enabling data underst abling data understan anding ding

Paralleli

Parallelize analysis tools e analysis tools

Streamli

Streamline use of analysi ne use of analysis tools tools

Real-time data search tools

Real-time data search tools

Su

Sustain ainability bility

robustness

robustness

Productize software

Productize software

work with vendors, computing

work with vendors, computing centers centers

Establish dialog with scientists

Establish dialog with scientists

Outreach,

Outreach,

partne

partner w r with scie th scientists ntists, ,

education (students,

education (students, scie scientists ntists)

SLIDE 4

Arie Shoshani

Organization Organization of the center:

f the center:

ba based on three-la sed on three-layer org yer organization of technologie nization of technologies

Integrated approach:

To provide a

To provide a sc scie ientif ntific ic workflow and dashboard workflow and dashboard cap capability ability

To support data mining and

To support data mining and analysis tools analysis tools

To accelerate sto

To accelerate storage and age and acces access to data s to data

Benefits scientists by

Hiding underlying parallel

Hiding underlying parallel technology technology

End-to-end support of

End-to-end support of applications applications

Permitt

Permitting assembly of ng assembly of modules using w modules using workf rkflow

w

descr descript ption tool

n tool
Trac

Tracking king data management data management tasks through we tasks through web-based b-based dashboa dashboards ds

Parallel NetCDF Parallel Virtual File System Storage Resource Manager (SRM)

Hardware, Operating Systems, and Storage Systems

Data Analysis and Feature Identification Active Storage

Data Mining and Analysis (DMA) Layer Storage Efficient Access (SEA) Layer

Specialized Workflow components

Scientific Process Automation (SPA) Layer

Workflow Management Engine (Kepler) Analysis Parallel R Statistical Efficient indexing (Bitmap Index) Adaptable I/O System (ADIOS) Parallel I/O (ROMIO) Scientific Dashboard

SLIDE 5

Arie Shoshani

High Performance Technologies High Performance Technologies Usability and effectiveness Usability and effectiveness Enabling Data Understanding Enabling Data Understanding Results Results

SLIDE 6

Arie Shoshani

The I/O Software Stack The I/O Software Stack

SLIDE 7

Arie Shoshani

PVFS on IBM Blue Gene/P PVFS on IBM Blue Gene/P

SLIDE 8

Arie Shoshani

Speeding data transfer with PnetCDF Speeding data transfer with PnetCDF

P0 P0 P1 P1 P2 P2 P3 P3

netCDF netCDF Parallel File System Parallel File System Parallel netCDF Parallel netCDF

P0 P0 P1 P1 P2 P2 P3 P3

Parallel File System Parallel File System

Illustration: A. Tovey Illustration: A. Tovey

Early performance testing showed PnetCDF outperformed HDF5 for some critical access patterns. The HDF5 team has responded by improving their code for these patterns, and now these teams actively collaborate to better understand application needs and system characteristics, leading to I/O performance gains in both libraries. Enables high performance parallel I/O to netCDF data sets Achieves up to 10-fold performance improvement

ver HDF5

Inter-process communication

Contacts: Rob Ross, ANL, Alok Choudhari, NWU

SLIDE 9

Arie Shoshani

Improving IO in accelerator design simulation Improving IO in accelerator design simulation

n Jaguar/Cray XT*
n Jaguar/Cray XT*
Application: SLAC accelerator design

Application: SLAC accelerator design

Omeg

ega3P: a3P: simulatio simulation program th program that u at uses es higher-order te higher-order tetrahedral elements trahedral elements

Had bad reading patterns that do not scale

Had bad reading patterns that do not scale

Use netCDF files

Use netCDF files

(*) Lie-Quan (Rich) Lee (SLAC) and Stephen Hodson (ORNL)

Before (in seconds)

N-CPUs Writing Time Solver Time 128 30.27 634.74 256 59.26 324.16 512 146.24 163.30 1024 340.15 94.86 2048 499.21 45.86 4096 965.64 26.08

Time for Writing File >> Time for Solver !!!

Time for Writing File >> Time for Solver !!!

Scaling from Regular meshes To adaptive meshes

SLIDE 10

Arie Shoshani

Using Parallel-netCDF Using Parallel-netCDF instead of Netcdf and using MPI_Info instead of Netcdf and using MPI_Info

Contact: Alok Choudhari, NWU

I/O Time

200 400 600 800 1000 1200 128 256 512 1024 2048 4096 num of CPUs Time in seconds Writing-netCDF Writing Parallel- netCDF Solver time

Time for writing data reduced 100 times Time for Writing File << Time for Solver Expected to behave better for larger problem sizes.

After (in seconds)

NCPUs Writing Time Solver Time

512 1.50 163.30 1024 3.27 94.86 2048 7.90 45.86

SLIDE 11

Arie Shoshani

Parallel netCDF (no hints) Parallel netCDF (no hints)

Block dep

Block depiction of 2 ction of 28 GB file GB file

Record variable scattered

Record variable scattered

Reading i

Reading in way too much data way too much data!

Y axis larger here

Y axis larger here

Default “cb_buffer_size

Default “cb_buffer_size” hint not good for int not good for interleaved netCDF record variables interleaved netCDF record variables

ffset

time

SLIDE 12

Arie Shoshani

Parallel netCDF (hints) Parallel netCDF (hints)

With tuning, much le

With tuning, much less reading ss reading

Bette

Better effi r efficiency, bu ciency, but still short t still short of MPI-IO

f MPI-IO
Still some overlap

Still some overlap

“cb_buffer_size”

“cb_buffer_size” now size of one netCDF

w size of one netCDF

record record

Better effi

Better efficiency, at slight perf cost ciency, at slight perf cost

ffset

time

SLIDE 13

Arie Shoshani

Parallel netCDF (current SVN) Parallel netCDF (current SVN)

Development effort to relax netCDF file

Development effort to relax netCDF file format limits format limits

No need for re

No need for record variables cord variables

Data nice and comp

Data nice and compact like MPI-IO and HDF5 act like MPI-IO and HDF5

Ra

Rank nk 0 rea 0 reads head s header, broadcasts to , broadcasts to ot

the

hers rs

Mu

Much more sc ch more scal alable approa approach ch

Approach

chin ing M g MPI- I-IO effic efficienc ency

Maintains netCDF be

Maintains netCDF benefits nefits

Portab

rtable, se , self lf-d

descr

crib ibing ing, etc. etc.

ffset

time

Contacts: Rob Ross, ANL, Alok Choudhari, NWU

SLIDE 14

Arie Shoshani

MPI-IO MPI-IO Driver

Driver for Lustre

for Lustre

Availabl

Available for Beowulf e for Beowulf cl clus uster ters an and Cr d Cray ay XT XT

Overcome the restriction of a prop

Overcome the restriction of a proprietary MPI-IO stack on Cray XT rietary MPI-IO stack on Cray XT

Enabled arbitrary striping sp

Enabled arbitrary striping specification over Cray XT ecification over Cray XT

Lustre strip

tre stripe-align ligned file d file doma domain part in partit itioni ning ng

Released via MVAPICH-

Released via MVAPICH-1.0 and MPICH2-1.0.7 1.0 and MPICH2-1.0.7 IO Service PE

Application sysio liblustre

MPI-IO

SeaStar Torus Software Diagram on Cray XT

AD_Sysio AD_Lustre

Posix Syscall

IO Service PE Performance on an 80-node beowulf cluster

Contact: W. Yu, PNNL

SLIDE 15

Arie Shoshani

Caching with Caching with I/O

I/O delegate

delegate

Allocate a dedicate group of

Allocate a dedicate group of processes to perform I/O processes to perform I/O

Uses a small percentage (<

Uses a small percentage (< 10 %) of additional resource 10 %) of additional resource

Entire memory space at delega

Entire memory space at delegates can be used for caching tes can be used for caching

Co

Coll llec ecti tive I/O I/O o

ff-l

ff-load

I/O delegate size is 3%

A. Nisar, W. Liao, and A. Choudhary. Scaling

Parallel I/O Performance through I/O Delegate and Caching System. SC 2008.

SLIDE 16

Arie Shoshani

S3D-IO on Cray XT S3D-IO on Cray XT Performance/Productivity Performance/Productivity

Problem:

Problem:

Number of fi

Number of files created are often generated les created are often generated per processor per processor

Causes proble

Causes problems with ms with archiving and future archiving and future acces access s

Approach

Approach

Parallel I/O

Parallel I/O (MPI-IO (MPI-IO) op

ptimizatio

timization

One file p

e file per variable d r variable during I/O ring I/O

Requires multi-pro

ires multi-processo cessor co r coordinatio

rdination d

during ring I/O I/O

Achievement

Achievement

Shown to scale t

Shown to scale to 10s of thousands 10s of thousands of

f

processors on production systems processors on production systems

better perfo

tter performan mance b ce but elimin t eliminatin ating th g the need e need to create 100K+ fi to create 100K+ files les

16

Contacts: Rob Ross, ANL, Alok Choudhari, NWU

SLIDE 17

Arie Shoshani

Active Storage in Parallel File Systems Active Storage in Parallel File Systems

Active Storage exploits the old concept

Active Storage exploits the old concept of moving

f moving computing

computing to the data source to the data source

Avoi

Avoids ds data mov data movement across the networ ment across the network i k in pa parallel mach rallel machin ine b e by allo allowing wing application applications u use e com compute resourc ute resources on the I/O nod s on the I/O nodes of s of the cluster for data processing the cluster for data processing

Active Sto

tive Storag age efficien e efficiently deals tly deals with b with both strip th striped and n and netCDF tCDF files, elimin files, eliminating > 95% of th ating > 95% of the e networ network tra k traffi ffic i c in climate applications climate applications

Developed for Luster and PVFS fi

Developed for Luster and PVFS file le systems systems

P P P P Network FS FS

compute nodes I/O nodes

Y=foo(X)

x Y P P P P Network FS FS

compute nodes I/O nodes

Y=foo(X)

Active Storage Traditional Approach

Contact: J. Nieplocha et. al, PNNL

SLIDE 18

Arie Shoshani

Active Storage Application: Active Storage Application: High Throughput Proteomics High Throughput Proteomics

9.4 Tesla High Throughput Mass Spectrometer 1 Experiment per hour 5000 spectra per experiment 4 MByte per spectrum Per instrument: 20 Gbytes per hour 480 Gbytes per day

Application Problem

Given 2 float input number for target mass and tolerance, find all the possible protein sequences that would fit into specified range

Active Storage Solution

Each OST receives its part of the float pair sent by the client stores the resulting processing

utput in its Lustre OBD (object-based disk)

500 1000 1500 2000 3.55 7.1 14.2 28.4 56.8 Output Size (GB) C o m p le tio n T im e (s e c o n d s ) NoAS AS

Next generation technology will increase data rates x200

SLIDE 19

Arie Shoshani

Searching Problems in Data Intensive Sciences Searching Problems in Data Intensive Sciences

Find the

Find the HEP HEP colli collision events with sion events with the most the most dist distinct inct signature of Quark Gluon Plasma signature of Quark Gluon Plasma

Find the ignition

Find the ignition ke kerne rnels i in a a com combust ustion

n simulat

simulation

n
Track a l

Track a layer of exploding yer of exploding supernova supernova These are not typical These are not typical database searches: database searches:

Large hig

Large high-dimensional

dimensional data sets

data sets (1000 time steps X 1000 X 1000 X 1000 cells X 100 variables) (1000 time steps X 1000 X 1000 X 1000 cells X 100 variables)

No modification of i

No modification of individual dividual records during queri records during queries, i.e., s, i.e., append-only data append-only data

Complex questions: 500 < Tem

Complex questions: 500 < Temp < 1000 & < 1000 && CH3 > 10 CH3 > 10-4

4 && …

&& …

Large answers (hit thousands or millions of records)

Large answers (hit thousands or millions of records)

Seek collective features such as

Seek collective features such as regions of interest, histogra regions of interest, histograms, ms, etc. etc.

Other application domains

Other application domains: :

real-time analysis of ne

real-time analysis of network intrusion attacks twork intrusion attacks

fast

fast tracki tracking of c g of combust ustion f

n flame fr

e fronts over t

nts over time

me

accelerating mol

accelerating molecular docking i ecular docking in biology applications biology applications

query-dr

query-driven visualization iven visualization

SLIDE 20

Arie Shoshani

FastBit: accelerating analysis FastBit: accelerating analysis

f very large datasets
f very large datasets
Most data analysis algorithm

Most data analysis algorithm cannot handle a whole dataset cannot handle a whole dataset

Th

Therefo erefore, mo e, most data anal st data analysis task ysis tasks are performed on a subset of s are performed on a subset of the data the data

Need: very fast

Need: very fast indexin indexing for real-tim for real-time an e analysi alysis

FastBit is an extremely effic

FastBit is an extremely efficient compressed bitma ent compressed bitmap indexing indexing technology technology

Can

Can search search billion data values in seconds billion data values in seconds

FastBit improves the search speed by

FastBit improves the search speed by 10x – 10x – 100x of times than best known 00x of times than best known indexing methods indexing methods

Uses a

Uses a paten patented ed compression techniques compression techniques

Size: FastBit indexes are modest in size compared to well-

Size: FastBit indexes are modest in size compared to well- known database indexes known database indexes

On average abou

On average about 1/3 of data volum t 1/3 of data volume co compared to 3-4 tim mpared to 3-4 times in co s in common indexes (e.g. B-trees) indexes (e.g. B-trees)

SLIDE 21

Arie Shoshani

Query-Driven Visualization Query-Driven Visualization

Collaboration between SDM and VACET

Collaboration between SDM and VACET

Use FastBit indexes to effi

Use FastBit indexes to efficiently select th ciently select the mo most in st interesting d teresting data fo ta for visu r visualization alization

Above example: laser wakefield accelerator simulation

Above example: laser wakefield accelerator simulation

VORPAL produces 2D and 3D simulation

VORPAL produces 2D and 3D simulations of pa s of part rtic icle les in laser wa s in laser wake kefie field

Finding and tracking particles

Finding and tracking particles with la large momentum is ke rge momentum is key y to design the accelerator to design the accelerator

Brute-fo

Brute-force algorithm is rce algorithm is qua quadrat ratic (taking 5 minu (taking 5 minutes o es on 0. 0.5 mil particles 5 mil particles), F ), FastBit time i t time is lin linear ear in the number of results (takes 0.3 s, in the number of results (takes 0.3 s, 1000 X speedup 1000 X speedup) Contact: John Wu, Wes Bethel, LBNL

SLIDE 22

Arie Shoshani

Bin-Based Parallel Coordinate Display Bin-Based Parallel Coordinate Display

Integrate FastBit with H5Part, a HD

Integrate FastBit with H5Part, a HDF5 package for particle physics F5 package for particle physics data data

Use FastBit to compute histograms ef

Use FastBit to compute histograms effic ficiently ently

Bin-based parallel coordinate disp

Bin-based parallel coordinate display reduces the number of lines lay reduces the number of lines displayed on screen, reduces visual displayed on screen, reduces visual clutter, reduces response time clutter, reduces response time

FastBit further speeds up th

FastBit further speeds up the response time further e response time further

Contact: John Wu, Wes Bethel, LBNL

SLIDE 23

Arie Shoshani

FastBit Speeds up Historgraming FastBit Speeds up Historgraming

Time needed to compute desired histograms

Time needed to compute desired histograms

Custo

Custom code that directly code that directly uses the raw data directly uses the raw data directly

FastBit can be 1000 X faster

FastBit can be 1000 X faster than the custom code (left) than the custom code (left)

FastBit mai

FastBit maintains the performa tains the performance advantage on a parallel nce advantage on a parallel system system Lower is better

~ 104 X

SLIDE 24

Arie Shoshani

Flame Front Tracking in Combustion Simulation Flame Front Tracking in Combustion Simulation using FastBit using FastBit

Finding & tracking of combustion flame fronts Searching for regions that satisfy particular criteria is a

challenge. FastBit efficiently finds regions of interest.

Cell id Cell identificatio entification

Identify all cells that satisfy user specified conditions: “600 < Temperature < 700 AND HO2 concentr. > 10-7”

Region growing Region growing

Connect neighboring cells into regions

Region tracking Region tracking

Track the evolution of the features through time

Contact: John Wu, LBNL (kwu@lbl.gov)

SNL: Drs. J. Chen, W. Doyle NCSU: Dr. T. Echekki

SLIDE 25

Arie Shoshani

Use of FastBit for Molecular Docking Use of FastBit for Molecular Docking

FastBit has been released as open-source

FastBit has been released as open-source

Example of us

Example of use by others e by others

Jochen Schloss

Jochen Schlosser [schlosser@z r [schlosser@zbh. h.uni uni-ha hambur urg. g.de] Center for Bioinformatics, University of Hamburg

Problem: Structure-b

Problem: Structure-based virtual s sed virtual screening, standard reening, standard setup tup

Name Score 1bef -16,4 4dab -12,3 4d2a -11,6 … …

n ligands n docking runs

Hit list

One target protein

SLIDE 26

Arie Shoshani

Use of FastBit for Molecular Docking Use of FastBit for Molecular Docking

Specification of the descriptor as triangle geometry
Types of interaction centers

Types of interaction centers

Triangle

Triangle side side lengths lengths

Interaction directions

Interaction directions

80 bul

80 bulk di dimens mensions

ns
Receptors
Receptor d
r descripto

scriptors are g s are generated rated similarly similarly

Using complementary information where necess

Using complementary information where necessary ary

Idea: Usage of pharmacophore constraints on receptor triangles
Reduces num

Reduces number of que

f querie

ies

Improved query selectivity

Improved query selectivity because the pharmacophore because the pharmacophore tends to be inside tends to be inside the protein cavity the protein cavity

Results

TrixX-BMI is an efficient tool for virtual screening with average runtime in sub-second range
With pharmacophore constraints using FastBit, speedup 140 – 250

SLIDE 27

Arie Shoshani

High Performance Technologies High Performance Technologies Usability and effectiveness Usability and effectiveness Enabling Data Understanding Enabling Data Understanding

Results Results

SLIDE 28

Arie Shoshani

Automate the

Automate the monitoring monitoring pip pipelin line

transfer

transfer of s

f simul

mulation output t tion output to remote remote machine chine

execution o

ion of conv convers ersion routin

n routines

es, ,

imag

age cr e creation, tion, data ar data archi chiving ing

and the

and the code coupling code coupling pip pipelin line

Ru

Run s simu mulation tion on

n a

a large rge su super percompu puter ter

ch

check k line near stab ar stabil ilit ity on anot y on another her machine chine

Re-r

Re-run si simulati tion i if nee needed ed

Requi

Requirements for ents for Pe Petas tascale computing ale computing

Par

Parallel proc process essing ng

Robustness

Robustness

Configurab

Configurability lity

Easy to

Easy to us use

Das

Dashboard fro ard front-end nd

Dynamic mo

Dynamic monit nitoring ring

Contact: Scott Klasky, et. al, ORNL

Workflow automation requirements in Fusion Workflow automation requirements in Fusion Center for Plasma Edge Simulation (CPES) project Center for Plasma Edge Simulation (CPES) project

SLIDE 29

Arie Shoshani

Real-Time Real-Time Monitoring a simulation Monitoring a simulation Plus archiving Plus archiving

NetCDF files

NetCDF files

− Transfer Transfer files to e2e system on-the-fly files to e2e system on-the-fly − Generate plots Generate plots using grace library using grace library − Archi Archive e NetCDF files at the end of simulation NetCDF files at the end of simulation

Binary files

Binary files

− Transfer Transfer to e2e system using to e2e system using bbcp bbcp − Conv Convert ert to HDF5 format to HDF5 format − Start up AVS/Express Start up AVS/Express service service − Generate im Generate images ages with AVS/Express with AVS/Express − Archi Archive HDF5 files in larg HDF5 files in large chunks to HPSS e chunks to HPSS

Generate

Generate movies movies from the images from the images

Stop simulation if

Stop simulation if it does not progress it does not progress properly properly

SLIDE 30

Arie Shoshani

The Kepler Workflow The Kepler Workflow

Kepler is a workfl

Kepler is a workflow execu

w execution system based on Ptol

tion system based on Ptolemy (open source from UCB) emy (open source from UCB)

SDM center work

SDM center work is in the development of is in the development of components for scient components for scientifi ific applications applications (called actors) (called actors)

SLIDE 31

Arie Shoshani

visualize and compare shots

Real-time visualization and analysis capabilities Real-time visualization and analysis capabilities

n dashboard
n dashboard

SLIDE 32

Arie Shoshani

Simulation Steering: Coupling XGC-0 and M3D Codes

The

The processing loop t processing loop transfers data regularly ansfers data regularly

from the machine that runs XGC-0 (jag

from the machine that runs XGC-0 (jaguar) uar)

to anothe

to another machine (ewok) r machine (ewok)

for equilibrium and linear stabilit

for equilibrium and linear stability computations. y computations.

If th

If the linear stability test fails e linear stability test fails

a job is prepared and submitted to perform nonlinear parallel M3D-MPP

a job is prepared and submitted to perform nonlinear parallel M3D-MPP computation. computation.

These represent sub-workflows

(conceptual diagram)

SLIDE 33

Arie Shoshani

Using Kepler to Perform Parameter Using Kepler to Perform Parameter Studies in Subsurface Sciences Studies in Subsurface Sciences

Hybrid Multiscale Modeling Benchmark Problem Hybrid Multiscale Modeling Benchmark Problem

App Contact: Karen Schuchardt, PI, PNNL SDM Contact: Terence Critchlow, PNNL

SLIDE 34

Arie Shoshani

Workflow for parameter studies Workflow for parameter studies Workflow for parameter studies Workflow for parameter studies

Setup Stomp

User works within a “Study” where a Study can be represented as a graph

f processes and data inputs/outputs. Some processes are triggered by

the user, others appear as by-products of user actions.

Stomp.in parameters Launch Stomp1.in Job

utputs
utputs

Some Analysis graphics Launch parameters Some Analysis More data graphics more Analysis

1. Baseline computation

Setup Stomp branch

2. Vary permeability in material 2

…

Stomp1.in Stomp2.in Job Job

utputs
utputs
utputs
utputs

Stomp2.in Job

utputs
utputs
3. Vary other parameters…

SLIDE 35

Arie Shoshani

High Performance Technologies High Performance Technologies Usability and effectiveness Usability and effectiveness Enabling Data Understanding Enabling Data Understanding

More Results More Results

SLIDE 36

Arie Shoshani

Storage Resource Managers (SRMs): Storage Resource Managers (SRMs): Middleware for storage interoperability Middleware for storage interoperability and data movement and data movement

SLIDE 37

Arie Shoshani Tomcat servlet engine Tomcat servlet engine MCS Metadata Cataloguing Services MCS Metadata Cataloguing Services RLS Replica Location Services RLS Replica Location Services SOAP RMI MyProxy server MyProxy server MCS client RLS client MyProxy client GRAM gatekeeper GRAM gatekeeper

CAS Community Authorization Services CAS Community Authorization Services

CAS client disk

MSS Mass Storage System HPSS High Performance Storage System

disk HPSS High Performance Storage System disk disk

DRM Storage Resource Management DRM Storage Resource Management

HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management HRM Storage Resource Management gridFTP gridFTP gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server gridFTP server

penDAPg

server

penDAPg

server gridFTP Striped server gridFTP Striped server

LBNL LLNL ISI NCAR ORNL ANL

DRM Storage Resource Management DRM Storage Resource Management

SRM use in Earth Science Grid SRM use in Earth Science Grid

3100 users 120 TBs

SDM Contact: A. Sim, A. Shoshani, LBNL

SLIDE 38

Arie Shoshani

Create identical Directory and issue SRM-COPY (thousands of files) SRM-GET (one file at a time) GridFTP GET (pull mode)

stage files archive files Network transfer

Get list

f files

From directory Recovers from file transfer failures

Anywhere

Disk Cache

DataMover SRM

(performs writes)

LBNL/ ORNL

Disk Cache

SRM

(performs reads)

BNL

Recovers from staging failures Recovers from archiving failures

SRM as DataMover: Performs “rcp –r directory” SRM as DataMover: Performs “rcp –r directory”

n the WAN
n the WAN

50X reduction in the error rates, from 1% to 0.02% in the STAR project

SLIDE 39

Arie Shoshani

Capturing Provenance in Workflow Capturing Provenance in Workflow Framework Framework

Provenance, Tracking & Meta-Data (DBs and Portals) Control Plane (light data flows) Execution Plane (“Heavy Lifting” Computations and data flows)

Kepler

Process provenance
the steps performed in the workflow,

the progress through the workflow control flow, etc.

Data provenance
history and lineage of each data item

associated with the actual simulation (inputs, outputs, intermediate states, etc.)

Workflow provenance
history of the workflow evolution and

structure

System provenance
Machine and environment information
compilation history of the codes
information about the libraries
source code
run-time environment settings

SDM Contact: Mladen Vouk, NCSU

SLIDE 40

Arie Shoshani

FIESTA: Framework for Integrated End-to-end SDM Technologies and Applications

Supercomputers + Analytics Nodes Kepler Dashboard Storage Orchestration Auth Data Store Rec API Disp API

Management API

Access Trust

Provenance is capture Provenance is captured i d in a data store a data store and used by dashboard and used by dashboard

SLIDE 41

Arie Shoshani

Dashboard uses provenance for finding location Dashboard uses provenance for finding location

f files and automatic download with SRM
f files and automatic download with SRM

Download window

SDM Contact: Scott Klasky, ORNL

SLIDE 42

Arie Shoshani

Dashboard is used for job launching Dashboard is used for job launching and real-time machine monitoring and real-time machine monitoring

Allow for secure

logins with OTP.

Allow for job

submission.

Allow for killing

jobs.

Search old jobs.
See collaborators

jobs.

SLIDE 43

Arie Shoshani

Adaptable I/O system (ADIOS) Adaptable I/O system (ADIOS)

Allows plug-ins for different I/O implementations.

Allows plug-ins for different I/O implementations.

Abstracts the API from the method used f

Abstracts the API from the method used for I/O. r I/O.

Simple API, almost as easy as F90 write statement.

Simple API, almost as easy as F90 write statement.

Best practices/optimize IO rout

Best practices/optimize IO routines for all supported transports ines for all supported transports “for free” “for free”

Componentization.

Componentization.

Thin API

Thin API

XML fi

XML file le

data groupings with annotation

data groupings with annotation

IO method selection

IO method selection

buffer si

buffer sizes zes

Common tools

Common tools

Buffering

Buffering

Scheduling

Scheduling

Pluggable IO routines

Pluggable IO routines

Main advantages for users

Main advantages for users

No need

need t to cha change code code whe when running running on va

n various p

rious plat atforms rms

Change onl

Change only external X y external XML file L file

Asy

Asynchronous chronous I I/O s O saves ves c comp mput uting ing cyc cycles es

External Metadata (XML file)

Scientific Codes

ADIOS API MPI-CIO LIVE/DataTap MPI-IO POSIX IO pHDF-5 pnetCDF Viz Engines Others (plug-in) bufferin g schedul e feedback

SDM Contact: Scott Klasky, ORNL

SLIDE 44

Arie Shoshani

ADIOS Overview ADIOS Overview

ADIOS is an IO componentization, which allows us to

ADIOS is an IO componentization, which allows us to

Abstract the API from the IO implementation

Abstract the API from the IO implementation

Switch from synchronous to

Switch from synchronous to asynchronous IO at runtime asynchronous IO at runtime

provide

provide fast IO at runtim fast IO at runtime

Combines

Combines

Fast

Fast I/O routines I/O routines

Easy

Easy to use to use

Scalable

Scalable architecture architecture (1000s cores) millions of p (1000s cores) millions of processes

cesses
QoS

QoS

Metadata rich output

Metadata rich output

Visualization applied during simulations

Visualization applied during simulations

Analysis, compression techniques applied during

Analysis, compression techniques applied during simulations simulations

Provenance tracking

Provenance tracking

SLIDE 45

Arie Shoshani

Initial ADIOS performance. Initial ADIOS performance.

June 7, 2008: 24 hour GTC run on Jaguar at ORNL

June 7, 2008: 24 hour GTC run on Jaguar at ORNL

93% of machine

93% of machine (28,672 cores) (28,672 cores)

MPI-OpenMP mixed model on qu

MPI-OpenMP mixed model on quad-core nodes (7168 MPI procs) ad-core nodes (7168 MPI procs)

three interruptions tota

three interruptions total (simple no l (simple node f failure) with ilure) with 2 10+ 2 10+ hou hour ru r runs

Wrote 65 TB of data at >20 GB

Wrote 65 TB of data at >20 GB/sec (25 TB for post analysis) /sec (25 TB for post analysis)

IO overhead ~3% of wall clock time.

IO overhead ~3% of wall clock time.

Mixe

xed I d IO methods of sy methods of synchr nchronous MP

nous MPI-IO an

and PO d POSIX SIX IO IO co conf nfigured in th igured in the XML e XML fi file le

DA

DART: <2% overhead for RT: <2% overhead for writing 2 TB/hour with writing 2 TB/hour with XGC XGC code. code.

DataTap vs. Posix

DataTap vs. Posix

– 1 file per process (Posix). 1 file per process (Posix). – 5 secs for GTC computation. 5 secs for GTC computation. – ~25 seconds for Posix IO ~25 seconds for Posix IO – ~4 seconds with DataTap ~4 seconds with DataTap

SLIDE 46

Arie Shoshani

Extendable Arrays Extendable Arrays

Dens

Dense arrays that grow dynamicall e arrays that grow dynamically by extent of dimensions, or y by extent of dimensions, or number of dimensions need to be number of dimensions need to be restructured. How can that b

restructured. How can that be

avoided? avoided?

Example

Example

A 2-D array initiall

A 2-D array initially dened as A[3][3] and y dened as A[3][3] and then extended by 2 then extended by 2 columns, then by 1 columns, then by 1 row, w, f follo llowed b by 1 1 co colu lumn mn and so and so o

n.
Dev

Develop loped libraries d libraries

Inserting blocks

Inserting blocks

Reading any array

Reading any array sub-st sub-structure ructure

Sp

Sparse arrays arse arrays

Developed new method

Developed new method for HDF5 for HDF5

Balan

lanced Exten ced Extendible ible Hashing Hashing

SDM Contact: Ekow Otoo, LBNL

SLIDE 47

Arie Shoshani

High Performance Technologies High Performance Technologies Usability and effectiveness Usability and effectiveness Enabling Data Understanding Enabling Data Understanding

Results Results

SLIDE 48

Arie Shoshani

Scientific data understanding: Scientific data understanding: from Terabytes to a Megabytes from Terabytes to a Megabytes

Goal: solving the problem of d

Goal: solving the problem of data ov ta overload erload

Use sc

Use scientif ientific ic data mining techniques data mining techniques to analyze data from various SciDAC to analyze data from various SciDAC applications applications

Techniques borrowed from

Techniques borrowed from image and video proce image and video processing, machine learning, ssing, machine learning, statistics, pattern recognition, … statistics, pattern recognition, … Raw Data Target Data Preprocessed Data Transformed Data Patterns Knowledge Data Preprocessing Pattern Recognition Data Fusion Sampling Multi-resolution analysis De-noising Object - identification Feature- extraction Normalization Dimension- reduction Classification Clustering Regression Interpreting Results Visualization Validation

An iterative and interactive process

SLIDE 49

Arie Shoshani

Sapphire: scientific data mining Sapphire: scientific data mining

research

research in robu in robust, accurate, s st, accurate, scalable algorith calable algorithms

modular, extensible

modular, extensible software software

an

analysi alysis of data fro

f data from practical pro

practical problem lems

Leve

Leverage rage funding t funding through D rough DOE NNSA, E NNSA, LLNL LD LLNL LDRD, RD, GSEP Sc GSEP SciD iDAC AC project, and SDM SciDAC Center project, and SDM SciDAC Center

2006

De-noise data Background- subtraction Identify objects Extract features Sample data Fuse data Multi-resolution- analysis Data items Features

RDB: Data Store

Normalization Dimension- reduction Decision trees Neural Networks SVMs k-nearest neighbors Clustering Evolutionary algorithms Tracking …. FITS BSQ PNM View . . . Display Patterns Sapphire Software Public Domain Software Sapphire & Domain Software User Input & feedback Components linked by Python

SDM Contact: Chandrika Kamath, LLNL

SLIDE 50

Arie Shoshani

Separating signals in climate data Separating signals in climate data

We used independent componen

We used independent component analysis to separate El t analysis to separate El Niño and volcano signals in climate simulations Niño and volcano signals in climate simulations

Showed that the technique can

Showed that the technique can be used to enable be be used to enable better tter comparisons of simulations comparisons of simulations

Collaboration with Ben Santer (LLNL)

SLIDE 51

Arie Shoshani

Joint work with PPPL

Joint work with PPPL (Klasky, Pomphrey, (Klasky, Pomphrey, Monticello) Monticello)

Classify each of the

assify each of the no nodes: qu des: quasiperiodic, asiperiodic, is islands, separatrix lands, separatrix

Connect

nnections betwee ions between the nodes n the nodes

Want accurate and ro

Want accurate and robu bust classificati st classification

n,

, va valid when few points in e d when few points in each node node

Classification of puncture (Poincaré) plots for Classification of puncture (Poincaré) plots for NCSX NCSX

1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4 1.1 1.2 1.3 1.4 1.5 1.6 1.7 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25

National Compact Stellarator Experiment Quasiperiodic Islands Separatrix

1.2 1.3 1.4 1.5 1.6 1.7 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2

Collaboration with J. Breslau, N. Pomphrey, D. Monticello(PPPL), S. Klasky(ORNL)

SLIDE 52

Arie Shoshani

1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 −0.4 −0.3 −0.2 −0.1 0.1 0.2 0.3 0.4

Polar Coordinates Polar Coordinates

Transform the (x,y) data to Polar coordinates (r,

Transform the (x,y) data to Polar coordinates (r,θ). ).

Adv

Advantag ntages of polar coordinates: es of polar coordinates:

Radial exaggeration reveals some featur

Radial exaggeration reveals some features that are hard to see otherwise. es that are hard to see otherwise.

Automatically restricts analys

Automatically restricts analysis is to radia to radial ba band w nd with data, i th data, ignor noring i ng inside de and and

uts
utside.

de.

Easy to handle

Easy to handle r rotationa tational inva invariance. ance.

−3 −2 −1 1 2 3 0.32 0.33 0.34 0.35 0.36 0.37 0.38 0.39 0.4 0.41 0.5 1 1.5 2 2.5 −0.8 −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 −3 −2 −1 1 2 3 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 1.05 1.35 1.4 1.45 1.5 1.55 −0.08 −0.06 −0.04 −0.02 0.02 0.04 0.06 0.08 −2.5 −2 −1.5 −1 −0.5 0.5 1 1.5 2 2.5 0.078 0.08 0.082 0.084 0.086 0.088 0.09 0.092 0.094 0.096 1.1 1.2 1.3 1.4 1.5 1.6 1.7 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 0.25 −3 −2 −1 1 2 3 0.225 0.23 0.235 0.24 0.245 0.25 0.255 0.26 0.265 0.27 0.275 1.2 1.3 1.4 1.5 1.6 1.7 −0.25 −0.2 −0.15 −0.1 −0.05 0.05 0.1 0.15 0.2 −3 −2 −1 1 2 3 0.285 0.29 0.295 0.3 0.305 0.31 0.315 0.32 0.325 0.33

SLIDE 53

Arie Shoshani

Piecewise Polynomial Fitting: Computing Piecewise Polynomial Fitting: Computing polynomials polynomials

In each interval, compute the poly

In each interval, compute the polynomial coeff nomial coefficients cients to fit to fit 1 polynomial to the data. 1 polynomial to the data.

If the error is high, split the data into an upper and lower

If the error is high, split the data into an upper and lower

group. Fit 2 polynomials to
group. Fit 2 polynomials to the data, one to each group.

the data, one to each group.

−3 −2 −1 1 2 3 0.285 0.29 0.295 0.3 0.305 0.31 0.315 0.32 0.325 0.33 Node 32 −3 −2 −1 1 2 3 0.285 0.29 0.295 0.3 0.305 0.31 0.315 0.32 0.325 0.33 Node 32 −3 −2 −1 1 2 3 0.32 0.33 0.34 0.35 0.36 0.37 0.38 0.39 0.4 0.41 Node 42 −3 −2 −1 1 2 3 0.32 0.33 0.34 0.35 0.36 0.37 0.38 0.39 0.4 0.41 Node 42

Blue: data. Red: polynomials. Black: interval boundaries.

SLIDE 54

Arie Shoshani

Classification Classification

The number of polynomials n

The number of polynomials needed to fit the data and the eeded to fit the data and the number of gaps gives the info number of gaps gives the information needed to classify rmation needed to classify the node: the node:

Number of polynomials Number of polynomials Gaps Gaps

ne
ne

two two Zer Zero Quasip Quasiperiodic eriodic Separatrix Separatrix > Zero > Zero Islands Islands

2 Polynomials 2 Gaps Islands 2 Polynomials 0 Gaps Separatrix

−3 −2 −1 1 2 3 0.285 0.29 0.295 0.3 0.305 0.31 0.315 0.32 0.325 0.33 Node 32 −3 −2 −1 1 2 3 0.32 0.33 0.34 0.35 0.36 0.37 0.38 0.39 0.4 0.41 Node 42

SLIDE 55

Arie Shoshani

How do we extract representative How do we extract representative features for an orbit? features for an orbit?

Variation in the data makes it diff

Variation in the data makes it difficult to identify good icult to identify good features and extract them in a robust way features and extract them in a robust way

Issues with labels assigned to orbi

Issues with labels assigned to orbits ts

Next steps: characterizing is

Next steps: characterizing island chains and separatrix land chains and separatrix

rbits
rbits

Identifying missing orbits

SLIDE 56

Arie Shoshani

Understand the turbulence which causes Understand the turbulence which causes leakage of the fusion plasma leakage of the fusion plasma

Req

Requirements for fusion – irements for fusion – high temperature and high temperature and confined plasma confined plasma

Fine-scale turbulence at the

Fine-scale turbulence at the edge causes leakage of edge causes leakage of plasm plasma from the center to the from the center to the edge edge

Loss of confinement

Loss of confinement

Heat loss of plasma

Heat loss of plasma

Eros

Erosion or n or vapor vaporizat zation of

n of the

the containment wall containment wall

SLIDE 57

Arie Shoshani

Tracking blobs in fusion plasma Tracking blobs in fusion plasma

Using image and video processi

Using image and video processing techniques to identify ng techniques to identify and track blobs in experimental and track blobs in experimental data from NSTX to validate data from NSTX to validate and refine theories of and refine theories of edge turbulence edge turbulence

t t+1 t+2 Denoised

riginal

After removal

f background

Detection

f blobs

Collaboration with S. Zweben, R. Maqueda, and D. Stotler (PPPL)

SLIDE 58

Arie Shoshani

Example frames to segment (sequence Example frames to segment (sequence 113734: frames 1-50) 113734: frames 1-50)

SLIDE 59

Arie Shoshani

We are investigating several image We are investigating several image segmentation methods segmentation methods

Techniques tried:

chniques tried:

Immers

mersion-Based: n-Based: bas basic i c imme mmers rsion, n, const constraine ained watershed, d watershed, watershed watershed merging merging

Region Growing: seeded region

Region Growing: seeded region growing, seed competition growing, seed competition

Model-Based: 2-D Gaussian fit

Model-Based: 2-D Gaussian fit

Challenges

Challenges

how do we select the para

how do we select the parameters in meters in an algo an algorith rithm, m,

how d

do we h we handle th ndle the variability in th e variability in the data especially fo e data especially for longer sequences, r longer sequences,

how do the choices of al

how do the choices of algorithms and parameters gorithms and parameters influence the “science”, … influence the “science”, …

Why is this difficult?

Why is this difficult?

coherent structures are poorly unders

coherent structures are poorly understood empiri tood empirically and not understood cally and not understood theoretically theoretically

no know

no known ground-tr n ground-truth th

noisy imag

isy images

variation within a sequence

variation within a sequence

Work in progress

SLIDE 60

Arie Shoshani

Data I ntensive Data I ntensive Statistical Com puting Statistical Com puting

Parallel R (pR) Technology Parallel R (pR) Technology for Data Intensive Statistical Computing for Data Intensive Statistical Computing

Technical computing
Matrix and vector

formulations

Data Visualization and

analysis platform

Image processing,

vector computing Statistical com puting and graphics

http://www.r-project.org

Developed by R. Gentleman & R. Ihaka
Expanded by community as open source
Extensible via dynamically loadable libs

Contact: Nagiza Samatova, ORNL (samatovan@ornl.gov)

SLIDE 61

Arie Shoshani

Task and Data Parallelism in Task and Data Parallelism in pR pR

Likelihood Maximization Re-sampling schemes: Bootstrap, Jackknife Markov Chain Monte Carlo (MCMC) Animations

Task-parallel analyses:

k-means clustering Principal Component Analysis Hierarchical clustering Distance matrix, histogram, etc.

Data-parallel analyses: Goal: Parallel R ( pR) aim s: ( 1 ) to autom atically detect and execute task-parallel analyses; ( 2 ) to easily plug-in data-parallel MPI -based C/ Fortran codes ( 3 ) to retain high-level of interactivity, productivity and abstraction

Task & Data Parallelism in pR Task & Data Parallelism in pR

Task Parallelism Data Parallelism

SLIDE 62

Arie Shoshani

ProRata use in OBER Projects ProRata use in OBER Projects

DOE OBER Projects Using ProRata:

J. of Proteom e Research
Vol. 5, No. 11, 2006

>1,0 0 0

downloads

Jill Banfield, Bob Hettich: Acid Mine Drainage
Michelle Buchanan: CMCS Center
Steve Brow n, Jonathan Mielenz: BESC BioEnergy
Carol Harw ood, Bob Hettich: MCP R. palustris

SLIDE 63

Arie Shoshani

Dashboard Interface to pR Dashboard Interface to pR

Scott Klasky Roselyne Nobert Generated by pR

SLIDE 64

Arie Shoshani

SDM center collaboration SDM center collaboration with applications with applications

currently in progress problem identified interest expressed

Application Domains Workflow Technology (Kepler) Metadata And provenance Data Movement and storage Indexing (FastBit) Parallel I/O (pNetCDF, etc.) Parallel Statistics (pR, …) Feature extraction Active Storage

Clim limate M e Modeling ling ( (Dra rake)

workfl workflow

w

pNetCDF CDF pM pMatla lab

Astr trop

physi

hysics ( (Blo lond ndin)

da data m move vement da dashboard

Comb mbust ustion (

n (Jackie Chen)

da data m move vement dist stribut buted a d analysis Data DataMover ver-Li Lite te flam ame f e fron

nt

Global al A Access ccess pM pMatla lab tranient ev nt even ents ts

Comb mbust ustion (

n (Bell

ell)

Data DataMover ver-Li Lite te

Fu Fusi sion ( (PPPL PPPL)

poincare care p plot

ts

Fu Fusi sion (CPES) (CPES)

data-mov

ve, cod

code-coup

uple

Das Dashboard ard Data DataMover ver-Li Lite te Toroi roidal al m meshes es pR pR Blob track tracking

Materials - s - QBOX BOX (Ga (Galli) li)

XML XML

Hi High gh Ene Energy gy Ph Physics

La Lattice ice-QCD QCD SRM, SRM, Data DataMover ver even event fi finding

Grou

undwate

water Mo r Modeling

ident entified ed 4- 4-5 w 5 work rkflows

Accelar Accelarato ator S Scie ience (Ry nce (Ryne) e)

MPIO MPIO-SRM

SN SNS

workfl workflow

w

Data Data Entry try t tool

l (DEB

(DEB)

Bi Biol

logy

ScalaB ScalaBlast last ProR ProRata ScalaB ScalaBlast last

Clim limate e Clo Cloud mod modeli ling ng (R (Randall ll)

pNetCDF CDF cloud mod

ud modeling

ng

Data-to-Model Cov l Coversion (

n (Kot
tam

amathi) hi) Biolog

gy (H2

y (H2) Fusi sion (

n (RF)

F) (B (Bachelor

r)

poincare care p plot

ts

Su Subs bsurface M face Modeling (Licht chtner)

Ove Over A AMR

Flow w

w with

th str strong ng shoc shocks ( (Lele)

cond

ndit

ition ional s l stat atistics

Fu Fusion (ex (exten ended M d MHD) D) (Ja (Jardi din) n) Nanoscience (Rack) cience (Rack)

pM pMatla lab

t
the

her acti r activitie ies

integrate w e with th Lust Luster

SLIDE 65

Arie Shoshani

SDM center collaboration SDM center collaboration with other centers/institutes with other centers/institutes

currently in progress problem identified interest expressed

Centers & institutions Workflow Technology (Kepler) Metadata And provenance Data Movement and storage Indexing (FastBit) Parallel I/O (pNetCDF, etc.) Parallel Statistics (pR, …) Feature extraction Active Storage Open Science Grid

SRM-te SRM-tester ster

Earth System Grid

SRM a SRM and D d DML

Petascale Storage Institute

Posix-I six-IO

Vis Institute (Ma)

quer ery-ba y-base sed d vi vis put pa paralle rallel I/ l I/O in O in Vi Vis pR pR

Vis Center (Bethel)

work workflow

w in

in vi vis quer ery-ba y-base sed d vi vis pR pR

SLIDE 66

Arie Shoshani

Summary Remarks (1)

SDM center has developed data management tools that

provide

High performance
now at petascale, planning for exascale
across the I/O software stack
Specialized Indexing technologies
Parallel analysis tools
Usability and effectiveness
Developed FIESTA: a Framework for Integrated

End-to-end SDM Technologies and Applications

Based on workflow and dashboard technologies
Provide real-time monitoring, repeated analysis, code coupling
Future: pre-production, post production (analysis) workflows
Integrate I/O efficient tools via common API
Future: Allow analysis pipeline where data is
Simple to use data movement tools
Enabling data understanding
Framework for use of multiple techniques – analysis pipeline
Parallel statistics tools, specialized for several application domains
Use if indexing in query-based visualization

SLIDE 67

Arie Shoshani

Summary Remarks (2)

SDM center spends much effort on
Sustainability and usability
Working with vendors on I/O and file systems– Cray, IBM
Working with data centers – ANL, ORNL, NERSC
Packaging and releasing open source products – PVFS, ROMIO, pNetCDF,

FastBit, pR, Kepler, …

SDM center developed or enhanced many products that are in use today
Current SDM projects also looking to next generation of systems and

applications - active storage, pNFS, architectures, I/O forwarding and aggregation, asynchronous I/O, parallel analysis tools, extendable arrays, …

Establishing contacts with scientists
Successfully collaborated with scientist from various disciplines: Fusion,

Combustion, Astrophysics, groundwater, biology, climate, material science, …

Collaboration with other centers/institutes: Vacet (query-based Vis), PDSI (APIs

for generic file systems), IUSV (efficient I/O for vis), ESG (SRM), OSG (SRM), CEDPS (SRM log analysis), PERI (through Dashboard).

Holding tutorials at SC and other conferences: PVFS, ROMIO, pNetCDF, Kepler,

Sapphire, …

Educating students at: UCD, NCSU, NWU, Utah; postdocs at LBNL, ORNL, PNNL
Future
Focus on scaling, robustness, ease of use
Engaging additional scientists and applications, based on current successes
Identify problems based on above activities, and perform needed research

SLIDE 68

Arie Shoshani

The END The END

SLIDE 69

Arie Shoshani

Extra slides Extra slides

SLIDE 70

Arie Shoshani

Scientific Workflow Requirements Scientific Workflow Requirements

Unique requirements of scientific WFs
Moving large volumes between modules
Tightlly-coupled effic

Tightlly-coupled efficient data movement ent data movement

Specification of granularity-based iteration
e.g. In spa

e.g. In spatio-tem io-temporal simulati poral simulations

ns –

a time step is a a time step is a “granule granule”

Support for data transformation
complex data types (including file formats, e.g. netCDF, HDF)

complex data types (including file formats, e.g. netCDF, HDF)

Dynamic steering of workflow by user
Dynamic user examination of results

Dynamic user examination of results

Developed a working scientific work flow system
Automatic microarray analysis
Using web-wrapping tools developed by the center
Using Kepler WF engine
Kepler is an adaptation of the UC Berkeley tool, Ptolemy