Grid Enabled Neurosurgical Grid Enabled Neurosurgical Imaging Using - - PowerPoint PPT Presentation

Grid Enabled Neurosurgical Grid Enabled Neurosurgical Imaging Using Simulation g g g

http://wiki realitygrid org/wiki/GENIUS http://wiki.realitygrid.org/wiki/GENIUS

Introduction

The GENIUS project aims to model large scale patient specific cerebral blood flow in clinically relevant time frames specific cerebral blood flow in clinically relevant time frames Objectives: Objectives:

• To study cerebral blood flow using patient-specific image-based models.
• To provide insights into the cerebral blood flow & anomalies.

p g

• To develop tools and policies by means of which users can better exploit

the ability to reserve and co-reserve HPC resources.

• To develop interfaces which permit users to easily deploy and monitor

simulations across multiple computational resources.

• To visualize and steer the results of distributed simulations in real time
• To visualize and steer the results of distributed simulations in real time

H LB HemeLB

Efficient fluid solver for modelling brain bloodflow called HemeLB: HemeLB:

• Uses the lattice-Boltzmann method
• Efficient algorithms for sparse geometries

g p g

• Topology-aware graph growing partitioning technique
• Optimized inter and intra machine communications
• Optimized inter- and intra-machine communications
• Full checkpoint capabilities.

Getting the Patient Specific Data Getting the Patient Specific Data

Data is generated by MRA scanners at the National

trilinear

scanners at the National Hospital for Neurosurgery and Neurology

trilinear interpolation 5122 pixels x 100 slices, p res: 0.468752 mm x 0.8 mm 20482 682 bi l Our graphical-editing tool 20482 x 682 cubic voxels, res: 0.469 mm

Cross-site Runs with MPI-g

GENIUS has been designed to run across multiple machines using MPI-g

• Some problems won’t fit on a single machine, and require

the RAM/processors of multiple machines on the grid.

• MPI-g allows for jobs to be turned around faster by using

small numbers of processors on several machines - essential for clinician

• HemeLB performs well on cross site runs, and makes use
• f overlapping communication in MPI-g

H LB/MPI R i HemeLB/MPI-g Requires Co-Allocation Co-Allocation

• We can reserve multiple resources for

We can reserve multiple resources for specified time periods C ll ti i f l f t ti

• Co-allocation is useful for meta-computing

jobs like HemeLB, viz and for workflow applications.

• We use HARC - Highly Available Robust

We use HARC Highly Available Robust Co-scheduler (developed by Jon Maclaren at LSU) at LSU).

Slide courtesy Jon Maclaren

HARC HARC

• HARC provides a secure co-allocation service

C p o des a secu e co a ocat o se ce

– Multiple Acceptors are used – Works well provided a majority of Acceptors stay alive – Paxos Commit keeps everything in sync – Gives the (distributed) service high availability Deployment of 7 acceptors > Mean Time To Failure years – Deployment of 7 acceptors --> Mean Time To Failure ~ years – Transport-level security using X.509 certificates

• HARC is a good platform on which to build portals/other

services

– XML over HTTPS - simpler฀than SOAP services – Easy to interoperate with – Very easy to use with the Java Client API

Real Time Visualisation and Steering Real Time Visualisation and Steering

A t l t H LB k th t t b

• A way to let HemeLB know the parameters to be

steered --> we use the RealityGrid steering system to steer the input data on the fly system to steer the input data on the fly. O i i t d ll thi f di t ib t d (

• One aim is to do all this for distributed (cross-

site) simulations

• For medical applications, need may be urgent

Application Hosting Environment

• Need to utilize resources from globally distributed

Need to utilize resources from globally distributed grids

– Administratively distinct Administratively distinct – Running different middleware stacks

• Wrestling with middleware can't be a limiting step
• Wrestling with middleware can t be a limiting step

for scientists Need tools to hide comple it of nderl ing grids

• Need tools to hide complexity of underlying grids

PDA/Cellphone Visualisation and Steering PDA/Cellphone Visualisation and Steering

• RealityGrid: a tool for modelling and

y g simulating very large or complex condensed matter structures…

• Human Factors issues key
• Human Factors issues key
• Large-scale applications, scheduling

when resources become available, ‘around the clock’ computation…

• AHE application launching built in to

client

• User

interaction/workflo w: not necessarily c e t y 9:00am – 5:00pm, from the ‘desk’ environment

AHE HARC integration AHE - HARC integration

U f th

• Users can co-reserve resources from the

AHE GUI client using HARC

• When submitting a job, users can either run

their jobs in normal queues, or use one of j q , their reservations

• AHE passes reservation through to the
• AHE passes reservation through to the

Globus GRAM AHE client ses HARC Ja a client API to

• AHE client uses HARC Java client API to

manage reservations

AHE R G & WS GRAM I t ti AHE ReG & WS GRAM Integration

• For steerable applications, AHE server starts up a

ReG Steering Web Service (SWS) for the li ti & t th d i t f i th application, & sets the end point reference in the job’s environment

• AHE can submit jobs to GridSAM (described in

JSDL) WS GRAM (d ib d i JDD) b JSDL) or WS GRAM (described in JDD) by applying a XSL Transform to the job specific WS- ResourceProperties document ResourceProperties document AHE > WS GRAM l h i l lti it

• AHE --> WS GRAM can launch single or multisite

MPIg jobs

Conclusions

• We have developed computationally efficient tools to

i l t i l t d i li ti t ifi manipulate, simulate and visualize patient specific vascular systems Goal is to better comprehend blood flow behavior of

• Goal is to better comprehend blood flow behavior of

normal and anomalous cerebral systems and provide patient specific models for clinical guidance in surgical patient specific models for clinical guidance in surgical

• perations
• Application Hosting Environment extended to launch

pp g MPI-g cross site runs, integrate ReG steering and visualisation, as well as making HARC cross site ti reservations