Realistic modelling of complex Realistic modelling of complex - - PowerPoint PPT Presentation

▶
realistic modelling of complex realistic modelling of
SMART_READER_LITE
LIVE PREVIEW

Realistic modelling of complex Realistic modelling of complex - - PowerPoint PPT Presentation

Dec 06, 2023 179 likes •510 views

Realistic modelling of complex Realistic modelling of complex Supercomputing, Visualization & e-Science problems on Grids problems on Grids Manchester Computing John Brooke (University of Manchester) Peter Coveney PI RealityGrid

slide-1
SLIDE 1

Manchester Computing

Supercomputing, Visualization & e-Science

Realistic modelling of complex Realistic modelling of complex problems on Grids problems on Grids

John Brooke (University of Manchester) Peter Coveney PI RealityGrid (University College London) Stephen Pickles (University of Manchester) Thanks also to the other RealityGrid co-Investigators John Darlington (Imperial College) Steve Kenny and Roy Kalawsky (Loughborough University) John Gurd (University of Manchester) Mike Cates (University of Edinburgh) Adrian Sutton (University of Oxford)

slide-2
SLIDE 2

http://www.realitygrid.org 2

The RealityGrid project

Mission: “Using Grid technology to closely couple high performance computing, high throughput experiment and visualization, RealityGrid will move the bottleneck out of the hardware and back into the human mind.” Scientific aims: to predict the realistic behavior of matter using diverse simulation methods (Lattice Boltzmann, Molecular Dynamics and Monte Carlo) spanning many time and length scales to discover new materials through integrated experiments.

slide-3
SLIDE 3

http://www.realitygrid.org 3

Partners

Industrial

  • Schlumberger
  • Edward Jenner Institute for

Vaccine Research

  • Silicon Graphics Inc
  • Computation for Science

Consortium

  • Advanced Visual Systems
  • Fujitsu

Academic

  • University College London
  • Queen Mary, University of London
  • Imperial College
  • University of Manchester
  • University of Edinburgh
  • University of Oxford
  • University of Loughborough
slide-4
SLIDE 4

http://www.realitygrid.org 4

Access Grid meetings for problem solving

slide-5
SLIDE 5

http://www.realitygrid.org 5

RealityGrid

User with laptop/PDA (web based portal) HPC resources Scalable MD, MC, mesoscale modelling “Instruments”: XMT devices, LUSI,… Visualization engines Steering ReG steering API Storage devices Grid infrastructure (Globus, Unicore,…) Performance control/monitoring VR and/or AG nodes

Moving the bottleneck out of the hardware and into the human min Moving the bottleneck out of the hardware and into the human mind… d…

slide-6
SLIDE 6

http://www.realitygrid.org 6

RealityGrid Characteristics

Grid-enabled (Globus, UNICORE) Component-based, service-oriented Steering is central

– Computational steering – On-line visualisation of large, complex datasets – Feedback-based performance control – Remote control of novel, grid-enabled, instruments (LUSI)

Advanced Human-Computer Interfaces (Loughborough) Everything is (or should be) distributed and collaborative High performance computing, visualization and networks All in a materials science domain

– multiple length scales, many "legacy" codes (Fortran90, C, C++, mostly parallel)

slide-7
SLIDE 7

http://www.realitygrid.org 7

Three dimensional Lattice-Boltzmann simulations

  • Code (LB3D) written in Fortran90 and

parallelized using MPI.

  • Scales linearly on all available resources.
  • Fully steerable.
  • Future plans include move to parallel data

format PHDF5.

  • Data produced during a single large scale

simulation can exceed hundreds of gigabytes or even terabytes.

  • Simulations require supercomputers
  • High end visualization hardware and parallel

rendering software (e.g. VTK) needed for data analysis.

3D datasets showing snapshots from a simulation of spinodal decomposition: A binary mixture of water and oil phase separates. ‘Blue’ areas denote high water densities and ‘red’ visualizes the interface between both fluids.

slide-8
SLIDE 8

http://www.realitygrid.org 8

Exploring parameter space through computational steering

Initial condition: Random water/ surfactant mixture. Self-assembly starts. Rewind and restart from checkpoint. Lamellar phase: surfactant bilayers between water layers. Cubic micellar phase, low surfactant density gradient. Cubic micellar phase, high surfactant density gradient.

slide-9
SLIDE 9

http://www.realitygrid.org 9

Computational Steering - Why?

Terascale simulations can generate in days data that takes months to understand Problem: to efficiently explore and understand the parameter spaces of materials science simulations Computational steering aims to short circuit post facto analysis

– Brute force parameter sweeps create a huge data-mining problem – Instead, we use computational steering to navigate to interesting regions of parameter space – Simultaneous on-line visualization develops and engages scientist's intuition – thus avoiding wasted cycles exploring barren regions, or even doing the wrong calculation

slide-10
SLIDE 10

http://www.realitygrid.org 10

Computational steering – how?

  • We instrument (add "knobs" and "dials" to) simulation codes through a

steering library

  • Library provides:

– Pause/resume – Checkpoint and windback – Set values of steerable parameters – Report values of monitored (read-only) parameters – Emit "samples" to remote systems for e.g. on-line visualization – Consume "samples" from remote systems for e.g. resetting boundary conditions

  • Images can be displayed at sites remote from visualization system,

using e.g. SGI OpenGL VizServer, or Chromium

  • Implemented in 5+ independent parallel simulation codes, F90, C, C++
slide-11
SLIDE 11

http://www.realitygrid.org 11

Philosophy

Provide right level of steering functionality to application developer Instrumentation of existing code for steering

– should be easy – should not bifurcate development tree

Hide details of implementation and supporting infrastructure

– eg. application should not be aware of whether communication with visualisation system is through filesystem, sockets or something else – permits multiple implementations – application source code is proof against evolution of implementation and infrastructure

slide-12
SLIDE 12

http://www.realitygrid.org 12

Steering and Visualization

Simulation Visualization Visualization Client

Steering library Steering library Steering library Display Display Display

data transfer

slide-13
SLIDE 13

http://www.realitygrid.org 13

Architecture

Communication modes:

  • Shared file system
  • Files moved by UNICORE daemon
  • GLOBUS-IO
  • SOAP over http/https

Simulation Visualization Visualization Client

Steering library Steering library Steering library Data mostly flows from simulation to visualization. Reverse direction is being exploited to integrate NAMD&VMD into RealityGrid framework.

data transfer

slide-14
SLIDE 14

http://www.realitygrid.org 14

Steering in the OGSA

Steering client Simulation Steering library Visualization Visualization Registry Steering GS Steering GS connect publish find bind publish bind Client

Steering library Steering library

Steering library

data transfer

slide-15
SLIDE 15

http://www.realitygrid.org 15

Steering in OGSA continued…

  • Each application has an associated OGSI-compliant “Steering Grid

Service” (SGS)

  • SGS provides public interface to application

– Use standard grid service technology to do steering – Easy to publish our protocol – Good for interoperability with other steering clients/portals – Future-proofed next step to move away from file-based steering or Modular Visualisation Environments with steering capabilities

  • SGSs used to bootstrap direct inter-component connections for large

data transfers

  • Early working prototype of OGSA Steering Grid Service exists

– Based on light-weight Perl hosting environment OGSI::Lite – Lets us use OGSI on a GT2 Grid such as UK e-Science Grid today

slide-16
SLIDE 16

http://www.realitygrid.org 16

Steering client

  • Built using C++ and Qt library – currently

have execs. for Linux and IRIX

  • Attaches to any steerable RealityGrid

application

  • Discovers what commands are supported
  • Discovers steerable & monitored

parameters

  • Constructs appropriate widgets on the fly
  • Web client (portal)

under development

slide-17
SLIDE 17

http://www.realitygrid.org 17

RealityGrid-L2: LB3D on the L2G

SGI OpenGL VizSe Visualization SGI Onyx Vtk + VizServer

program lbe use lbe_init_module use lbe_steer_module use lbe_invasion_module

Steering (XM File based communication via s filesystem: Steering GUI X output is tunnelled back using ssh. ReG steering GUI rver Laptop Vizserver Client Steering GUI GLOBUS used to launch jobs hared L) Simulation Data GLOBUS-IO Simulation LB3D with RealityGrid Steering API

slide-18
SLIDE 18

http://www.realitygrid.org 18

Performance Control

application

component 1 component 2 component 3 application performance steerer

component performance steerer component performance steerer component performance steerer

slide-19
SLIDE 19

http://www.realitygrid.org 19

Advance Reservation and Co-allocation: Summary of Requirements

  • Computational steering + remote, on-line visualization demand:

– co-allocation of HPC (processors) and visualization (graphics pipes and processors) resources – at times to suit the humans in the loop

  • advanced reservation
  • For medium to large datasets, Network QoS is important

– between simulation and visualization, – visualisation and display

  • Integration with Access Grid

– want to book rooms and operators too

  • Cannot assume that all resources are owned by same VO
  • Want programmable interfaces that we can rely on

– must be ubiquitous, standard, and robust

  • Reservations (agreements) should be re-negotiable
  • Hard to change attitudes of sysadmins and (some) vendors
slide-20
SLIDE 20

http://www.realitygrid.org 20

Reality Grid and the UK/USA TeraGrid Project

Linking US Extended Terascale Facilities and UK HPC resources via a Trans-Atlantic Grid

  • We plan to use these combined resources as the basis for an exciting

project

– to perform scientific research on a hitherto unprecedented scale

  • Computational steering, spawning, migrating of massive simulations for

study of defect dynamics in gyroid cubic mesophases

  • Visualisation output will be streamed to distributed collaborating sites via the

Access Grid

  • Workshop presentation with FZ Juelich and HLRS, Stuttgart on the theme of

computational steering.

  • At Supercomputing, Phoenix, USA, November 2003
slide-21
SLIDE 21

http://www.realitygrid.org 21

Challenges

RealityGrid will stretch performance envelope at many levels

– Computation: must scale to 100s of processors – Networks: projected need for 1 Gbps sustained – Visualization: must keep up with simulation.

Interoperability and integration Human presence and interaction

– Integration with the Access Grid – REALISTE proposal to FP6 “Solving Complex problems on Grids”

Advanced Reservation and Co-allocation are key

– Need better support from scheduling infrastructure – Hence RealityGrid's involvement in GRAAP-WG at GGF

slide-22
SLIDE 22

Manchester Computing

Supercomputing, Visualization & e-Science

Computational Computational Materials Science Materials Science

slide-23
SLIDE 23

http://www.realitygrid.org 23

Computational Materials Science

RealityGrid uses HPC for large-scale simulation work in various areas: Electronic structure studies of condensed matter & materials

– (clays, clay-polymer nanocomposites): plane wave DFT

Atomistic/molecular simulation: molecular dynamics

– NAMD, LAMMPS, Moldyn,…

Mesoscale simulation:

– lattice gas & lattice-Boltzmann (LB3D, LUDWIG, …) – dissipative particle dynamics

Multiscale/hybrid methods

slide-24
SLIDE 24

http://www.realitygrid.org 24

Bridging length and time scales

Macroscopic (irreversible)

Boltzmann equation Lattice- Boltzmann

Mesoscopic (irreversible) Microscopic (reversible) Computational/Continuum Fluid Dynamics Dissipative Particle Dynamics Lattice Gas Molecular Dynamics

slide-25
SLIDE 25

http://www.realitygrid.org 25

Lattice gas methods

3D Lattice Gas method: Binary immiscible phase separation

Beta=0.03, just below the spinodal point Beta=0.04

slide-26
SLIDE 26

http://www.realitygrid.org 26

Lattice gas methods

3D Lattice Gas method: Binary and ternary immiscible phase separation Invasion of a porous medium with residing fluid. Only oil and water [1]

Ternary system: two immiscible fluids plus surfactant. Only oil density shown. Shear Flow, lattice size=64^3, shear rate=0.25, reduced density=0.18 [2]

[1] Love P J, Maillet J-B, Coveney PV, Phys Rev E 64 61302 (2001); [2] Love P J and Coveney P V, Phil Trans R Soc London A360, 357(2002)

slide-27
SLIDE 27

http://www.realitygrid.org 27

Lattice Boltzmann methods

Lattice Boltzmann simulation movie of phase separation in an initially homogeneous mixture of two immiscible fluids. Experimentally this occurs when a fluid mixture is quenched below the spinodal point in its phase

  • diagram. Different length

scales are obtained, as has been seen experimentally

Chin J and Coveney PV, Physical Review E 66 016303 (2002)

slide-28
SLIDE 28

Manchester Computing

Supercomputing, Visualization & e-Science

Instrumentation Instrumentation

London University Search Instrument (LUSI) X-Ray Microtomography (XMT)

slide-29
SLIDE 29

http://www.realitygrid.org 29

London University Search Instrument

LUSI is located at and developed by Queen Mary College, University of London Aim: Find ceramics (e.g. rare earth metal oxides) with interesting / valuable properties (e.g. high temperature superconductivity) Motivation: theory cannot indicate how to construct a compound with a particular

  • property. Established methodology in pharmaceutical industry uses automated

sample generation and testing. Let's apply the same idea in materials science, exploring properties that are difficult to predict: superconductivity, luminescence, dielectric response…

Furnace XY Table Instruments Printer

slide-30
SLIDE 30

http://www.realitygrid.org 30

LUSI - schematic

Database New materials

c c

Robot

c c

Predictions Neural network Measured data

slide-31
SLIDE 31

http://www.realitygrid.org 31

XMT

  • X-Ray Microtomography in

Dentistry at QM, or using synchrotron X-ray source at ESRF

  • Produces large amounts of

data:

– Storage – Provenance

  • Visualisation

– Data sets are large – If done in real time we can get experimental steering

Rendered image of a 1.6 mm length of a microtomographic data set of a human vertebral body, about 40 mm in

  • diameter. Sample from Prof. Alan Boyde.

J.C. Elliott, G.R. Davis, P. Anderson, F.S.L. Wong, S.E.P. Dowker, C.E. Mercer. Anales de Química Int Ed 93, S77- S82, 1997.

slide-32
SLIDE 32

http://www.realitygrid.org 32

XMT

  • Simulation,

visualization and data gathering coupled via RealityGrid

  • Expensive synchrotron

beam time resources

  • ptimally used to obtain

sufficient resolution for simulation

  • Local testbed providing

grid enablement model for European synchrotron facility