Open Grid Service Architecture - Data Access & Integration - - PowerPoint PPT Presentation

▶
open grid service architecture data access integration
SMART_READER_LITE
LIVE PREVIEW

Open Grid Service Architecture - Data Access & Integration - - PowerPoint PPT Presentation

Sep 21, 2022 506 likes •742 views

Open Grid Service Architecture - Data Access & Integration (OGSA-DAI) Dave Pearson OGSA-DAI Programme Manager dave.pearson@oracle.com Nov 2003 UK French Grid Workshop OGSA-DAI Programme Nov 2003 UK French Grid Workshop OGSA-DAI

slide-1
SLIDE 1

Nov 2003 UK French Grid Workshop

Open Grid Service Architecture - Data Access & Integration (OGSA-DAI)

Dave Pearson OGSA-DAI Programme Manager

dave.pearson@oracle.com

slide-2
SLIDE 2

Nov 2003 UK French Grid Workshop

OGSA-DAI Programme

slide-3
SLIDE 3

Nov 2003 UK French Grid Workshop

OGSA-DAI Motivation

  • UK e-Science Core Programme

– Database Taskforce – Grid data requirements scoping study – Data Access and Integration BoF at GGF4 – DAIS Working Group in Global Grid Forum

  • Exploit emerging Grid computing concepts (OGSA)
  • Web Services adoption
  • DBMS adoption
  • Meet needs of Grid application developers for UK e-Science

community

– Grid Pilot projects

slide-4
SLIDE 4

Nov 2003 UK French Grid Workshop

UK Grid Pilot Projects

Computational Fluid Dynamics

Geodise

Bioinformatics, Environmental & Earth sciences

DiscoveryNet

Virtual Observatory

AstroGrid

Interactive steering of data & compute intensive analysis – chemistry, earth sciences

RealityGrid

Combinatorial (bio-)chemistry

CombiChem

Distributed Aircraft Maintenance Engineering

DAME

eScience workbench, Bioinformatics

myGrid

Research Focus

slide-5
SLIDE 5

Nov 2003 UK French Grid Workshop

OGSA-DAI Requirements

  • Transient and persistent data
  • Data sources & Resources
  • Maintain complex data models
  • Information/Knowledge
  • Record and maintain ‘data about data’
  • Metadata
  • Establish reliability and quality of data
  • Provenance
  • Make data more accessible
  • Publishing and Discovery
  • Restrict who can read and modify data
  • Access control
  • Receive data in a state ready to use
  • Transformation
  • Personalise data
  • Analysis/Interpretation
  • Quality of Service
  • Managing data
slide-6
SLIDE 6

Nov 2003 UK French Grid Workshop

DAI Programme of Work

  • Two phases

– Phase 1 Feb – Sept 2002 complete – Phase 2 Oct 2002 – Jul 2003 – Maintenance activities

  • Funding for next phase approved - DAIT
  • Grid data service reference implementations

– OGSA compliant – DB2, MySQL, ORACLE, XINDICE

  • Integration with Globus toolkit
  • Input to Global Grid Forum on standards

recommendations

  • Collaborative Programme
slide-7
SLIDE 7

Nov 2003 UK French Grid Workshop

Participating Organisations

Edinburgh Parallel Computing Centre North East eScience Centre National eScience Centre North West eScience Centre IBM UK & IBM US Oracle Corporation UK

slide-8
SLIDE 8

Nov 2003 UK French Grid Workshop

Open Grid Services Architecture

  • Web services

– Standard interface definition mechanisms: multiple protocol bindings, multiple implementations, local/remote transparency – Address discovery & invocation of services

  • Grid Service extensions

– Transient services – State management in a distributed environment

  • Resource virtualisation
  • Multiple hosting targets: J2EE, .NET, “C”, …
slide-9
SLIDE 9

Nov 2003 UK French Grid Workshop

Data Access and Integration

slide-10
SLIDE 10

Nov 2003 UK French Grid Workshop

OGSA-DAI Positioning - Vision

OGSA-DAI Distributed Query OGSA-DAI Basic Services Data Grid Infrastructure – Location, Delivery, Replication… Resource Grid Infrastructure – OGSA… Database, Communication, OS… Technology

slide-11
SLIDE 11

Nov 2003 UK French Grid Workshop

App Specifics

Drivers Query (Create Retrieve Update Delete) Data Format

OGSA-DAI Basic Services OGSA-DAI Distributed Query

Delivery

Database, Communication, OS… Technology

Location Meta Data Notification OGSA Lifetime

GDS GDSR GDSF

OGSA-DAI Positioning - Today

slide-12
SLIDE 12

Nov 2003 UK French Grid Workshop

Request to Registry for sources of data about “x” Registry responds with Factory handle Request to Factory for access to database Factory creates GridDataService Factory returns handle of GDS to client Client queries GDS with SQL, XPath, XQuery etc Query results returned XML

SOAP/HTTP service creation API interactions

Registry GDSR Factory GDSF

OGSA-DAI Behaviours

Analyst

Database

(Xindice, MySQL Oracle, DB2)

Grid Data Service GDS Consumer

OR delivered to consumer as XML GDS interacts with database

slide-13
SLIDE 13

Nov 2003 UK French Grid Workshop

Delivery Patterns

A G Q S + R A G Q + D S R C A G Q + U S A G Q S U P

Retrieve Update/Insert Pipeline

A G1 = P Q1 + D S1 U/R A G Q S D C R A G Q + D S I P U I G2 = C S2 Q2 A G1 = P Q1 S1 U/R G2 = C S2 Q2 + D I

1. 2. 3. 4. 6. 5. 7. 8.

slide-14
SLIDE 14

Nov 2003 UK French Grid Workshop

Statement Operations

  • Receives statement
  • Performs statement against a data resource
  • Returns results

Statement Component and Database Connections Statement Results Data Resource

slide-15
SLIDE 15

Nov 2003 UK French Grid Workshop

Statement Document

<?xml version="1.0" encoding="UTF-8" ?> <GridDataServiceScript> <Header> … Name, versioning, configuration specification </Header> <Body> <Documentation>Update with data delivered with the script</Documentation> <DefineParameter name="table">myimages</DefineParameter> <DefineParameter name="id">id = 2</DefineParameter> <Statement name="xyz" dataResource="MyDataResource"> SELECT * FROM <UseParameter reference="table" /> WHERE <UseParameter reference="id" /> </Statement> <Delivery name="delivery1"> <Mechanism type="bulk" /> <Mode type="full" /> <From>xyz</From> <To>response</To> </Delivery> <Execute name="execute1">xyz</Execute> </Body> </GridDataServiceScript>

slide-16
SLIDE 16

Nov 2003 UK French Grid Workshop

Statement Architecture

Engine/GDS-S JDBC Connection XMLDB Connection JDBC API Relational DB XML DB Statement Statement Activity Statement Handlers JDBC Statement Handlers XMLDB Statement Handlers XML:DB API

slide-17
SLIDE 17

Nov 2003 UK French Grid Workshop

OGSA-DAI Security

  • Grid Security Infrastructure Authentication
  • The role mapper takes credentials and a database

name and returns a dbRole

  • dbRole contains username and password

username Credentials Role Mapper dbRole Database Name password

slide-18
SLIDE 18

Nov 2003 UK French Grid Workshop

OGSA-DAI Product

slide-19
SLIDE 19

Nov 2003 UK French Grid Workshop

OGSA-DAI Product

  • Current release 3.0.2
  • OGSA-DAI

– 1183 downloads as at 27th Oct 2003 – 461 downloads for R3 & R3.0.2 – >350 UK downloads

  • DQP prototype

– 77 downloads since 1st Sept 2003

  • Web site

– 471 registered users – Transfers to users gigabytes / month

  • Aug 1.38GB

Sep 1.63GB Oct 1.81GB

Binary, source and documentation available for download from:

www.ogsa-dai.org.uk

slide-20
SLIDE 20

Nov 2003 UK French Grid Workshop

DAIT Next Phase of Development

slide-21
SLIDE 21

Nov 2003 UK French Grid Workshop

OGSA-DAI road map 1

  • R3.1.0

Jan 04

– Tech. Preview part of R4

  • User Group: inaugural meeting

Q1 04

  • R4.0.0

April 04

– Performance & monitoring – Additional DBMS’s supported – Additional SQL supported – DBMS management operations

  • archive, restore, bulk load

– File access – Client libraries – Installation wizard – User support, courses, training material, performance report

slide-22
SLIDE 22

Nov 2003 UK French Grid Workshop

OGSA-DAI road map 2

  • R5 October 04

– Compliance with DAIS standards proposal – Distributed Relational Query Processing – Improved dependability and security integration – Extended & integrated XML and relational facilities – Distributed transaction participation – Coordinated OGSA-DAI contributor community

  • R6 April 05

– Integrated with GT3 – New facilities depend on user priorities, context and research – OGSA-DAI components from contributor community

  • R7 October 05

– Maintainable release for the user community