Service-Oriented Science: Scaling eScience Impact Ian Foster - - PowerPoint PPT Presentation
Service-Oriented Science: Scaling eScience Impact Ian Foster Computation Institute Argonne National Lab & University of Chicago Acknowledgements Carl Kesselman, with whom I developed many ideas (& slides) Bill Allcock, Charlie
2
Carl Kesselman, with whom I developed
Bill Allcock, Charlie Catlett, Kate Keahey,
Ann Chervenak, Ewa Deelman, Laura
Karl Czajkowski, Steve Tuecke @ Univa Numerous other fine colleagues in NESC,
NSF & DOE for research support
3
4
System-level problems require integration
Of expertise Of data sources (“data deluge”) Of component models Of experimental modalities Of computing systems
Internet enables decom position
“When the network is as fast as the
5
Decom pose across network
Select & compose services Select “best of breed” providers Publish result as new services
Decouple resource & service providers
Data Archives Analysis tools Discovery tools Users
Fig: S. G. Djorgovski
6
1.
2.
3.
“The Anatomy of the Grid”, Foster, Kesselman, Tuecke, 2001
Experimental Model
Computational Model
COORD. COORD.
Experimental Model
8
Service-oriented Grid
Provision physical
Service-oriented applications
Wrap applications as
Compose applications
“The Many Faces of IT as Service”, ACM Queue, Foster, Tuecke, 2005
9
Define membership & roles; enforce laws &
I.e., policy for service-oriented architecture Addressing dynamic membership & policy
Build, buy, operate, & share infrastructure
Decouple consumer & provider For data, programs, services, computing,
Address dynamics of community demand
10
Identify VO participants and roles
For people and services
Specify and control actions of members
Empower members delegation Enforce restrictions federate policy
A 1 2 B 1 2 A B 1 10 1 10 1 16 Access granted by community to user Site admission- control policies
Effective Access
Policy of site to community
11
Public key authentication & delegation Access control lists (“gridmap” files)
MyProxy: online credential repository VOMS, ACL/ gridmap management
12
Attribute Assertions
C asserts that S has attribute A with value V
Authentication and digital signature
Allows signer to assert attributes
Delegation
C asserts that S can perform O on behalf of C
Attribute mapping
{ A1, A2… An} vo1 { A’1, A’2… A’m} vo2
Policy
Entity with attributes A asserted by C may
13
Attribute Authority (ATA)
Issue signed attribute assertions
Authorization Authority (AZA)
Decisions based on assertions & policy VO A Service VO ATA VO AZA VO User A VO User B
14
Attribute Authority (ATA)
Issue signed attribute assertions
Authorization Authority (AZA)
Decisions based on assertions & policy VO A Service VO ATA VO AZA VO User A Delegation Assertion User B can use Service A VO User B Resource Admin Attribute
15
Attribute Authority (ATA)
Issue signed attribute assertions
Authorization Authority (AZA)
Decisions based on assertions & policy VO A Service VO ATA VO AZA VO User A Delegation Assertion User B can use Service A VO User B Resource Admin Attribute VO Member Attribute VO Member Attribute
16
Attribute Authority (ATA)
Issue signed attribute assertions
Authorization Authority (AZA)
Decisions based on assertions & policy VO A Service VO ATA VO AZA Mapping ATA VO B Service VO User A Delegation Assertion User B can use Service A VO-A Attr VO-B Attr VO User B Resource Admin Attribute VO Member Attribute VO Member Attribute
17
Local policy
authority CAS or VOMS issuing SAML
SSL/ WS-Security with Proxy Certificates
Authz Callout: SAML, XACML
MyProxy
18
Purpose
Access to large data
Policies
Per-collection control Different user classes
Implementation (GT)
Portal-based User
PKI, SAML assertions
Experience
> 2000 users > 100 TB downloaded
Optional review
See also: GAMA (SDSC), Dorian (OSU)
19
Define membership & roles; enforce laws &
I.e., policy for service-oriented architecture Addressing dynamics of membership & policy
Build, buy, operate, & share infrastructure
Decouple consum er & provider For data, program s, services, com puting,
Address dynam ics of com m unity dem and
20
Virtualize external services as VO services
Create new services from existing ones
“Service-Oriented Science”, Science, 2005
21
Negotiate service
Delegate and deploy
Provision to deliver
Configure environment Host layered functions Community A Community Z
22
Establish service agreement with service
E.g., WS-Agreement
Delegate use to VO user
User A
VO Admin
User B
VO User
23
WSRF (or WS-Transfer/ WS-Man, etc.), Globus GRAM, Virtual Workspaces
24
Data Mgm t Security Com m on Runtim e Execution Mgm t I nfo Services GridFTP
Authentication Authorization
Reliable File Transfer
Data Access & Integration
Grid Resource Allocation & Management
Index
Community Authorization
Data Replication Community Scheduling Framework Delegation Replica Location Trigger Java Runtime C Runtime Python Runtime WebMDS Workspace Management Grid Telecontrol Protocol
Credential Mgmt
Globus Toolkit Version 4: Software for Service-Oriented Systems, LNCS 3779, 2-13, 2005
25
Guidelines (Apache) Infrastructure (CVS, email, bugzilla, Wiki) Projects Include …
26
GT4 service for the creation, monitoring, &
High-level workspace description Web Services interfaces for monitoring &
Multiple implementations
Dynamic accounts Xen virtual machines (VMware virtual machines)
Virtual clusters as a higher-level construct
27
deploy, suspend
Client request VM EPR inspect & manage use existing VM image
Create VM image
VM Factory VM Repository VM Manager create new VM image Resource VM start program
28
OSG cluster Xen hypervisors TeraGrid cluster
“Virtual Clusters for Grid Communities,” Zhang et al., CCGrid 2006
29
Interface
Upload-push Upload-pull Deploy Undeploy Reload “HAND: Highly Available Dynamic Deployment Infrastructure for GT4,” Li Qi et al., 2006
30
Take a set of provisioned services …
This is traditional service composition
But must also be concerned with emergent
See the work of the agent & PlanetLab
“Brain vs. Brawn: Why Grids and Agents Need Each Other," Foster, Kesselman, Jennings, 2004.
31
Birmingham•
Cardiff
AEI/Golm
32
Pull “missing” files to a storage system
GridFTP Reliable File Transfer Service GridFTP
“Design and Implementation of a Data Replication Service Based on the Lightweight Data Replicator System,” Chervenak et al., 2005
33
Pull “missing” files to a storage system
GridFTP Local Replica Catalog Replica Location Index Reliable File Transfer Service Local Replica Catalog GridFTP
“Design and Implementation of a Data Replication Service Based on the Lightweight Data Replicator System,” Chervenak et al., 2005 Replica Location Index
34
Pull “missing” files to a storage system List of required Files
GridFTP Local Replica Catalog Replica Location Index Data Replication Service Reliable File Transfer Service Local Replica Catalog GridFTP
“Design and Implementation of a Data Replication Service Based on the Lightweight Data Replicator System,” Chervenak et al., 2005 Replica Location Index
35
Hypervisor/ OS Deploy hypervisor/ OS
Physical machine Procure hardware VM Deploy virtual machine
JVM Deploy container Deploy service GridFTP
36
Hypervisor/ OS Deploy hypervisor/ OS
Physical machine Procure hardware VM VM Deploy virtual machine
JVM Deploy container DRS Deploy service GridFTP LRC
GridFTP
37
Replica catalog, User-level multicast, …
38
Purpose
On-demand “stacks”
Challenge
Rapid access to 10-
Time-varying load
Solution
Dynamic acquisition
+ + + + + + = +
Sloan Data
Web page
Service
39
40
Com m unity based science will be the norm
Requires collaborations across sciences—
Many different types of com m unities
Differ in coupling, membership, lifetime, size
Must think beyond science stovepipes
Community infrastructure will increasingly
Scaling requires a separation of concerns
Providers of resources, services, content
Small set of fundam ental m echanism s
41
Globus Alliance
www.globus.org
Dev.Globus
dev.globus.org
Open Science Grid
www.opensciencegrid.org
TeraGrid
www.teragrid.org
Background
www.mcs.anl.gov/ ~ foster