What is Grid Computing? Overview & Definitions GridKa-School - - PowerPoint PPT Presentation

What is Grid Computing? Overview & Definitions

GridKa-School 2006 P.Malzacher@gsi.de

What is Grid Computing?

Best thing since the WWW. Don’t worry, the grid will solve all our computational and data problems! Just click “Install”

• r

The grid is “merely an excuse by computer scientists to milk the political system for more research grants so they can write yet more lines of useless code” [The Economist, June 21, 2001] “A distraction from getting real science done” [McCubbin]

The term Grid computing

• riginated in the 1990s

as a metaphor for making computer power as easy to access as an electric power grid.

Researchers in many locations need to share resources Scientific instruments, data stores and computers in many locations

Before Grids

FTP, telnet, blood, sweat and tears… and little support for collaboration

There must be a better way of doing this!!!

(next three slides stolen from Brian Coghlan)

Resources connect to “The Grid” Researchers in many locations need to share resources Scientific instruments, data stores and computers in many locations

With Grids

Middleware

Workstation Mobile Access Data-storage, Sensors, Experiments Internet, networks Supercomputer, PC-Cluster Visualising

DLEWARE

1989-2001 invented Tim Berners- Lee at CERN the WWW to share information.

Web Server, Web Clients Agreed protocols: HTTP, HTML, URLs Anyone can access information and post their

• wn

Moved quickly into public use.

The Web is a service for sharing information over the Internet, the Grid is a service for sharing computer power and data storage capacity over the Internet. The Grid goes well beyond simple communication between computers, and aims ultimately to turn the global network of computers into one vast computational resource.

Grids are the evolution of distributed systems to a wide area, multi-organizational context.

The Grid Problem

From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”

Flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources. Enable communities (“virtual organizations”) to share geographically distributed resources as they pursue common goals

• - assuming the absence of…

central location, central control,

• mniscience,

existing trust relationships.

What is the Grid? A Three Point Checklist Ian Foster, July 2002

A Grid is a system that:

• 1. coordinates resources that are not subject to centralized

control

• 2. using standard, open, general-purpose protocols and

interfaces

• 3. to deliver nontrivial qualities of service.

Grid Checklist:

• 1. coordinates resources that are not subject to

centralized control …

A Grid integrates and coordinates resources and users that live within different control domains for example, the user’s desktop vs. central computing; different administrative units of the same company;

• r different companies;

and addresses the issues of security, policy, payment, membership, and so forth that arise in these settings. Otherwise, we are dealing with a local management system.

Virtual Organizations: Collecting resources around the world. Provide seamless access, with a single sign on.

Virtual organisation: people and resources collaborating - across admin, organisational boundaries.

Single sign-on I connect to one machine – some sort of “digital credential” is passed on to any other resource I use, basis of:

Authentication: How do I identify myself to a resource without username/password for each resource I use? Authorisation: what can I do? Determined by My membership of VO VO negotiations with resource providers

Grid middleware runs on each resource User just perceives “shared resources” with no concern for location or owning organisation

Grid Checklist:

• 2. using standard, open, general-purpose

protocols and interfaces

A Grid is built from multi-purpose protocols and interfaces that address such fundamental issues as authentication, authorization, resource discovery, and resource access. It is important that these protocols and interfaces be standard and open. Otherwise, we are dealing with an application-specific system.

Grid and Web Services Convergence

The definition of WS Resource Framework means that the Grid and Web services communities can move forward on a common base.

Web Service OGSA Application

Web services: use case

Grid Checklist:

• 3. to deliver nontrivial qualities of service.

A Grid allows ist constituent resources to be used in a coordinated fashion to deliver various qualities of service, relating for example to response time, throughput, availability, and security, and/or co-allocation of multiple resource types to meet complex user demands, so that the utility of the combined system is significantly greater than that of the sum of its parts.

Grid Perspectives

Users Viewpoint: A virtual computer which minimizes time to completion for my application while transparently managing access to inputs and resources Programmers Viewpoint: A toolkit of applications, services, and API’s which provide transparent access to distributed resources Administrators Viewpoint: An environment to monitor, manage and secure access to geographically distributed computers, storage and networks.

The Integrating Role of Grid Infrastructure

Coarse Grained Dev / Test

Grid I nfrastructure

Multiple applications and workload types Multiple resource types and instances Consistent & open management interface Consistent & open enactment interface End-to-end Quality of Service

Fine Grained Data Driven Workflow

Economic foundations for infrastructure:

Infrastructure: Everything that supports a number of applications and is specific to none; Including capabilities assisting the development, the provisioning, the administration and the execution of applications. Economic foundations for infrastructure supporting applications: Sharing of resources, Reuse of designs, Productivity-enhancing tools.

Benefits of sharing

Lowered costs for provisioning and operating a new application based on an infrastructure already in place; Software requires a material infrastructure (hosts, data access, networks) to support is execution, sharing that material infrastructure can exhibit economics of scale. Statistical multiplexing to relieve congestions.

Reuse of designs:

Reusable modules, components and frameworks seek to contain one of the most important costs, that of development. On the other hand, designing and developing reusable software modules, components and frameworks take considerably more time, effort, and expense than single-purpose designs. There must be a compelling expectation of multiple uses before this becomes economical feasible.

Tools

Productivity enhancing tools are crucial to operation and development. By automating many tasks tools reduce time, effort, and expense, and contribute to quality. For a homogeneous infrastructure the cost of developing and learning the use of such tools are shared. Operation and administration processes and scripts can be shared.

Two Key Grid Computing Groups

The Globus Alliance is a community of organizations and individuals developing fundamental technologies behind the "Grid," which lets people share computing power, databases, instruments, and other on- line tools securely across corporate, institutional, and geographic boundaries without sacrificing local autonomy. The Globus Toolkit is an open source software toolkit used for building Grid systems and applications. It is being developed by the Globus Alliance and many others all over the world. A growing number of projects and companies are using the Globus Toolkit to unlock the potential of grids for their cause.

The Global Grid Forum (www.ggf.org) Grid standard activities First meeting in June of 1999 Heavy involvement of Academic Groups and Industry Process

Meets three times annually Solicits involvement from industry, research groups, and academics

GGF18 Washington DC, USA September 11-14, 2006

What End Users Need

(Slide from Globus Primer)

Secure, reliable, on- demand access to data, software, people, and

• ther

resources (ideally all via a Web Browser!)

How it Really Happens

(Slide from Globus Primer) Web Browser Compute Server Data Catalog Data Viewer Tool Certificate authority Chat Tool Credential Repository Web Portal Compute Server

Resources implement standard access & management interfaces Collective services aggregate &/ or virtualize resources Users work with client applications Application services

• rganize VOs & enable

access to other services

Database service Database service Database service Simulation Tool Camera Camera Telepresence Monitor Registration Service

How it Really Happens

(Slide from Globus Primer)

Implementations are provided by a mix of Application-specific code “Off the shelf” tools and services Tools and services from the Grid community (Globus +

• thers using the same standards)

Glued together by… Application development System integration

Generic platform and generic Grid services D-Grid Integration Project

Astro Grid Medi Grid C3 Grid HEP Grid In Grid Text Grid ONTOVERSE Wikinger C3 Grid

Grid Computing & Knowledge management & e-Learning e-Science =

PC² RRZN TUD RZG LRZ RWTH FZJ FZK FHG/ ITWM Uni-KA

D-Grid WPs: Middleware & Tools, Infrastructure, Network & Security, Management & Sustainability

Middleware:

Globus 4.x gLite (LCG) UNICORE GAT and GridSphere

Data Management:

SRM/dCache OGSA-DAI Meta data schemas

VO Management:

VOMS and Shibboleth

Where are the performance metrics for success of the Grid?

(Jennifer M. Schorf)

No more “Grid” papers, just a footnote that states “This work was achieved using the Grid”. Supercomputer centers don’t give a user the choice of using their machines or the Grid, that line doesn’t exist. SuperComputing demos can be run at any time of the year. Basic Grid middleware comes with the operating system.

What is Grid Computing?

Best thing since the WWW. Don’t worry, the grid will solve all our computational and data problems! Just click “Install” The grid is “merely an excuse by computer scientists to milk the political system for more research grants so they can write yet more lines of useless code” [The Economist, June 21, 2001] “A distraction from getting real science done” [McCubbin]

The grid is still work in progress, but it can solve our problems, because we design it too! We must work closely with the developers as it evolves, providing our requirements and testing their deliverables in our environment.

The Computer engineering hierarchy.

The layers emulate the structure of computer systems, with each layer contributing critical capabilities to the layers above it. The pillars on either side support all the layers with key development activities, such as tool creation, and deployment characteristics, such as security. Platform Network Grid Environment Interface Organization Development Deployment

Paul S. Rosenbloom, IEEE Computer

The grid layer converts networks of platforms into shared-

resource pools. The main challenge is in providing services that yield uniform access across the full set of networked resources despite their distribution across geographical and organizational boundaries. Issues include cross-domain authorization and security, resource discovery and allocation, data movement and interoperability, workflow management, and metadata description. Platform Network Grid Environment Interface Organization Development Deployment

Paul S. Rosenbloom, IEEE Computer

The grid layer supports the development of environments that embody the data, information knowledge, and models that a domain comprises as well as the processes that operate on these environments to generate additional content and insights, such as calculation, fusion, reasoning and simulation. The main challenge of the environment layer is how to organize resources over entire domains of interest such as a science or engineering discipline. Platform Network Grid Environment Interface Organization Development Deployment

Paul S. Rosenbloom, IEEE Computer

Users access environments through interfaces. The main challenge is providing a low-overhead way for people to effectively interact with the content of the environment layer. Platform Network Grid Environment Interface Organization Development Deployment

Paul S. Rosenbloom, IEEE Computer

On the top of the interface is the organization layer, where the main challenge is supporting groups of people – and goal-

• riented systems such as agents and robots – in working toward

common goals. Issues include dynamically creating such

• rganizations; enabling them to span geographical,
• rganizational, and political boundaries; providing coordination

and collaboration assistance; and converting loose communities into effective organizations. Platform Network Grid Environment Interface Organization Development Deployment

Paul S. Rosenbloom, IEEE Computer

For More Information on the Grid