e-Science Introduction Eric Yen e-Science Workshop, March 2011 - PowerPoint PPT Presentation

e-Science Introduction Eric Yen e-Science Workshop, March 2011

Outline • Workshop Overview • E-Science Basics • Landscape of e-Science • Application Development Concept • Security Infrastructure • Exemplar Applications 2

e-Science Workshop Overview • Objectives • Help user communities to take advantage of the global DCI – World Wide Grid • Engage close collaboration among regional user communities and with the Grid community • Target Audience • Both users and Grid/e-Science engineer • Of course, this is also good for novice to understand the e-Science, application development, related technology and the collaboration. • Two workshop on Natural Disaster Mitigation and Life Science are arranged. 3

4

e-Science Basics • “e-Science is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it. ... e-Science will change the dynamic of the way science is undertaken.” • By John Taylor, former Director General of Research Councils UK • Vision: a globally connected scholarly community promoting the highest quality scientific research • e-Science refers to either computationally intensive science or data intensive science that is carried out in highly distributed computing environment. • WLCG, EGEE, EGI, TeraGrid, OSG, EUAsiaGrid, ! 5

e-Infrastructure/ Cyberinfrastructure • Driven by Data Deluge • Turning data into insight and knowledge base efficiently • Open, consistent and well-designed data format, interface, protocol and quality code • Searchability, accessibility and sustainability • Resources and Tools are shared cross- disciplinarily • Enable Service-Oriented Science • “scientific research enabled by distributed networks of interoperating services” • New e-Infrastructure is required to host both the data and services 6

Distributed Computing Infrastructure for e-Science • Enabling collaboration to realize that the whole is grater than the sum of parts • WWG realized the global e-Infrastructure to share resources over Internet • Cloud offers versatile granularity and new usage patterns to the DCI services • Granularity: service-oriented layers in infrastructure, platform, software, data, network, etc. • Usage pattern: on-demand elasticity • More user customized and user controlled environment on remote Mário Campolargo European Commission - DG INFSO – resources OGF 23, Barcelona June 2008 7

The Grid • “a software infrastructure that enables flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources”. • Foster, Kesselman and Tuecke • Features • No central control • Production quality • Open standards and open architecture 8

Data-Driven Multiscale Collaborations for Complexity Great Challenges of 21st Century ! Multiscale Collaborations • General Relativity, Particles, Geosciences, Bio, Social... • And all combinations... ! Science and Society being transformed by CI and Data • Completely new methodologies • “The End of Science” (as we know it) ! CI plays central role • No community can attack challenges • Technical, CS, social issues to solve ! Places requirements on computing, software, networks, *Small groups still important! tools, etc 9 Source: Ed Seidel � !"

10 10

••• 11

EGI-InSPIRE • Integrated Sustainable Pan-European Infrastructure for !"#$%&'(#)*#+,-./01%(#1.#!(1&#/,#2,1. � Researchers in Europe • A 4-year project with " 25M EC contribution – Project cost " 70M – Activity cost ~ " 330M • EGI – European Grid Initiative – Deploying Technology Innovation • Distributed Computing continues to evolve – Grids, Desktops, Virtualisation, Clouds – Enabling Software Innovation • Provide reliable persistent technology platforms – Today: Tools built on gLite/UNICORE/ARC – Supporting Research Innovation • Infrastructure for data driven research – Support for international research (e.g. ESFRI) www.egi.eu EGI-InSPIRE RI-261323

e-Science in Asia • #$%&'($)*" • +&,-'./0$1.22*"2.'-&".34"152)5'.22*"4$%&'(&"$3"3.)5'&" • 6&%&2",7"(1$&3891"1,22.:,'.8,3",;&3"'&<&1)&4":*"3&)=,'>"1,33&18%$)*" • ?0&"'&-$,3".("."=0,2&")'.4$8,3.22*"$3&@/&'$&31&4"$3"'&-$,3.2"1,22.:,'.8,3" • +'$4(".34"A2,54("$3"B($." • C30,D,-&3&,5("+'$4(".34"A2,54("=$)0"2$D$)&4",/&'.8,3("&@/&'$&31&E" D.>$3-"1,22.:,'.8,3"4$F152)G" • H0*"&IJ1$&31&"$3"B($."K" • +2,:.2"$37'.()'51)5'&"$("&().:2$(0$3-"L5$1>2*" • ?.>&".4%.3).-&",7"(0.'$3-".34"1,22.:,'.8,3"),":'$4-&")0&"-./":&)=&&3"B($." .34")0&"=,'24" • ?,".44'&((")0&"10.22&3-&",7"'&-$,3.2"1,,/&'.8,3" " M+MMNMOB($.+'$4"0.%&"0&2/&4":5$24$3-")0&"53(&&3"P&-$,3.2"A,22.:,'.8,3G"Q3&"0,/&(" " D.3*",)0&'("=$22"0.//&3"(,,3R" " www.egi.eu EGI-InSPIRE RI-261323 14 "

Enabling Grids for E-sciencE 345%$$%./#60,70%((# 345%%'%'# 348%5/&9,.(:# !"#0,$%#;&(#<%=#>,0#/?%#80,2%5/#(-55%((@# Computational Chemistry Social Science Bioinformatics and Biomedical High Energy Physics S T Mitigation of natural disasters � www.egi.eu EGI-InSPIRE RI-261323

Features of Distributed Applications • C3)&',/&'.:$2$)*U"H,'>".1',(("D528/2&"4$()'$:5)&4" '&(,5'1&(" • #$()'$:5)&4"J1.2&IQ5)U"O82$V&"D528/2&"4$()'$:5)&4" '&(,5'1&("1,315''&3)2*" • M@)&3($:$2$)*U""J5//,')"3&="/.W&'3(N.:()'.18,3(E" /',-'.DD$3-"D,4&2(E"75318,3.2$)*"X" $37'.()'51)5'&" • B4./8%$)*U"P&(/,34"),"<51)5.8,3(",7"4*3.D$1" '&(,5'1&".34".%.$2.:$2$)*",7"4*3.D$1"4.)." • J$D/2$1$)*U"B11,DD,4.)&"4$()'$:5)&4"1,31&'3(".)" 4$Y&'&3)"2&%&2("&.($2*" Source: SAGA SZ"

Middleware Services for Grid App Seismic Wave Propagation Weather HEP Drug Discovery Simulation and Simulation Hazard Mapping Need to explore in more detail the requirements and scientific workflows GUI, CLI or Portal, application packages, together with client Application services Collective Application specific services, such as checkpointing, job (application- management, failover, staging, distributed data discovery and specific) backup, and workflow engine, customized services, etc. Collective Resource discovery, resource brokering, system monitoring, (Generic) community authorization, certificate revocation Access to computation, data; access to information about resource Resource matchmaking, system structure, status, and performance. Communication (IP), service discovery, authentication, Connectivity authorization, delegation Fabric Storage system, computers, networks, code repositories, catalogs S["

Grid and Cloud Logical Architecture Earth Environ. Social Life Security, Information, Accounting & Scienc Change Science Science ! e s Dynamic Computing Model (Application Environment) EMI Stacks API Monitor Distributed Resource Management & Services Job Management Data Service Service VM & Dynamic Resource Management Hardware Fabric

From the Basic Grid Use Case • On behalf of an authorized user, " AAI and single sign on services for versatile IaaS, PaaS and SaaS. • Tools interrogate the information system, " resource discovery or dynamic provisioning • Locates an optimal execution resource, submits the job to the execution resource, which in turn interprets the submitted job description and locates and fetches the necessary input data from a remote storage - also on behalf of the user. • Resource on-demand provisioning model with customized application environment • Elastic and efficient resource matchmaking model according to user-defined metrics and requirements • Storage space and file system on-demand • Job overflow and scalable automatically • Upon the completion, the newly created data is uploaded to a storage resource where this user is authorized (as a member of a Virtual Organization), registered in the necessary data indexing catalogs, and the job record is updated in the accounting and monitoring system. • Support streaming and minimization congestion, avoid duplicate transmission, by P2P technology • Enhanced accounting and monitoring system • Network virtualization S!"