IDGF International Desktop Grid Federation First Release of Desktop - - PowerPoint PPT Presentation
IDGF International Desktop Grid Federation First Release of Desktop Grids for e-Science Road Map Taipei, 2011-03-18 Ad Emmen, AlmereGrid DEGISCO project 1 DEGISO WP4 2011-03-18 version 1 Desktop Grids Introduction IDGF 2 DEGISO WP4
DEGISO WP4 2011-03-18 version 1
IDGF International Desktop Grid Federation
First Release of Desktop Grids for e-Science Road Map
Taipei, 2011-03-18
Ad Emmen, AlmereGrid
DEGISCO project
1
DEGISO WP4 2011-03-18 version 1
2
DEGISO WP4 2011-03-18 version 1
3
DEGISO WP4 2011-03-18 version 1
“On average, a desktop computer remains idle for around 60-85% of the time. Networks like EGI distribute computing tasks involving large amounts of data among the processing capacity of many thousands of separate desktop computers, putting their idle processor cycles to productive use. EGI-InSPIRE will give European researchers access to the aggregated processing power of 200,000 desk-top computers hosted by more than 300 centres around the world. The Commission is contributing €25 million over four years to the €73 million project. Other funding is provided from national sources such as National Grid Initiatives (NGI). “
3
DEGISO WP4 2011-03-18 version 1
“On average, a desktop computer remains idle for around 60-85% of the time. Networks like EGI distribute computing tasks involving large amounts of data among the processing capacity of many thousands of separate desktop computers, putting their idle processor cycles to productive use. EGI-InSPIRE will give European researchers access to the aggregated processing power of 200,000 desk-top computers hosted by more than 300 centres around the world. The Commission is contributing €25 million over four years to the €73 million project. Other funding is provided from national sources such as National Grid Initiatives (NGI). “
3
N
e n t i r e l y T r u e
c t u a l l y i t w i l l b e t h r
g h E D G I / D E G I S C O / I D G F S h e g i v e s a b
t 3 m i l l i
t
h e s e p r
e c t s
DEGISO WP4 2011-03-18 version 1
Number of Computers per Grid for a number of European Grids. These figures count computers (Desktop Grids) and nodes (Service Grids). The number of CPU's/cores can be larger. A computer can be part of more than one Grid. The BBC Climate Change project is currently not active. Resources owners per Grid for a number of European
Desktop Grids and sites for Traditional Service Grids. Note the large difference between Desktop Grids and Service Grids.
4
DEGISO WP4 2011-03-18 version 1
Number of Computers per Grid for a number of European Grids. These figures count computers (Desktop Grids) and nodes (Service Grids). The number of CPU's/cores can be larger. A computer can be part of more than one Grid. The BBC Climate Change project is currently not active. Resources owners per Grid for a number of European
Desktop Grids and sites for Traditional Service Grids. Note the large difference between Desktop Grids and Service Grids.
4
DEGISO WP4 2011-03-18 version 1
The International Desktop Grid Federation (IDGF) brings together: Desktop Grid operators Application developers Everyone else interested in Desktop Grid computing Open membership
5
DEGISO WP4 2011-03-18 version 1
6
DEGISO WP4 2011-03-18 version 1
First decade of Desktop Grids: Isolated projects Did do everything on their own Seti@Home, Folding@Home, WCG,.. But science is changing. Collaboration is getting more and more important Computational infrastructures for scientists did
Integrating scientific user environments Integrating infrastructures
7
DEGISO WP4 2011-03-18 version 1
Several local efforts to connect Desktop grids into wider computational infrastructures existed An effort to do this globally for Europe: EDGeS EDGeS was EC project 2008 - 2010 Connected Desktop Grids to EGEE
Xtremweb, BOINC gLite
Ported two dozen applications to this infrastructure Connected about 120.000 Desktop PCʼs
8
DEGISO WP4 2011-03-18 version 1
9
DEGISO WP4 2011-03-18 version 1
10
BOINC XtremWeb-HEP-E OurGrid
Job handler interface Grid handler interface
gLite Plugin Unicore Plugin Arc Plugin Job Source X-Web Client Job Source BOINC Client Job Source OurGrid Client Job database Queue Manager Bridge gLite (EGI) Unicore Grid KnowArc Grid
DEGISO WP4 2011-03-18 version 1
Set-up of the International Desktop Grid Federation (IDGF) to gather people, expertise and experience IDGF Launch: September 2010, Amsterdam at EGITF Initial support by two projects: EDGI 2010 – 2012 DEGISCO 2010 – 2012
11
Scope of the EDGI project
EDGeS scope only for compute intensive applications for EGEE (gLite) EDGI scope for both compute and data intensive applications for EMI/EGI (gLite, ARC, Unicore) Extend Desktop Grids with Clouds for QoS
12
RI-261561 WP4 2010-08-12
“Expand European DCIs into ICPC countries by supporting the creation of new Desktop Grids for e-Science in those countries and in Europe and by connecting them using the EDGeS bridge technology. Support applications on this expanded infrastructure, disseminate, promote and provide training about this expanded infrastructure and its usage”.
European e-Infrastructure EDGeS did connect 120.000 Desktop Grid computers with 150.000 core EGEE and
Goal of DEGISCO: Connect non-EU countries (ICPC countries in Europe, Asia and Latin-America) to this European infrastructure Help them to set-up local Desktop Grids
13
DEGISO WP4 2011-03-18 version 1
Support to use Desktop Grids by NGIs/EGI is provided by the International Desktop Grid Federation (IDGF) IDGF provides a number of services Your institute, organisation, or company and you as a person can become member of IDGF Help connecting Desktop Grids into the EGI infrastructure is an important goal So integrating Desktop Grids into EGI infrastructure, into the European Distributed Computing Infrastructure
14
DEGISO WP4 2011-03-18 version 1
IDGF is a member organisation You can become member, your organisation can, your company can, so do it IDGF provides services to its members. Services aimed at improving cooperation between those involved in Desktop Grids:
Member administration, including interest group and chapter management Dissemination services News service Discussion fora Support services Meetings and event organisation
15
DEGISO WP4 2011-03-18 version 1
Member administration, including interest group and chapter management
There are several interest groups that one can join. Two chapters: European and International
Dissemination services
Tell what you do, know what others do. Share experiences. Joint campaigns
News service
Blog, RSS/Atom, Twitter News for and from members
Discussion fora
16
DEGISO WP4 2011-03-18 version 1
Create material for citizens that works If it does: translate and reuse in other countries In IDGF already: French, Dutch, Hungarian, Russian, Chinese, Danish, Portuguese, Spanish, and even English speakers Re-use campaign elements that work:
material for schools media campaigns
Do joint campaigns
17
DEGISO WP4 2011-03-18 version 1
Create material for citizens that works If it does: translate and reuse in other countries In IDGF already: French, Dutch, Hungarian, Russian, Chinese, Danish, Portuguese, Spanish, and even English speakers Re-use campaign elements that work:
material for schools media campaigns
Do joint campaigns
17
http://www.omroepflevoland.nl/nieuws/recent? NewsKey=5479D73C07E37DA0C12575A5004E84D5
Good message
With EDGeS@Home you can help solve the worldʼs energy crisis Donate unused computing time to fusion research Program ISDEP of Spanisch scientists
Good initial coverage
Fusion communication departments Press, radio Available in local languages, local press releases
But...
Also lessons learned
DEGISO WP4 2011-03-18 version 1
EDGeS@Home - Donate to Energy Research
18
Good message
With EDGeS@Home you can help solve the worldʼs energy crisis Donate unused computing time to fusion research Program ISDEP of Spanisch scientists
Good initial coverage
Fusion communication departments Press, radio Available in local languages, local press releases
But...
Also lessons learned
DEGISO WP4 2011-03-18 version 1
EDGeS@Home - Donate to Energy Research
18
Good message
With EDGeS@Home you can help solve the worldʼs energy crisis Donate unused computing time to fusion research Program ISDEP of Spanisch scientists
Good initial coverage
Fusion communication departments Press, radio Available in local languages, local press releases
But...
Also lessons learned
DEGISO WP4 2011-03-18 version 1
EDGeS@Home - Donate to Energy Research
18
Good message
With EDGeS@Home you can help solve the worldʼs energy crisis Donate unused computing time to fusion research Program ISDEP of Spanisch scientists
Good initial coverage
Fusion communication departments Press, radio Available in local languages, local press releases
But...
Also lessons learned
DEGISO WP4 2011-03-18 version 1
EDGeS@Home - Donate to Energy Research
18
DEGISO WP4 2011-03-18 version 1
Expert support for Desktop Grid operators Expert support for application developers
Tutorials on Desktop Grids and integration into the standard scientific e-Infrastructure About one each month already Next one: April, EGI UF, Vilnius
19
DEGISO WP4 2011-03-18 version 1
An existing infrastructure connecting gLite with BOINC, XtremWeb and OurGrid desktop grids About two dozen applications have already been ported (as part of the EDGeS project) Support provided by EDGI and DEGISCO projects New developments from EDGI and DEGISCO
20
DEGISO WP4 2011-03-18 version 1
Road map contains a number of Best practices in setting up Desktop Grids and application supporting Technology neutral: BOINC, XtremWeb, OurGrid, etc. Legal issues and green aspects In a consultation phase now: so provide comments and suggestions.
21
DEGISO WP4 2011-03-18 version 1
22
DEGISO WP4 2011-03-18 version 1
22
DEGISO WP4 2011-03-18 version 1
22
DEGISO WP4 2011-03-18 version 1
23
DEGISO WP4 2011-03-18 version 1
23
DEGISO WP4 2011-03-18 version 1
23
DEGISO WP4 2011-03-18 version 1
24
DEGISO WP4 2011-03-18 version 1
24
DEGISO WP4 2011-03-18 version 1
24
DEGISO WP4 2011-03-18 version 1
25
DEGISO WP4 2011-03-18 version 1
Helps organisations in deciding on setting up a Desktop Grid Linking it into eScience infrastructures: EGI, Clouds, Grids Advice on all levels from technical to political
26
DEGISO WP4 2011-03-18 version 1
Helps organisations in deciding on setting up a Desktop Grid Linking it into eScience infrastructures: EGI, Clouds, Grids Advice on all levels from technical to political
26
DEGISO WP4 2011-03-18 version 1
Provide Road maps on how to set-up desktop grids and; Integrate them into the European eScience eInfrastructure Road maps on technology, organisation and politics A practical guide: a real road map Several versions You are invited to comment and contribute
27
DEGISO WP4 2011-03-18 version 1
A management part provides a general overview and advice
You can decide whether it makes sense in your situation to start a Desk top Grid
A detailed part gives “how to” information
What do you need to do. Where to get more guidance Based on years of experience condensed in several DEGISCO technical deliverables IDGF experience input
28
DEGISO WP4 2011-03-18 version 1
29
DEGISO WP4 2011-03-18 version 1
29
DEGISO WP4 2011-03-18 version 1
Best practices
Some successes of Desktop Grids
Survey
Are people interested in donating computing time?
Ecosystem
The eScience ecosystem: EGI, Taiwan Grid, Gisela, OGF,...
IDGF
International Desktop Grid Federation is key for further development
Companies
Desktop Grids in Business
Core Roadmaps
30
DEGISO WP4 2011-03-18 version 1
Road map - a general guide Road map - for local desktop grid at university or research centre Road map - Companies to set up and participate as part of a "Corporate Social Responsibility or Sustainable entrepreneurship” Green aspects of Desktop Grids Communication and marketing of Desktop Grids EDGeS@Home example The Future: Desktop Grids beyond the Cloud
31
DEGISO WP4 2011-03-18 version 1
1 Roadmaps 2 Technical Documentation 3 Communication and marketing 4 Legal and organisational information 5 Green Desktop Grids 6 Glossary
32
DEGISO WP4 2011-03-18 version 1
33
DEGISO WP4 2011-03-18 version 1
34
Applications
Science Grid applications
Build-on-Grid Own house construction
HealthGrid diagnosis agent based
AlmereGrid infrastructure
e- Education using Grid Strengthen knowledge infra- structure
BEinGRID backup/ restore
Broadband network
DEGISO WP4 2011-03-18 version 1
The university is set over four main campuses and some additional smaller locations in Central- and North-West- London Over 1800 of these machines are connected Grid can be used by the universityʼs researchers to run their compute intensive tasks
35
DEGISO WP4 2011-03-18 version 1
36
DEGISO WP4 2011-03-18 version 1
Short answer: Yes, they are Of course, there is more needed to actually let them participate
37
DEGISO WP4 2011-03-18 version 1
38
Scientific applications are OK Defense and commercial not
DEGISO WP4 2011-03-18 version 1
39
DEGISO WP4 2011-03-18 version 1
40
DEGISO WP4 2011-03-18 version 1
41
DEGISO WP4 2011-03-18 version 1
42
What do you want to achieve? When? Why?
Select goal Make a plan Select type Select/Deploy technology Deploy application Secure financing Study legal requirements Implement
Implement communication/ support Select user group
Answer all the important questions Who will you be serving? What are the needs of that users Volunteer Grid? Participate in existing Grid? Local Grid? BOINC? XtremWeb? OurGrid,... ? Get application running, test, validate What are we going to tell to whom? Who will take care of the volunteers?
Phase I Phase II Phase IV Phase V Phase III
DEGISO WP4 2011-03-18 version 1
43
DEGISO WP4 2011-03-18 version 1
44
Road map
Companies to set up and participate as part
Responsibility or Sustainable entrepreneurship”
Phase I: Check company polices Company polices restriction Phase II: Gather Desktop Grid related information from IDGF sources Phase III: Gather Desktop Grid related informatio
Check company polices Phase I Gather Desktop Grid related information Phase II from IDGF sources, build team and allocate (start finding) resources Gather Desktop Grid Phase III related information from local sources Implementation Phase IV Operation, training Phase V communication Phase VI Corporate Social Responsibility Programme
DEGISO WP4 2011-03-18 version 1
45
DEGISO WP4 2011-03-18 version 1
Are Desktop Grids green?
In Taipei: yes In Dubai they probably are not
Airco takes a lot of energy Within IDGF (DEGISCO) we are systematically investigating the energy costs of Desktop Grids You can join the survey and discussion forum Results useful for any Desktop Grid
46
DEGISO WP4 2011-03-18 version 1
Are Desktop Grids green?
In Taipei: yes In Dubai they probably are not
Airco takes a lot of energy Within IDGF (DEGISCO) we are systematically investigating the energy costs of Desktop Grids You can join the survey and discussion forum Results useful for any Desktop Grid
46
DEGISO WP4 2011-03-18 version 1
47
DEGISO WP4 2011-03-18 version 1
What can you do? Read the Road Map Use it Proved comments and feed back Help collect data Discuss on the Forum This helps to prepare the next, improved version
48
DEGISO WP4 2011-03-18 version 1 1 49
Globe adapted from http://upload.wikimedia.org/ wikipedia/commons/f/fa/ Globe.svg
DEGISO WP4 2011-03-18 version 1 1 49
Globe adapted from http://upload.wikimedia.org/ wikipedia/commons/f/fa/ Globe.svg