CoSADIE's Data Center Census Preliminary results Gabriel Stckle - - PowerPoint PPT Presentation

CoSADIE's Data Center Census Preliminary results Gabriel Stöckle gst@ari.uni-heidelberg.de CosADIE DC Forum Heidelberg June 10, 2013

Montag, 10. Juni 13

We want to find out

• About the Landscape of European Data Centers,

adoption of VO standards, etc.

• What are the requirements of the different Data

Centers for future VO activity

1 Montag, 10. Juni 13

The Former surveys

Analogous efforts 2008 (EuroVO-DCA ) and 2010 (EuroVO-AIDA) with 69 participants Approach: 5 Groups with Questions

• Introduction and Identification of the DC, 17
• Observational Archives and Data Products, 29
• Services / Tools / Software Suites, 17
• Theoretical Archives, 28
• Theory Services, 14 questions

Result: about 120(!) classes of DCs, very wide census, only scratching the surface

2 Montag, 10. Juni 13

A new approach - Iterating our knowledge

Complex, multi-part questionnaires are a starter, but

• mismatches between what clients wanted to say

and what was asked

• quality of responses difficult to derive

3

We try/tried:

• Allow Data Center and Ressource Description

as free text

• Fields are pre-filled from old answers to ease

answering process

• Answers are curated and eventually iterated

with the respondent

Montag, 10. Juni 13

We let Data Centers describe themselves

„Data Center Description „give whatever additional information you deem useful for describing your data center... “

• Aims, thematic/scientific focus, wavebands covered.
• What is your target audience? Can you estimate

what percentage of it is aware of your services?

• Do you offer dedicated user support facilities? “

http://g-vo.org/cosadie-census/showcase Showcase:

4

Montag, 10. Juni 13

For each resource, we would be interested in aspects like: Description When did you establish your data collection? Do you have expectations on its lifetime? Kind of archived data (images, spectra, catalogues, time series, etc). How was the data generated? ...

We let them describe their resources

5

Montag, 10. Juni 13

Its not that easy!

We collected

DCs by web search DCs in our own research networks DCs known from the former Surveys

contacted

All known DCs directly by email New DCs by mail plus phone

asked

National EuroVO representatives personally

Now, we have 96 DCs, including you?

6

www.g-vo.org/cosadie-census

Montag, 10. Juni 13

Its not that easy!

We collected

DCs by web search DCs in our own research networks DCs known from the former Surveys

contacted

All known DCs directly by email New DCs by mail plus phone

asked

National EuroVO representatives personally

Now, we have 96 DCs, including you?

6

www.g-vo.org/cosadie-census

Montag, 10. Juni 13

6 % 3 % 20 % 3 % 14 % 14 % 40 %

FR DE IT UK ES NL Others unknown/EU

Participation: Spain visible at DC Forum UK more visible in survey

15 % 10 % 3 % 7 % 7 % 11 % 18 % 28 %

Forum Participants per country Dc's in survey per country FR DE IT UK ES NL Others unknown/ EU Total 27 16 11 7 7 3 10 14 96 FR ES DE IT UK ESO/ ESA

• thers Total

14 7 5 5 1 2 1 35

Montag, 10. Juni 13

The answers are of different quality

We received 61 answers from 96 DCs from all fields of astrophysics Most didnt change their description or only in some details Only 22 Data Centers changed their description Satisfied with the answers we created? DCs annoyed by the questionnaire?

• Answering takes time
• people dont see what‘s the reason for a

different survey on the same topic? We need to intensify the communication.

8 Montag, 10. Juni 13

What is a Data Center after all?

Wikipedia: ... house computer systems, associated components, ... ... assigned to institutions with requirement in large data handling (finance, research)

10

Google Data Center, The Dalles In the VO: DCs are (slightly different) ... data publishers, metadata and service providers. .. providers of physical storage and computational ressources.

Montag, 10. Juni 13

Trying to classifify data centers

Specialists: are science driven, associated to scientific projects

• r observatories.

are research specific or observatory related. Generalists have overall concepts/technologies or resources for data handling. are without a specific scientific focus. International Organizations (ESO, ESA) run large European data centers.

12 Montag, 10. Juni 13

The Landscape: Example: Germany

DCs in the questionnaire: 16 Generalists: 2

AIP, ZAH

International Organization: 1

ESO Science Archive (Munich)

Observatory/project related: 6

HES, Euclid, eROSITA, Sonneberg, SUMER, ROSAT

Research Specific: 6

e.g. CDMS, Cologne Database of molecular spectra e.g. PHOENIX, Stellar atmosphere code e.g. GRBGEN, catalog of gamma ray bursts 1 entry is probably no DC but has some data and wants to share it via VO standards

13 Montag, 10. Juni 13

Example: Spain

DCs in the questionnaire: 7 Generalists: 1

LAEFF

International Organization: 1

ESA-ESAC

Observatory/project related: 2

AXIS XMM Int. Survey, DC of the Instituto de Astrofisica de Canarias

Research Specific: 2

AMIGA, Analysis of the interstellar Medium of Isolated GAlaxies, Radio Data Model for the VO PVOL-SVO, Planetary Virtual Observatory Laboratory, Observations of Giant Planety

14 Montag, 10. Juni 13

Anyway, here are some results:

The VO concept is widely known

• Services that are well established: Webservices,

TAP, SAMP

• Less known/accepted: ADQL

Technical support is necessary in setting up VO services/own archives Some scientific community‘s needs are not all solved by VO approaches Some DCs don‘t care about VO for different reasons, mainly data policy or missing manpower

9 Montag, 10. Juni 13

Different needs

Generalists/International Institutions care about

• interoperability
• reaching project related DCs with their standards
• overall technical expertise
• spreading general standards/approaches

Observatory/project related DCs care about

• expertise in their scientific field
• practical technical solutions for their scientific community
• Dont care about politics

Both care about: Citations and users doing research with their data or tools

15 Montag, 10. Juni 13

Next Steps

We have to

• Identify DCs requirements much clearer
• Communication has to intensify
• DC‘s answers are a starter for a discussion

This forum: place to dicuss your needs! Lets find out what is helping you to reach your goals.

16 Montag, 10. Juni 13

Next Steps

We have to

• Identify DCs requirements much clearer
• Communication has to intensify
• DC‘s answers are a starter for a discussion

This forum: place to dicuss your needs! Lets find out what is helping you to reach your goals.

16

P .S.: Please send your slides to dcforum2013@g-vo.org

Montag, 10. Juni 13