maximizing data accessibility at the esac science data
play

Maximizing data accessibility at the ESAC Science Data Centre J. - PowerPoint PPT Presentation

Dec 22, 2023 •20 likes •230 views

Maximizing data accessibility at the ESAC Science Data Centre J. Gonzlez-Nez, C. Arviset, J. Salgado ESAC Science Data Center (ESDC) Issue/Revision: 1.0 Reference: Asteriscs EDP ESDC Archives Status: Issued ESA UNCLASSIFIED - Releasable

  1. Maximizing data accessibility at the ESAC Science Data Centre J. González-Núñez, C. Arviset, J. Salgado ESAC Science Data Center (ESDC) Issue/Revision: 1.0 Reference: Asteriscs EDP ESDC Archives Status: Issued ESA UNCLASSIFIED - Releasable to the Public

  2. ESAC Science Data Centre The Digital Library of the Universe Ø Large set of science archives co-located at ESAC are a major research asset for community : • Astronomy, Planetary, Solar Heliospheric Ø Different types of data: • Raw data, calibrated processed data, high level data products, … • All data public and available on-line after a short proprietary period Ø Need to be kept readily available for future users and novel uses by various types of users: • Scientific Community (public access) • PI team and observers (controlled access) • Science Operations Team (privileged access) Ø Archive Strategy Plan for 5-20+ years Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 2 ESA UNCLASSIFIED - Releasable to the Public

  3. ESAC Science Archives Strategy Enable maximum scientific exploitation of data sets Enable efficient long-term preservation of data, software and knowledge, using modern technology Enable cost-effective archive production by integration in, and across, projects Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 3 ESA UNCLASSIFIED - Releasable to the Public

  4. International Collaboration and Interoperability amongst Archives IVOA - International Virtual Observatory Alliance • Formed in 2002, 20 member projects • Defines interoperability standards (VO framework) amongst astronomical (ground and space based) archives • Working Groups and Interest Groups per technical domain (data access, data model, registry, applications, semantics, operations, …) • http://www.ivoa.net/ IPDA - International Planetary Data Alliance • Formed in 2004, 12 space agencies • Defines archiving guidelines for planetary data • Defines interoperability standards amongst planetary archives • http://planetarydata.org/ Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 4 ESA UNCLASSIFIED - Releasable to the Public

  5. Data Architecture @ ESDC archives Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 5 ESA UNCLASSIFIED - Releasable to the Public

  6. OAIS to ABSI OAIS (2003) Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 6 ESA UNCLASSIFIED - Releasable to the Public

  7. OAIS to ABSI Archives Building System Infrastructure (2006) Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 7 ESA UNCLASSIFIED - Releasable to the Public

  8. ABSI based architecture Programmatic AIO science archive Browser GUI RPC Database SAMP VO services S*AP, SIAP Data Repository VO Apps Euro-VO Registry Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 8 ESA UNCLASSIFIED - Releasable to the Public

  9. Recent Drivers (Gaia, Euclid) 1. SOC integrated archives: a. Archive as a science operations building block b. Data processing metadata & intermediate products 2. Archive consortiums (DPAC, Euclid Consortium) a. Integration with WP-developed modules: Visualization, DM, Xmatch, Validation etc. 3. Large data volumes: a. Distributed data storage b. ‘Big Data’ analysis techniques Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 9 ESA UNCLASSIFIED - Releasable to the Public

  10. Archives Integration within Consortium Archive is fully part of Science Operations, from the start Strong collaboration with SGS and Consortium Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 10 ESA UNCLASSIFIED - Releasable to the Public

  11. Archives Integration within Consortium Archive is fully part of Science Operations, from the start Strong collaboration with SGS and Consortium Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 11 ESA UNCLASSIFIED - Releasable to the Public

  12. Data collections stats for Gaia Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 12 ESA UNCLASSIFIED - Releasable to the Public

  13. The Gaia Archive Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 13 ESA UNCLASSIFIED - Releasable to the Public

  14. Gaia Archive Current Architecture Programmatic TAP+, VOSpace science archive Browser GUI VO services TAP+ RDBMS SAMP TAP+, S*AP Distributed Data Repository VO Apps Euro-VO Registry Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 14 ESA UNCLASSIFIED - Releasable to the Public

  15. Extending VO protocols: TAP + TAP+ I/F Command Public area line tools • Publicly released catalogues Restricted area External • Catalogues during validation or Apps proprietary exploitation Data User Space Validation • User-uploaded data Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 15 ESA UNCLASSIFIED - Releasable to the Public

  16. ESASky : Multi-mission visualization ESASky backend Obs. metadata MOCs Catalogues TAP, HTTP All-Sky HIPS Maps sky.esa.int science archive science archive Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 16 ESA UNCLASSIFIED - Releasable to the Public

  17. ASTERICS at ESDC archives Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 17 ESA UNCLASSIFIED - Releasable to the Public

  18. Ongoing collaborations 1. ADQL Auto-complete Provide suggestions in the Gaia Archive ADQL • Query editor Stelios Voutsinas (WFAU, University of Edinburgh) • GENIUS - ASTERICS 2. pgSphere development Extension maintenance, configuration • management Implementation of Gaia CU9 requested features • Markus Nullmeier (ARI, Universität Heidelberg) - • ASTERICS Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 18 ESA UNCLASSIFIED - Releasable to the Public

  19. Challenges Gaia Archive Preparation (2011) Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 19 ESA UNCLASSIFIED - Releasable to the Public

  20. Challenges 1. “Bringing the code to the data” a. GAVIP project (Docker) b. TAP+: user areas, user defined functions 2. Cloud Services a. ESAC Cloud 3. Large scale processing (Spark, Hadoop): a. Data Mining Work Package b. ‘Grand Challenges’ c. Prototype simple use cases Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 20 ESA UNCLASSIFIED - Releasable to the Public

  21. Thanks! http://www.cosmos.esa.int/web/esdc Juan Gonzalez | ASTERICS European Data Provider Forum | 16/06/2016 | Slide 21 ESA UNCLASSIFIED - Releasable to the Public

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Space Science Data at ESA Christophe Arviset Head of Data and Engineering Division ESA-ESAC,

Space Science Data at ESA Christophe Arviset Head of Data and Engineering Division ESA-ESAC,

Space Science Data at ESA Christophe Arviset Head of Data and Engineering Division ESA-ESAC, Directorate of Science Christophe.Arviset@esa.int Open Universe Workshop, Vienna, 20-22 Nov 2017 ESA UNCLASSIFIED - For Official Use 5 in Operations

802 views • 16 slides
Working with astrometric data - warnings and caveats - U. Bastian / X. Luri ESAC Nov 2016,

Working with astrometric data - warnings and caveats - U. Bastian / X. Luri ESAC Nov 2016,

Working with astrometric data - warnings and caveats - U. Bastian / X. Luri ESAC Nov 2016, Heidelberg Jan 2017, Lund Aug 2017 ESAC November 2016 Scientists dream Error-free data No random errors No biases No

918 views • 62 slides
ESASky, all skies in your browser Bruno Mern ESAC Science Data Centre (ESDC), European Space

ESASky, all skies in your browser Bruno Mern ESAC Science Data Centre (ESDC), European Space

ESASky, all skies in your browser Bruno Mern ESAC Science Data Centre (ESDC), European Space Agency On behalf of Fabrizio Giordano, Elena Racero, Henrik Norman, Deborah Baines, Beln Lpez Mart, Jess Salgado, Sara Alberola, Christophe

169 views • 14 slides
Maximizing the Value of Data Analytics for Operational Risk Intelligence Don't just do data

Maximizing the Value of Data Analytics for Operational Risk Intelligence Don't just do data

Maximizing the Value of Data Analytics for Operational Risk Intelligence Don't just do data analytics. Transform data analysis activities into ORM programs. Presented by: Sergiu Cernautan, CPA, CISA; Director of GRC Strategy, ACL Services, Ltd.

740 views • 36 slides
NP02 data management and accessibility 10/31/2019 Elisabetta Pennacchio, IPNL 1 This

NP02 data management and accessibility 10/31/2019 Elisabetta Pennacchio, IPNL 1 This

NP02 data management and accessibility 10/31/2019 Elisabetta Pennacchio, IPNL 1 This presentation aims to describe NP02 data management and accessibility by discussing the following points: 1. Raw data description and data flow 2. Online data

1.08k views • 84 slides
Learning From Data Lecture 23 SVMs: Maximizing the Margin A Better Hyperplane Maximizing the

Learning From Data Lecture 23 SVMs: Maximizing the Margin A Better Hyperplane Maximizing the

Learning From Data Lecture 23 SVMs: Maximizing the Margin A Better Hyperplane Maximizing the Margin Link to Regularization M. Magdon-Ismail CSCI 4100/6100 recap: Linear Models, RBFs, Neural Networks Linear Model with Nonlinear Transform

349 views • 19 slides
Maximizing Accessibility: Engaging People With Disabilities in the Linux Community Spencer

Maximizing Accessibility: Engaging People With Disabilities in the Linux Community Spencer

Maximizing Accessibility: Engaging People With Disabilities in the Linux Community Spencer Hunley State of Accessibility in Linux: Past CLIs Primitive GUIs Improvements with GNOME 2 and KDE enviroments Linux State of

352 views • 9 slides
DataCamp Data Types for Data Science DataCamp Data Types for Data Science Data Set Overview

DataCamp Data Types for Data Science DataCamp Data Types for Data Science Data Set Overview

DataCamp Data Types for Data Science DataCamp Data Types for Data Science Data Set Overview Date,Block,Primary Type,Description, Location Description,Arrest,Domestic, District 05/23/2016 05:35:00 PM,024XX W DIVISION ST,ASSAULT,SIMPLE,

430 views • 13 slides
DataCamp Data Types for Data Science DataCamp Data Types for Data Science Data types Data type

DataCamp Data Types for Data Science DataCamp Data Types for Data Science Data types Data type

DataCamp Data Types for Data Science DataCamp Data Types for Data Science Data types Data type system sets the stage for the capabilities of the language Understanding data types empowers you as a data scientist DataCamp Data Types for Data

526 views • 23 slides
The I VOA Architecture Christophe Arviset (ESA/ ESAC) Head of the Science Archives and Computer

The I VOA Architecture Christophe Arviset (ESA/ ESAC) Head of the Science Archives and Computer

The I VOA Architecture Christophe Arviset (ESA/ ESAC) Head of the Science Archives and Computer Engineering Unit Science Operations Department Chair of the IVOA Technical Coordination Group ADASS XXI, Paris, 8 th November 2011 I VOA

400 views • 19 slides
ESASky: A new window for Solar System Data Exploration Elena Racero, Fabrizio Giordano & Juan

ESASky: A new window for Solar System Data Exploration Elena Racero, Fabrizio Giordano & Juan

ESASky: A new window for Solar System Data Exploration Elena Racero, Fabrizio Giordano & Juan Gonzalez On behalf of ESAC Science Data Centre (ESDC), European Space Agency In Collaboration with Benoit Carry, Observatoire de la Cote d'Azur

395 views • 18 slides
The European Surveillance of Antim icrobial Consum ption Project Arno Muller ESAC History

The European Surveillance of Antim icrobial Consum ption Project Arno Muller ESAC History

The European Surveillance of Antim icrobial Consum ption Project Arno Muller ESAC History Background Before 2001 Some national data available, but not comparable Few pan-European data available European Conference on

682 views • 29 slides
Data stories with The Pudding So what is data storytelling? Data academia & data science

Data stories with The Pudding So what is data storytelling? Data academia & data science

Data stories with The Pudding So what is data storytelling? Data academia & data science dense complex static/non-interactive often not accompanied by an easy-to-follow narrative Data art beautiful abstract

647 views • 5 slides
EMIS/DS 1300: A Practical Introduction to Data Science Slides by Michael Hahsler Data + Science

EMIS/DS 1300: A Practical Introduction to Data Science Slides by Michael Hahsler Data + Science

EMIS/DS 1300: A Practical Introduction to Data Science Slides by Michael Hahsler Data + Science = Results? Data Science ? Data Sources Results What is Data Science? Data science is a concept to unify statistics, data analysis, machine

635 views • 30 slides
Maximizing Talent Acquisition Data Sharlis Anthony & Mike DAndrea Client Services

Maximizing Talent Acquisition Data Sharlis Anthony & Mike DAndrea Client Services

Maximizing Talent Acquisition Data Sharlis Anthony & Mike DAndrea Client Services Division Office of Human Resources National Institutes of Health Agenda 1 2 3 4 5 Discussion: Leveraging Examples of How TA Data What is Talent

607 views • 19 slides
Using Genetic Algorithms for Maximizing Technical Efficiency in Data Envelopment Analysis Juan

Using Genetic Algorithms for Maximizing Technical Efficiency in Data Envelopment Analysis Juan

Data Envelopment Analysis Valid Solutions Genetic algorithm Hybrid metaheuristics Conclusions and future works Using Genetic Algorithms for Maximizing Technical Efficiency in Data Envelopment Analysis Juan Aparicio 1 enez 2 Domingo Gim

195 views • 17 slides
CPSC 213 Globals Machine model for access to global variables; static and dynamic arrays

CPSC 213 Globals Machine model for access to global variables; static and dynamic arrays

Learning Goals 1 Memory Endianness and memory-address alignment CPSC 213 Globals Machine model for access to global variables; static and dynamic arrays and structs Pointers Pointers in C, & and * operators, and

685 views • 23 slides
1 How How do do we get the electrons to do the work? we get the electrons to do the work? We

1 How How do do we get the electrons to do the work? we get the electrons to do the work? We

CS 240 CS 240 Computer Science I Computer Science I Acknowledgements: These slides were created by Dr. Travis Doom with information, graphics, materials, or aid kindly provided by Dr. Matt Rizki, Gaddiss Starting Out with Java

955 views • 66 slides
The shape of an algebraic variety 68 th Kuwait Foundation Lecture University of Cambridge,

The shape of an algebraic variety 68 th Kuwait Foundation Lecture University of Cambridge,

The shape of an algebraic variety 68 th Kuwait Foundation Lecture University of Cambridge, November 1st, 2007 Carlos T. Simpson C.N.R.S., Laboratoire J. A. Dieudonn e Universit e de Nice-Sophia Antipolis Equations Topology X algebraic

278 views • 23 slides
Recent results and and perspectives perspectives on on Recent results cosmic ray matter and

Recent results and and perspectives perspectives on on Recent results cosmic ray matter and

Aspen Winter 2009 Workshop on Physics at the LHC era Recent results and and perspectives perspectives on on Recent results cosmic ray matter and cosmic ray matter and antimatter from antimatter from Pamela experiment experiment

1.08k views • 91 slides
TARRANT COUNTY COLLEGE DISTRICT EXCELLENCE The Respiratory Care Program prepares graduates to

TARRANT COUNTY COLLEGE DISTRICT EXCELLENCE The Respiratory Care Program prepares graduates to

RESPIRATORY CARE INFORMATION SESSION TRINITY RIVER EAST CAMPUS TARRANT COUNTY COLLEGE DISTRICT EXCELLENCE The Respiratory Care Program prepares graduates to be successful, confident, and competent professional therapists who provide

436 views • 17 slides
What happens to babies born during health worker strikes? Willa Friedman Anthony Keats

What happens to babies born during health worker strikes? Willa Friedman Anthony Keats

What happens to babies born during health worker strikes? Willa Friedman Anthony Keats University of Houston Wesleyan University June 2018 Collegio Carlo Alberto Friedman and Keats Strikes and Births June 2018 1 / 23 Pumwani strike 2013

469 views • 30 slides
Lecture 02: user environment its a series of tubes Hands-On Unix System Administration DeCal

Lecture 02: user environment its a series of tubes Hands-On Unix System Administration DeCal

Lecture 02: user environment its a series of tubes Hands-On Unix System Administration DeCal 2012-09-10 1 / 22 Review Shell interaction Basic commands Shell variables Shell expansion Review Pipes 2 / 22 Shell interaction

260 views • 22 slides
DESY Experiences in Hydroforming of Elliptical RF Cavities W. Singer Introduction

DESY Experiences in Hydroforming of Elliptical RF Cavities W. Singer Introduction

DESY Experiences in Hydroforming of Elliptical RF Cavities W. Singer Introduction Hydroforming technique (necking, expansion) Nb tubes for hydroforming Examples of RF performance W. Singer. DESY Experiences in Hydroforming.

471 views • 32 slides

More recommend