Euclid OULE3 Jean-Luc Starck on behalf of the OULE3 team Euclid - - PowerPoint PPT Presentation

CEA - Irfu

Euclid OULE3

http://wiki.cosmos.esa.int/euclid/index.php/EC_SGS_OU_LE3

Jean-Luc Starck

• n behalf of the OULE3 team

Euclid

!!!!!!!

Euclid Consortium

OU-LE3

vendredi 8 janvier 16

2

OU-LE3 Internal Structure

Internal Data External Data 4 PFs 3 PFs Gal Clustering 6 PFs Weak Lensing 9 PFs Clusters 15 PFs Time Domain 7 PFs MWNG 3 PFs

From other OUs

Note: each box contains two WPs: one in charge

• f the implementation
• f the code and one in

charge of its validation Internal data WP is at the interface between

• ther OUs data and OU

LE3 WPs. External data WP is at the interface between non Euclid data and OU LE3 WPs.

47 Processing Functions (PF)

vendredi 8 janvier 16

3

!!!!!!!

Euclid Consortium

2015 SRR LE3 Review & Documentations

è SRR Documents: V1.0 in July 2014, V2.0 in October 2014 V3.0 in December .... V4.0 delivered on July 15. è Requirement Specification Document (RSD): è Validation Plan Document: è Development Plan Document è LE3 required simulation Document è Flowdown matrix requirement table è LE3 RIDs answer (Feb 2015) è LE3 Actions answer (July 2015)

The last 18 months have intense in term of documentation:

vendredi 8 janvier 16

4

!!!!!!!

Euclid Consortium

SRR Review

• Impossibility to reach a Maturity Level 2A at SGS PDR for all the LE3 PFs

– Wide scope and large number of LE3 PFs – Low number of skilled C++/Python developers within LE3 organizaIon (OU and SDC)

• Significant number of TBCs/TBDs in GDPRD requirements directly impacIng LE3 processing.
• Clarify the input catalog for LE3
• Error propagaIon

RIDs: 6 majors + 5 minors => 7 actions The four main actions consisted in answering these points:

vendredi 8 janvier 16

5

!!!!!!!

Euclid Consortium

Closest Developers Around 5-6 FTE per year, with only 3 real software engineers from SDC.

2015 2016 2017 2018 2019 FR - WL 0.8 1.8 2 2 2 FR - Clusters 0.95 0.95 0.25 0.25 0.25 IT - Clusters 0.5 0.5 0.5 0.5 0.5 IT - Galaxies 1.9 2.4 2.4 2.4 FR 1.75 1.75 2.25 2.25 2.25 UK (WL) 0.9 0.9 0.9 0.9 0.2 IT 2.4 2.9 2.9 2.9 0.5 TOTAL 5.05 5.5 6.05 6.05 3.65

vendredi 8 janvier 16

6

!!!!!!!

Euclid Consortium

LE3 Priorities

• P1 : PFs related to GDPRD Requirements (Primary Core Science) : primary goals of the project

(i,e. cosmological parameters) Examples of P1 PFs are 2-point statistics for galaxy clustering and weak lensing.

==> 12 PFs

• P2: PFs related to Cosmology + Existing algorithms + Available manpower

(secondary Core Science: cosmologically relevant information

Examples of P2 PFs are mass mapping, 3-point statistics and galaxy clusters detection.

==> 6 PFs

• P3 Other PFs related to Legacy Science, but with no available manpower.

Examples of P3 PFs are all PFs that will deliver data products for galaxy cluster science.

==> 22 PFs

• P4 PFs with no existing requirements (mainly PFs with requirements depending on the survey

strategy): Also Legacy Science. Examples of P4 PFs are all PFs that apply to transient phenomena. ==> 7 PFs

vendredi 8 janvier 16

Euclid Consortium Meeting – Lausanne – OU-LE3 splinter - 10/06/15 7

• Give maximum priority to P1 and P2 PFs development :

– Development objec.ves for PFs ranked P1 :

• Start code migraIons into CODEEN environment :

– Use C++ and/or Python – Use reference CODEEN libraries (local derogaIons accepted)

• Increase Maturity Level of the PFs, ideally up to 2A
• Implement Data Model according to Euclid rules
• Perform sizing and performances analyses

– Development objec.ves for PFs ranked P2 :

• For each PF, perform trade-off analyses between prototypes migraIon into CODEEN

environment and reuse of Third Party SoZware (see next slide)

• Perform sizing and performances analyses
• Provide punctual support to OU-LE3 prototyping acIviIes of P3 and

P4 PFs

LE3 development policy

From Jean-Jacques Metge

vendredi 8 janvier 16

8

!!!!!!!

Euclid Consortium

LE3-Reorganisation April 2015

PF Development Team Leader SDC actor OU actor OU-LE3 JL Starck, E. Branchini, F. Abdalla SDC-FR Manager

• M. Poncet

OU-LE3 WP Implementation Leader OU-LE3 WP Validation Leader

For each Processing Function (47)

OU-LE3 Dev JJ Metge National SDC-DEV LE3 Coordinator FR or IT or UK OU-LE3 Dev team = Integrated OU/SDC Team

vendredi 8 janvier 16

Euclid Consortium Meeting – Lausanne – OU-LE3 splinter - 10/06/15 9

• Use of Third Party SoZware can be considered for :

– The implementa.on of any P2 or P3 or P4 PF – The implementa.on of any V&V code used for the valida.on of any PF (from P1 to P4)

• Third party soZware :

– Are submiFed to formal deroga.on requests – Do not have to comply with CODEEN rules, but with a set of specific alleviated rules (see next slide) – Can be excep.onnally developed in other languages than C++ and Python (and consequently hosted as « standalone packages », ie binary code compiled

• utside the CODEEN environment)

– Will have IN ANY CASE to :

• respect CODEEN packaging standard, repository
• be deployed in SDC-PRODs through CODEEN

Third Party SoZware : definiIon

From Jean-Jacques Metge

vendredi 8 janvier 16

!!!!!!!

Euclid Consortium

OU-LE3 Update on LE3 activities

LE3-Implementation Activities

WL: 2PCF PF - Good progress of the new full C++ 2PCF-WL code. A first C++ version of the code has been delivered. Gal-clustering: 5/6 PFs (including all P1) have passed the first Maturity Level

• Gate. For the sixth one (Bispectrum) a C++ code has been implemented. Its

performances are being assessed. The flagship code (2-pt correlation function) is being reengineered and paralleled by SDC-IT. Code challenge ongoing for the C++ power spectrum code. Clusters: Main focus on the (P1) closer finder code. Third code-challenge

• completed. Results are being x-compared. Code for cluster richness implemented

and applied to real data. Code developed which is relevant to the estimate of the cluster selection function.

vendredi 8 janvier 16

11

!!!!!!!

Euclid Consortium

2PCF Weak Lensing Agenda

ML1B : in February 2016 ML2A: in June 2016 - by the technical Key Point 1- SGS-TK1 Status New python code of the 2PCF position-position estimator has been implemented Docs ready (code, user manual, SDD, performance doc, …) Code ready Tests ready – unitary – mock – validation it provides Shear-shear 2PCF, Shear-position 2PCF, Position- Position 2PCF estimators. it provides E-/B-mode second-order statistics. From FITS/ASCII files and Cartesian/Spherical coordinates, and Linear/Logarithmic scales. Performance C Athena code is the reference

Relative precision reached : 1e-6/1e-7 10% faster

vendredi 8 janvier 16

!!!!!!!

Euclid Consortium

OU-LE3 Update on LE3 activities

LE3-Validation Activities

Weak lensing: a python pipeline has been written which takes in input a catalog, reconstruct a mass map or an aperture mass, and count the peaks. It should be very useful to set up requirements for mass mapping.

• ongoing test with MICE simulations.
• ongoing discussions on partial validations (requirements are not there).

Gal-clustering.. Validation ongoing for 2 PFs: 2-point correlation and power

• spectrum. 2 main activities. Code challenge on existing Euclid-mock light-cone

catalogues (Durham). Creation of ideal locks with known 2-point clustering statistics and application of existing codes (Trieste).

• Clusters. Validation activity is identified with code challenges. Challenge n. 3 just

completed for cluster finders.

vendredi 8 janvier 16

!!!!!!!

Euclid Consortium

OU-LE3 Conclusions

n OU-LE3-DEV has a plan, with different levels of priority. n More interactions with both the SWGs and the SDCs. n The SWG Requirement Development plan is very late relative to the SDC development plan: è it is a problem for algorithm selection (especially when several alternative methods exist; e.g. mass mapping). è one solution is to implement basic algorithms, and to change them when the requirements are there (this requires flexibility from SDC) .

n Risks for LE3

è Most requirements remain TBD. è Management of the E2E pipeline: segments involved in some PFs in which LE3 is responsible for the deliverable, is in control of the final step, but does not (and should not) control of the previous steps. The risk here is related to the assumption that the pipeline is guaranteed to work. è Software engineer FTE remain weak for LE3. Athena code = WL 2PCF code in C, available on the web and used in large projects (CFHTLens). Estimated Workforce required to achieve 2A maturity level : 1.2 FTE è SDC computer may not be well adapted to LE3. è The use of third party software may not be possible.

n Next LE3 Meeting, Milan, January 25-27, 2016 http://euclidgc2016.brera.inaf.it

vendredi 8 janvier 16