From EGEE Operations Portal From EGEE Operations Portal towards EGI - - PowerPoint PPT Presentation

Enabling Grids for E sciencE Enabling Grids for E-sciencE

From EGEE Operations Portal From EGEE Operations Portal towards EGI Operations Portal

Hélène Cordier CNRS-IN2P3 IN2P3-CC CNRS-IN2P3, IN2P3-CC ISGC March 9th-12th 2010

www eu egee org

Taipei, Taiwan

EGEE-III INFSO-RI-222667

www.eu-egee.org

COD21 22 Sept 2009 http://cic.gridops.org/ ISGC, March 10th 2010

Enabling Grids for E-sciencE

Content

EGEE Operations portal

• customers : EGEE actors (site, federation, operators)

and main features of the Operations Portal

• EGEE Operations Model

EGEE Operations Model

• Situation at the end of EGEE-III

EGI Operations portal

• Backend Reorganization: Symfony
• Enhancement of standardized access mechanism: Lavoisier

Enhancement of standardized access mechanism: Lavoisier

• Current and future work

EGEE-III INFSO-RI-222667

2/12

Enabling Grids for E-sciencE

EGEE Operations Portal features

RESSOURCE PROVIDER USER : end-user or community

VO info Site info

Integration Tools Integration Tools CIC Portal

Monitoring tools User Support Communication tools

OPERATOR EGEE-III INFSO-RI-222667 OPERATIONS CENTER OPERATOR

3/12

Enabling Grids for E-sciencE

EGEE Operations portal services

Gstat Service Availability Monitoring

Production Production Ressource Ressource provider provider

Workflows Workflows & & Operations Tools Operations Tools C i ti C i ti Communication Communication

Overview Overview Operator Operator VO VO ops

• ps info

info VO ID VO ID Card Card Operator Operator Management Management VO life cycle VO life cycle

EGEE-III INFSO-RI-222667

4/12

Enabling Grids for E-sciencE

EGEE Daily Operations Model

O f – Overview of the ressource providers activity: SAM based Operations Dashboard created December 2004 – EGEE-I : Implementation centrally driven by 4 federations Procedures build-up EGEE II O ti d i b 11 f d ti – EGEE-II : Operations driven by 11 federations Procedures upgrade/ training material Build-up of QoS: metrics/failover Build up of QoS: metrics/failover – EGEE-III : decentralization of operations New model definition – Continuous improvement through quarterly meetings – De facto tight interaction with Sam, Samap, gstat, gocdb, dashboard g , p, g , g , – Training sessions and workhops on procedures and tools D t li d ti d hb d b d N i l i

EGEE-III INFSO-RI-222667

Decentralized operations dashboard based on Nagios relying on standard-based Web technology

5/12

Enabling Grids for E-sciencE

Implementation of Symfony Framework

Incidental

– Multi developpers environment O ti i ti f d l t – Optimisation of development – Streamlining of code for future customisation of workflows

St t l N d Structural Needs

– Optimize maintenance time – Integrate my sql for future customisation – Anticipate multi-linguistic feature – Provide packaging for EGI operational entities/ NGI or group of NGIs Advantages: Optimisation of conception, uniform structure for developments, maintenance, pattern automation, multi-OS platform and multi DB compatible. Drawbacks: Heavy initial reorganization workload

EGEE-III INFSO-RI-222667

Drawbacks: Heavy initial reorganization workload

6/12

Enabling Grids for E-sciencE

Enhancement of Lavoisier ref. CHEP 2006

Service Data Views Manager Data Sources

Data

Service Service Data Views Manager Data Sources

Data getView

flat file adapter flat file flat file

startup notified triggers:

Data View getView getView getView

flat file adapter flat file flat file flat file flat file

startup notified triggers:

Data View

SQL adapter

getDataView

RDBMS

Data View

about to expire triggers:

SQL adapter

getDataView

RDBMS RDBMS

Data View

about to expire triggers:

WS adapter

g

WS

Data View

WS adapter

g

WS WS

Data View processXSL

XSLT adapter

refreshed

Data View

triggers:

processXSL processXSL

XSLT adapter

refreshed

Data View

triggers:

config config config config

Advantages: Standardisation of data access , Uniformisation of information management Reliability Ease of maintenance

EGEE-III INFSO-RI-222667

management, Reliability, Ease of maintenance Drawbacks: Heavy-weight for single info source

7/12

Enabling Grids for E-sciencE

Dashboard status at EGI start

Customized views hosted centrally Beta-package for local hosting available Available in production : end of March 2010

Web portal module

User

p

Lavoisier module Database module

Internal data External data sources

REGIONAL SCENARIOS REGIONAL SCENARIOS (1) Full distributed model regional helpdesk, regional helpdesk, dashboard + DB (2) Partially distributed ( ) y model : dashboard + DB

EGEE-III INFSO-RI-222667

(3)Helpdesk distributed

8/12

Enabling Grids for E-sciencE

Current work

– Data access standardization via common re-engineering of front-end and back-end of the site and operational info d t b t th ith GOCDB t databases together with GOCDB team Ergonomic global info management All kind of decentralization scenarios All kind of decentralization scenarios – Re-design of existing key features of the portal to provide Re design of existing key features of the portal to provide information via standard xml format VO operational information, life-cycle; EGEE Broadcast – Customization with Operations portal v2.0 : https://operations-portal.in2p3.fr/ Dashboard, Broadcast, Downtimes subscription, VO life- cycle resource provider tools

EGEE-III INFSO-RI-222667

cycle, resource provider tools Integration of other DCI (GISELA)

9/12

Enabling Grids for E-sciencE

Future work

– Operations Portal within JRA1 activity in EGI-Inspire OTAG Operations Tools Advisory Group monthly OTAG – Operations Tools Advisory Group monthly

• tag@mailman.egi.eu

3rd meeting in Upsalla at the UF 5 3rd meeting in Upsalla at the UF 5 https://www.egi.eu/indico/categoryDisplay.py?categId=4

EGEE-III INFSO-RI-222667

10/12

Enabling Grids for E-sciencE

Need to know more

Try the dashboard beta-package: https://forge.in2p3.fr/documents/show/76 Open a GGUS ticket: dedicated CIC Portal SU / OTAG SU C t t i i f ti @i 2 3 f / f db k f Contact us: cic-information@in2p3.fr / feedback form

EGEE-III INFSO-RI-222667

11/12

Enabling Grids for E-sciencE

Backup Backup slides slides COD activity Transition to EGI

OE5 (PL, NL) - OE13 (FI) project-eu-egee-sa1-c-cod- followup@cern.ch

• POLE1 : CE, NE
• Nagios acceptance criteria/timeline

Model/metrics

p@ Oversight of daily operations Best practices/training

• Model/metrics
• POLE2 : NE
• Ops Manuels, Best practices

OTAG:

• tag@mailman.egi.eu

GOCDB O ti P t l

p , p

• Training guides, How-to
• Feedback on NGI readiness

POLE3 FR

GOCDB, Operations Portal, Dashboard, …

• POLE3 : FR
• Failover Procedure for recommendations
• Procedure on NGI operations set-up

Procedure on NGI operations set up

• Forge Forum/Wiki/Documents

EGEE-III INFSO-RI-222667

12/12

Enabling Grids for E-sciencE

Implications of L2 regionalization (regionalized instance deployed nationally)

• A regionalized tool requires

– interoperation between the national and the central instance interoperation between the national and the central instance (interoperation with non-EGI grids needs to be addressed in any case) – a scalable and reliable transport mechanism, which becomes a critical component p – has a software development/maintenance cost – committment/resources to test and then run it nationally in a reliable manner manner

• A regionalized tool allows for

– better scalability of the overall model easier c stomi ation – easier customization – use of native language (where relevant e.g. Helpdesk) – persistent storage of national data (e.g. accounting URs) – directly responsible of the service availability – one technical reference solution for all, instead of independent development efforts

EGEE-III INFSO-RI-222667

Tool Regionalization | Transition meeting, Amsterdam 03 March 2010

13

[courtesyTF : https://www.egi.eu/indico/contributionDisplay.py?contribId=12&sessionId=3&confId=1]