Automatic testing and certification procedure for IGI products in - - PowerPoint PPT Presentation

Automatic testing and certification procedure for IGI products in the EMI era and beyond Sara Bertocco INFN Padova

• n behalf of

IGI Release Team

EGI Community Forum 2013 Manchester, 10 April 2013

10 March 2013 EGI Community Forum 2013 - Manchester 2

IGI

Italian Grid Infrastructure

10 March 2013 EGI Community Forum 2013 - Manchester 2

IGI

Italian Grid Infrastructure

Italian Grid Initiative is a special project of Italian National Institute

• f Nuclear Physics (INFN).

It supports the collaboration between Regional Grids, Compute and Storage Centers, Research Institutions, Centers of Excellence to provide the Italian Grid Infrastructure. The Italian Grid Infrastructure (IGI) services enable and support the access and sharing of data and compute facilities in Italy and

• worldwide. It guarantees and

supports common services for e- research, common middleware releases, general guidelines for policies and standards.

10 March 2013 EGI Community Forum 2013 - Manchester 3

INFN-IGI Products

IGI products included in EMI releases:

CREAM CE (Computing Resource Execution and Management - Computing Element) CREAM-UI (CREAM User Interface) LRMS Modules (LSF, Torque, GE, SLURM) CEMON (Computing Element Monitor) Glite-CLUSTER StoRM SE (Storage Resource Manager Storage Element) StoRM UI (StoRM User Interface) WMS (Workload Management System) WMS-UI (WMS User Interface) VOMS VOMS-admin Argus PAP WNoDES

IGI products specific of IGI release:

• DGAS (DataGrid Accounting System)

HLR (Home Location register) GrelC GFAL JAVA APIs Secure Storage System Others

10 March 2013 EGI Community Forum 2013 - Manchester 4

EMI Release Cycle

10 March 2013 EGI Community Forum 2013 - Manchester 5

EMI Certification Process

10 March 2013 EGI Community Forum 2013 - Manchester 6

Goal:

Complete testing automation

Requirements Analyze Implement Design

Software Development Build&Integration Jenkins Repositories Git svn Install & config Software testing RobotFramework Puppet

10 March 2013 EGI Community Forum 2013 - Manchester

Activities: Testbed Topology

Padova Testbed

3 UI : - EMI1

• EMI 2
• gLite 3.2

3 UI : - EMI1

• EMI 2
• EMI 2 (SL5)

3 VOMS 3 CREAM-CE Torque 1 CREAM-CE SGE 8 WNs + 4 WNodes 1 BDII Top 2 BDII Site

CNAF Bologna Testbed

3 StoRM 5 WMS : - EMI1

• EMI 2
• EMI 3

2 ARGUS 6 CREAM-CE Torque 7 CREAM-CE LSF 1 CREAM-CE SLURM 20 WNs 1 BDII Site 2 HLR Server 1 ARGUS 2 StoRM 7

10 March 2013 EGI Community Forum 2013 - Manchester 8

Activities: Testbed Management

For each physical or virtual machine DHCP PXE Server DHCP PXE Server

O.S. Installation

Redahat Scientific Linux SL5 Redahat Scientific Linux SL6 Centos 5 or 6 Debian

Middleware Installation

Puppet Server Puppet Server

• ntp
• Egi trustanchor (lcg CA)
• Emi-testers-vo
• emi-prodrepo
• emi-rcrepo

Module Puppet

• emi2-argus-deploy
• emi-wmstesting-deploy
• emi-creamce-deploy

Generic Specific - Composed

Log creation

After the puppet modules execution. Two log files will be produced Installation Log Configuration Log

10 March 2013 EGI Community Forum 2013 - Manchester 9

Goal:

Complete testing automation

Requirements Analyze Implement Design

Software Development Build&Integration Jenkins Repositories Git svn Install & config Software testing RobotFramework Puppet

10 March 2013 EGI Community Forum 2013 - Manchester 10

Continuous Integration

We have all the pieces: Source Code Management: GITHUB, SVN CI Server(s): Jenkins Repositories: Nexus Test Automation Framework: RobotFramewrok

10 March 2013 EGI Community Forum 2013 - Manchester

Real examples

10 March 2013 EGI Community Forum 2013 - Manchester 12

Testing: Robot Framework

Robot Framework: test automation framework

Main features:

can be extended by test libraries Python or Java users can create reusable new keywords from existing ones easy-to-use tabular syntax for creating test cases in a uniform way keyword-driven, data-driven and behavior-driven (BDD) approaches easy-to-read reports and logs in HTML format. tagging to categorize and select test cases to be executed. simple library API for creating customized test libraries. command line interface XML based outputs for integration into existing build infrastructure

10 March 2013 EGI Community Forum 2013 - Manchester 13

Testing: Testsuites Management

CREAM-BLAH testsuite WMS testsuite VOMS testsuite StoRM testsuite Glite-UI testsuite How to write a robot framework integrable test-suite

10 March 2013 EGI Community Forum 2013 - Manchester 14

References

Our e-mail IGI Release Team <grid-release@lists.italiangrid.it> Our daily work pages https://wiki.italiangrid.it/twiki/bin/view/IGIRelease/WebHome https://wiki.italiangrid.it/twiki/bin/view/IGIRelease/CertTestBed The testsuites references https://github.com/italiangrid/voms-admin-legacy-testsuite https://github.com/italiangrid/voms-testsuite https://github.com/italiangrid/WMS-Test-Suite https://github.com/italiangrid/storm-testsuite http://nkua-emi.posterous.com/ https://twiki.cern.ch/twiki/bin/view/EMI/CREAMRobotESTests https://twiki.cern.ch/twiki/bin/view/EMI/CREAMStressTests https://wiki.italiangrid.it/twiki/bin/view/IGIRelease/Cream_blah_testsuites https://github.com/italiangrid/cream_blah_testsuites https://github.com/italiangrid/cream_performance_scalability_testshttps: //github.com/danido/EMITestbedTestsuite

10 March 2013 EGI Community Forum 2013 - Manchester 15

Questions ? Thanks !