Jozef Cernak, Marek Kocan, Eva Cernakova (P. J. Safarik University - - PowerPoint PPT Presentation

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

ARC tools for revision and nightly functional tests

Jozef Cernak,

Marek Kocan, Eva Cernakova

(P. J. Safarik University in Kosice, Kosice, Slovak Republic)

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Testing strategy in ARC
• ARC-EMI testing tools

– Build testing

• Nightly (every night)
• Revision (after each commit of developers)

– Automatic functional testing of nightly builds

• Future plans

– Increasing number of automatic functional tests – Automatic performance testing

• Conclusion

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 2

Outline

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Several roles in project: developers, testers and ARC release manager
• Adopted tests:

– Unit tests(developers) – Build tests (testers)

• Revision (Kosice)
• Nightly (Copenhagen)
• EMI-RC (ETICS)

– Functional tests (testers, developers)

• Manual
• Automatic (Kosice)

– Regression testing (testers, developers) – Performance testing (testers)

• Manual
• Automatic in development phase

– Large scale testing (in preparation phase)

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 3

Testing practice in ARC

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

ARC-EMI testing tools

Relation between ARC and EMI testing tools

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 4

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

Block diagram of automatic revision and Functional tools

Simple database server with PHP and Python scripts

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 5

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Infrastructure (local cluster)

– 2 SUN 2200M2 servers (2x4GB RAM, 250GB disk, AMD 64 bits) – 2 desktops (2x 1GB RAM, 500GB disk 32 bits) – 1 MAC mini (2GB RAM, 160GB, 32 bits)

• Software

– MySQL, PHP, Python – Code is free

• Distributed data processing

– SVN

Technical details

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 6

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Features

– Supported are several platforms of OS:

• CentOS (32,64),
• Debian(32,64),
• and MAX OS (64)

– Detailed identification of errors during building process

• Autogen
• Configuration
• make
• make dist
• Build

Revision automatic testing

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 7

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

– User friendly interface to define search criteria

• Daily
• Weakly
• Monthly
• Custom

– Revision number, etc (see next examples)

• Direct connection with SVN
• Warning messages for developers if code is broken

– CCCC metric – Quick mechanisms to identify breaking event

…continued.

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 8

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

Example of www interface for automatic testing tools

Home page: http://arc-emi.grid.upjs.sk/ (Revision database is slow)

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 9

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

Several criteria to search results of revision testing

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 10

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

Example of the results of daily revisions

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 11

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Features

– Automatic download code from SVN based on several criteria:

• trunk,
• revision code,
• nightly builds

– Automatic build – Automatic deployment – User friendly interface for:

• submitting proposals of test cases
• to search the results of functional tests

– Tests are grouped into two main groups:

• functional tests of server,
• functional tests of client.

Automatic functional tests

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 12

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

Interface to enter search criteria

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 13

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

The results of automatic functional tests

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 14

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• client_arcsub_to_arex_gridmap_simplejob_jsdl

– Tries to submit a simple JSDL job to a grid-map secured A-REX service.

• client_arcsub_to_grid-manager_gridmap_simplejob_jsdl

– Tries to submit a simple JSDL job to a grid-map secured ARC0 CE.

• client_arcsub_to_GIIS_simplejob_jsdl

– Submission of a simple JSDL job to a GIIS service.

• client_arcsub_to_ISIS_simplejob_jsdl

– Submission of a simple JSDL job to an ISIS cloud.

• client_arcsub_random_broker_isis_simplejob_jsdl

– Submission of a simple JSDL job to an ISIS cloud using random broker.

• client_arcsub_random_broker_giis_simplejob_jsdl

– Submission of a simple JSDL job to an GIIS cloud using random broker.

• client_arcsub_fastestqueue_broker_isis_simplejob_jsdl

– Submission of a simple JSDL job to an ISIS cloud using fastest queue broker.

• client_arcsub_fastestqueue_broker_giis_simplejob_jsdl

– Submission of a simple JSDL job to a GIIS service using fastest queue broker.

• Failed

– There was some error before we managed to start testing. The error could have been in code retrieval, installation or somewhere else.

Functional test scenaria

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 15

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Requirements:

– several servers and clients – Monitoring of several distributed processes:

• resources:

– Memory – CPU – Network usage

• Performance

– Scalability – Number of concurrent requests

• Reliability of services:

– Ratio between successful tasks and total tasks. Examples of tasks: job submission, file transfer and etc.

Automatic performance tests

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 16

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Proposed infrastructure:

– Grid servers and clients, we plan to use existing infrastructure – Communication layer

• XML-RPC client-server architecture

– Central database server – Common Python library on each server and client (for example start stop service) – Central control program

… continued.

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 17

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Automatic testing covers:

– Build process (ARC, ETICS), – Functional, – Performance tests (in progress).

• Utilization of automatic test tools

increases probability to find weak part of the software. On the other hand, manual tests are important.

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 18

Conclusions

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

• Testing in ARC

– http://wiki.nordugrid.org/index.php/Testing

• Revision tests

– http://download.nordugrid.org/builds/ – http://arc-emi.grid.upjs.sk/revisionTests.php

• Functional tests

– http://arc-emi.grid.upjs.sk/functionalTests.php

• Code:

– http://svn.nordugrid.org/trac/workarea/browser/ARCT estScriptshttp://svn.nordugrid.org/trac/workarea/bro wser/ARCTestScripts

References

Use Insert Header & Footer to set this field Use Insert Header & Footer to set this field 19

E M I I N F S O

• R

I

• 2

6 1 6 1 E M I I N

Thank you

Use Insert Header & Footer to set this field 20 Use Insert Header & Footer to set this field

EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611