Interfacing AliEn and ARC Interfacing AliEn and ARC for a - - PowerPoint PPT Presentation

▶
interfacing alien and arc interfacing alien and arc for a
SMART_READER_LITE
LIVE PREVIEW

Interfacing AliEn and ARC Interfacing AliEn and ARC for a - - PowerPoint PPT Presentation

Jan 25, 2023 659 likes •1.03k views

Interfacing AliEn and ARC Interfacing AliEn and ARC for a distributed Nordic T1 for a distributed Nordic T1 Why such an interface Why such an interface ARC and AliEn ARC and AliEn The Interface The Interface Tests Tests

slide-1
SLIDE 1

1 1 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Interfacing AliEn and ARC Interfacing AliEn and ARC for a distributed Nordic T1 for a distributed Nordic T1

  • Why such an interface

Why such an interface

  • ARC and AliEn

ARC and AliEn

  • The Interface

The Interface

  • Tests

Tests

  • Conclusion

Conclusion

slide-2
SLIDE 2

2 2 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Principle Principle

NDGF T1 9 sites

slide-3
SLIDE 3

3 3 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Benefits Benefits

  • Operation and Accounting simplification

Operation and Accounting simplification

– For ALICE: only 1 site

For ALICE: only 1 site

– For NDGF: only 1 middleware

For NDGF: only 1 middleware

  • Flexibility

Flexibility

– Resources can be moved within the Nordic countries

Resources can be moved within the Nordic countries

– Sites can be easily added at a regional level

Sites can be easily added at a regional level

slide-4
SLIDE 4

4 4 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Requirements of Requirements of an AliEn-ARC interface an AliEn-ARC interface

  • From AliEn

From AliEn

– Works like a normal site

Works like a normal site

– Simple module in the code (easy to maintain)

Simple module in the code (easy to maintain)

  • From NorduGrid

From NorduGrid

– Minimum configuration on sites

Minimum configuration on sites (No knowledge of AliEn for site admin) (No knowledge of AliEn for site admin)

– Not too much security exception (firewall, privileges...)

Not too much security exception (firewall, privileges...)

slide-5
SLIDE 5

5 5 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Grid Manager Gatekeeper GridFTP

RSL RSL

Front-end

Cluster

✓

ARC middleware ARC middleware

slide-6
SLIDE 6

6 6 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

ARC middleware ARC middleware

  • Very active development

Very active development

– Supported by EU via KnowARC project in 8 countries

Supported by EU via KnowARC project in 8 countries

Denmark Sweden Norway Hungary Germany Switzerland Slovakia UK Denmark Sweden Norway Hungary Germany Switzerland Slovakia UK

– Involved in European Middleware Initiative (EMI)

Involved in European Middleware Initiative (EMI)

  • Widely deployed

Widely deployed

– 70+ sites over 12 countries

70+ sites over 12 countries

– Almost 70 000 CPUs

Almost 70 000 CPUs

  • Package Management

Package Management

– Packages installed by hand by system administartors

Packages installed by hand by system administartors

– Advertised in Information System with RunTime Environment

Advertised in Information System with RunTime Environment

slide-7
SLIDE 7

7 7 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

AliEn Grid Solution AliEn Grid Solution

7

Central Services

Deployed for: ISS

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA

…

Opt. CE Pack Man Mon ALISA JA xrootd CM CE Pack Man Mon ALISA JA xrootd CM CE Pack Man Mon ALISA JA xrootd CM

Data transfer

Site Services

~ 70 in ALICE Oct 24, 2008 Pablo.Saiz@cern.ch

slide-8
SLIDE 8

8 8 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

AliEn AliEn

  • Specific to the ALICE experiment, also used by e.g.

Specific to the ALICE experiment, also used by e.g. FAIR (PandaGrid) FAIR (PandaGrid)

  • Development and services centralised at CERN

Development and services centralised at CERN

  • 40 000 CPUs on over 80 sites

40 000 CPUs on over 80 sites

  • Automatic installation of required packages (within

Automatic installation of required packages (within ALICE computing framework) ALICE computing framework)

  • Jobs pulled by pilots job

Jobs pulled by pilots job

slide-9
SLIDE 9

9 9 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

How the interface actually works How the interface actually works

slide-10
SLIDE 10

10 10 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA Opt.

CE Pack Man Mon ALISA xrootd CM

Interface between AliEn and ARC Interface between AliEn and ARC

PackMan checks for available packages on the ARC sites (LDAP query) ARC site ARC site ARC site

slide-11
SLIDE 11

11 11 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA Opt.

CE Pack Man Mon ALISA xrootd CM

Interface between AliEn and ARC Interface between AliEn and ARC

The Computing Element checks if there are jobs fitting its capacities

slide-12
SLIDE 12

12 12 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA Opt.

CE Pack Man Mon ALISA xrootd CM

Interface between AliEn and ARC Interface between AliEn and ARC

If a package is missing, PackMan sends installation jobs with ARC (“alien login -exec packman install ...”) The job also creates a RTE script to advertise the package in APPS/HEP/ALICE/ If the installation job fails, a list of uninstalled packages is created. PackMan will retry the installation regularly (exponential delay)

slide-13
SLIDE 13

13 13 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA Opt.

CE Pack Man Mon ALISA xrootd CM

Interface between AliEn and ARC Interface between AliEn and ARC

JA JA JA

The CE sends Job Agents to the sites via ARC. The RTE requirement guaranties that they will

  • nly go to sites with the appropriate packages.
slide-14
SLIDE 14

14 14 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA Opt.

CE Pack Man Mon ALISA xrootd CM

Interface between AliEn and ARC Interface between AliEn and ARC

JA JA JA

The JAs communicate with the Cluster Monitor via SOAP as on normal site. Requires only outbound connectivity from the nodes. The JAs are not allowed to require installation from the PackMan.

slide-15
SLIDE 15

15 15 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Implementation and Installation Implementation and Installation

  • AliEn:

AliEn:

– 2 Perl modules (Submission and PackMan)

2 Perl modules (Submission and PackMan)

– Few LDAP parameters

Few LDAP parameters

  • ARC

ARC

– Install AliEn (WN installation)

Install AliEn (WN installation)

– Create RTE script (environment variables)

Create RTE script (environment variables)

– Allow user with installation privileges

Allow user with installation privileges (may need extra plugin for special configuration) (may need extra plugin for special configuration)

slide-16
SLIDE 16

16 16 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Test Bed in Lund Test Bed in Lund

Aalborg Aalborg Central Services Central Services + VO-box + VO-box LUNARC LUNARC

  • Full Central Services

Full Central Services

  • VO-box on isolated computer

VO-box on isolated computer

  • Access to 2 ARC sites with special configuration

Access to 2 ARC sites with special configuration

– LUNARC, Aalborg

LUNARC, Aalborg

slide-17
SLIDE 17

17 17 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Test and Result Test and Result

  • Prototype Tested in Lund

Prototype Tested in Lund

– Submitted jobs requiring ROOT

Submitted jobs requiring ROOT

– ROOT installed, job runs on the two ARC sites

ROOT installed, job runs on the two ARC sites

– Output retrieved

Output retrieved

  • Minor problem with package installation on overloaded

Minor problem with package installation on overloaded site site

– Minimised by checking before retry

Minimised by checking before retry

– Installer user should have higher priority

Installer user should have higher priority

slide-18
SLIDE 18

18 18 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Bittorrent Alternative Bittorrent Alternative

  • Another solution for package management using

Another solution for package management using bittorent exists bittorent exists

  • Should work out of the box for a distributed AliEn site

Should work out of the box for a distributed AliEn site

  • Has been largely tested, but still has issues with sites

Has been largely tested, but still has issues with sites security configurations security configurations

slide-19
SLIDE 19

19 19 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Conclusion Conclusion

  • Creating a distributed AliEn site is possible

Creating a distributed AliEn site is possible

  • A prototype working with ARC is operational

A prototype working with ARC is operational

  • A few other features could be added

A few other features could be added

  • Exercise could be extended to other middleware (gLite,

Exercise could be extended to other middleware (gLite, UNICORE) UNICORE)

  • Unified Middleware Distribution (UMD)

Unified Middleware Distribution (UMD)

slide-20
SLIDE 20

20 20 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

backup backup

slide-21
SLIDE 21

21 21 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Features Still Missing Features Still Missing

  • AliEn auto update (triggered by MonALISA)

AliEn auto update (triggered by MonALISA)

  • Personal packages

Personal packages

  • Large scale test

Large scale test

slide-22
SLIDE 22

22 22 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

What is done by the interface What is done by the interface

slide-23
SLIDE 23

23 23 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Job Submission and Monitoring Job Submission and Monitoring

  • Works like any other submission system

Works like any other submission system

  • Had to be solved: communication to the JA (through

Had to be solved: communication to the JA (through firewalls) firewalls)

– Job Agent submission (“ngsub”)

Job Agent submission (“ngsub”)

– Job monitoring (“spy”): done with ARC (“ngcp”) if

Job monitoring (“spy”): done with ARC (“ngcp”) if allowed by ARC site allowed by ARC site

– Kill job => kill corresponding JA (“ngkill”)

Kill job => kill corresponding JA (“ngkill”)

  • The other communications (from the JA) only require

The other communications (from the JA) only require

  • utbound connectivity
  • utbound connectivity
slide-24
SLIDE 24

24 24 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Package Management Package Management

  • Install a package

Install a package

– Triggered by CE

Triggered by CE

– Send an ARC job (with special privileges)

Send an ARC job (with special privileges)

– Run 'alien packman' locally on the WN

Run 'alien packman' locally on the WN => Same as normal AliEn installation => Same as normal AliEn installation

  • Get a list of installed packages

Get a list of installed packages

– Installation job creates an ARC runtime environment

Installation job creates an ARC runtime environment

– VO-box can query the ARC information system

VO-box can query the ARC information system

slide-25
SLIDE 25

25 25 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Package Management: Package Management: Potential issues Potential issues

  • Job Agent should never call PackMan

Job Agent should never call PackMan (Risk of infinite loop)

(Risk of infinite loop)

– Special parameter in LDAP (WnForbidInstall)

Special parameter in LDAP (WnForbidInstall)

  • What if one installation job fails?

What if one installation job fails?

– Put RTE requirement in JA submission

Put RTE requirement in JA submission

– Keep table of sites missing package

Keep table of sites missing package Retry to install next time regularly (exponential delay) Retry to install next time regularly (exponential delay)

  • What if the installation fails

What if the installation fails

– How do we make sure the installation is done

How do we make sure the installation is done properly? (How do we correct?) properly? (How do we correct?)

– Same problem on any other site...

Same problem on any other site...

slide-26
SLIDE 26

26 26 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Grid Manager Gatekeeper GridFTP

Front-end

Cluster

ARC middleware ARC middleware

slide-27
SLIDE 27

27 27 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Grid Manager Gatekeeper GridFTP

RSL RSL

Front-end

Cluster

ARC middleware ARC middleware

Ask data location Ask nodes availability

slide-28
SLIDE 28

28 28 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Grid Manager Gatekeeper GridFTP

RSL

Front-end

Cluster

ARC middleware ARC middleware

Send job description + data (GridFTP)

slide-29
SLIDE 29

29 29 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Grid Manager Gatekeeper GridFTP

RSL

Front-end

Cluster

ARC middleware ARC middleware

Run the job

  • n the cluster
slide-30
SLIDE 30

30 30 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Grid Manager Gatekeeper GridFTP

RSL

Front-end

Cluster

ARC middleware ARC middleware

Copy the data to the SE Register the data in the catalog

slide-31
SLIDE 31

31 31 Interfacing AliEn and ARC for a distributed Nordic T1 Interfacing AliEn and ARC for a distributed Nordic T1 Philippe Gros – Lund University Philippe Gros – Lund University

11/03/10 11/03/10

Grid Manager Gatekeeper GridFTP

RSL RSL

Front-end

Cluster

✓

ARC middleware ARC middleware

slide-32
SLIDE 32

Central Services

Deployed for: ISS + one per

build server

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA

…

Site Services

~ 70 in ALICE Oct 24, 2008 32 Pablo.Saiz@cern.ch

Opt. SE CE Pack Man FTD Mon ALISA JA xrootd CM SE CE Pack Man FTD Mon ALISA JA xrootd CM SE CE Pack Man FTD Mon ALISA JA xrootd CM

AliEn Structure

slide-33
SLIDE 33

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker Authen Proxy IS Logger LDAP Mon ALISA

…

33

Opt. SE CE Pack Man FTD Mon ALISA JA xrootd CM SE CE Pack Man FTD Mon ALISA JA CM SE CE Pack Man FTD Mon ALISA JA xrootd CM

AliEn Structure

API xrootd

slide-34
SLIDE 34

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA

…

Opt. SE CE Pack Man FTD Mon ALISA xrootd CM SE CE Pack Man FTD Mon ALISA xrootd CM SE CE Pack Man FTD Mon ALISA xrootd CM

AliEn Structure

slide-35
SLIDE 35

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA

…

Opt. SE CE Pack Man FTD Mon ALISA xrootd CM JA SE CE Pack Man FTD Mon ALISA xrootd CM JA CE Pack Man FTD Mon ALISA xrootd CM JA

AliEn Structure

slide-36
SLIDE 36

File Catalogue Task Queue Transfers Broker Manager Opt. Opt. Manager Broker API Authen Proxy IS Logger LDAP Mon ALISA

…

Opt. SE CE Pack Man FTD Mon ALISA xrootd CM JA SE CE Pack Man FTD Mon ALISA xrootd CM JA CE Pack Man FTD Mon ALISA xrootd CM JA

AliEn Structure