gLite Installation Course Volker Bge, Christopher Jung, Yves Kemp - - PowerPoint PPT Presentation

▶
glite installation course
SMART_READER_LITE
LIVE PREVIEW

gLite Installation Course Volker Bge, Christopher Jung, Yves Kemp - - PowerPoint PPT Presentation

Sep 08, 2023 304 likes •659 views

Enabling Grids for E-sciencE gLite Installation Course Volker Bge, Christopher Jung, Yves Kemp Institut fr Experimentelle Kernphysik Institut fr Wissenschaftliches Rechnen Universitt Karlsruhe Forschungszentrum Karlsruhe

slide-1
SLIDE 1

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

EGEE and gLite are registered trademarks

www.eu-egee.org

gLite Installation Course

Volker Büge, Christopher Jung, Yves Kemp

Institut für Wissenschaftliches Rechnen Forschungszentrum Karlsruhe Institut für Experimentelle Kernphysik Universität Karlsruhe

slide-2
SLIDE 2

2

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

The LHC Computing Grid – Access

A job’s way through the grid:

UI JDL

Resource Resource Broker Broker Job Submission Job Submission Service Service Storage Storage Element Element Computing Computing Element Element

Job Status

Replica Replica Catalogue Catalogue

DataSets info Job Submit Event Job Query Job Status JDL Input “sandbox ” I n p u t “ s a n d b

  • x

” + B r

  • k

e r I n f

  • Globus RSL

Output “sandbox ” Output “sandbox ” Job Status Publish Expanded JDL SE & CE info

Logging & Logging & Book Book-

  • keeping

keeping Author. &Authen.

g r i d

  • p

r

  • x

y

  • i

n i t

Information Information Service Service

UI JDL UI JDL

Resource Resource Broker Broker Resource Resource Broker Broker Job Submission Job Submission Service Service Job Submission Job Submission Service Service Storage Storage Element Element Storage Storage Element Element Computing Computing Element Element Computing Computing Element Element

Job Status Job Status

Replica Replica Catalogue Catalogue Replica Replica Catalogue Catalogue

DataSets info DataSets info Job Submit Event Job Submit Event Job Query Job Query Job Status Job Status JDL Input “sandbox ” JDL Input “sandbox ” I n p u t “ s a n d b

  • x

” + B r

  • k

e r I n f

  • I

n p u t “ s a n d b

  • x

” + B r

  • k

e r I n f

  • Globus RSL

Globus RSL Output “sandbox ” Output “sandbox ” Output “sandbox ” Output “sandbox ” Job Status Job Status Publish Publish Expanded JDL Expanded JDL SE & CE info SE & CE info

Logging & Logging & Book Book-

  • keeping

keeping Logging & Logging & Book Book-

  • keeping

keeping Author. &Authen. Author. &Authen.

g r i d

  • p

r

  • x

y

  • i

n i t g r i d

  • p

r

  • x

y

  • i

n i t

Information Information Service Service Information Information Service Service

slide-3
SLIDE 3

3

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

gLite - Basics

The basic concepts:

Authorization: What may I do ?

  • certain permissions, duties etc.
  • “equivalent” to a visa or access list
  • Virtual Organisation Membership Service

User can adapt several different roles, e.g. software manager, normal user, …

Your certificate is registered in the VO dgtest as normal user

Authentication: Who am I ?

  • concept of certificates
  • “equivalent” to a passport, ID card etc.

You can find your school certificate on iwrgks-17-5.fzk.de

slide-4
SLIDE 4

4

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

The gLite Middleware I

VOMS Server

  • registry office of a VO
  • contains all users and

their roles within a VO dgrid-voms.fzk.de

https://dgrid-voms.fzk.de:8443/voms/dgtest/

LCG File Catalogue

  • global file index for a

Virtual Organisation

iwrlfc.fzk.de

Information Service

  • collects and publishes

information on resources connected to the LCG

iwrbdii.fzk.de

Resource Broker

  • “intelligence” of the grid
  • distributes incoming job

requests to matching resources

iwrrb.fzk.de Grid-wide services:

In addition we provide an working User Interface for you: iwrgks-17-5.fzk.de You can login on this machine with your schools account

slide-5
SLIDE 5

5

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

The gLite Middleware II

Site-wide services:

  • User Interface

– access point for the user to the grid

  • Computing Element

– portal to the local batch system of a site

  • Worker Node

– pbs grid worker node receiving jobs from CE

  • Storage Element

–

  • ffering disk space to a VO

– portal to the local storage

  • Monitoring Box

– collects and publishes information on grid jobs executed at a site

slide-6
SLIDE 6

6

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

The gLite Middleware III

Your job: Create your own site within the dgtest VO!

Therefore, you have to install (replace X with your group number):

  • User Interface

– iwrgks-X-5.fzk.de

  • Computing Element

– iwrgks-X-1.fzk.de

  • Worker Node

– iwrgks-X-4.fzk.de

  • Storage Element

– iwrgks-X-2.fzk.de

  • Monitoring Box

– iwrgks-X-3.fzk.de

Please install the nodes in this order and keep in mind that your virtual machines are hosted

  • n one physical

machine!

slide-7
SLIDE 7

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

EGEE and gLite are registered trademarks

www.eu-egee.org

XEN

The technical infrastructure of the course Access to Your machines

slide-8
SLIDE 8

8

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Installation on machines:

  • Needed resources:

– 16 groups – Every group installs 5 machines – A total of 80 machines needed – No high-performance machines needed

  • Best solution: Virtualization

– For a detailed introduction on Virtualization, please recall the slides from Volker’s talk yesterday:-) – We use Xen in this course

  • Infrastructure

– 20 Dual PIII with 1.2 GHz, 1 GB RAM and 30 GB free on disk – Each group works on his own physical host, but:

  • Five preinstalled Virtual Machines running
  • Use them as if they were normal machines
  • You do not have to care about the underlying virtualization
  • EXCEPT: Only use ONE MACHINE AT A TIME! All your machines use the

same hardware!

slide-9
SLIDE 9

9

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Installation details

  • The five machines provided to you have a minimal

linux installation: ScientificLinux Cern 3.0.6

  • What is ScientificLinux?

– Science community effort – Basically a recompiled RedHat Enterprise Server (in our case version 3) – Have a binary compatible distribution without the licence fees of a real RedHat Enterprise Server – https://www.scientificlinux.org/ For the generic SL – http://linux.web.cern.ch/linux/scientific3/ For the Cern flavor

  • Why ScientificLinux (Cern (3.0.6))?

– gLite middleware recommends this distribution – Possible to install on other systems, not tested extensively – Only RPMs provided for this OS

slide-10
SLIDE 10

10

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Organization:

  • Each group is composed of two or three people
  • Best: Everyone has a laptop, preferably Linux

– Minimum is an ssh client! Google for putty if your Windows installation does not already have one!

  • At least one person should have some knowledge of

Linux systems

– Command line, ssh&scp, linux editors…

  • You will get login information to your dedicated

machines from us

  • You will need the information of your User Certificate

given to you with the registration information

– If you lost them, ask Yves

slide-11
SLIDE 11

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

EGEE and gLite are registered trademarks

www.eu-egee.org

YAIM

Yet Another Installation Method Yet Another Installation Manager

slide-12
SLIDE 12

12

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

YAIM, RPMs and apt

  • ScientificLinux is an RPM based distro

– Packages are provided in RPM form

  • General installation tool/wrapper apt

– apt known from Debian helps resolving dependencies between packages, automatically installs additional packages – Works like Debian:

  • apt-cache search XY
  • agt-get install XYZ ABC
  • Except that the packages are RPMs not DEB
  • YAIM is a wrapper for apt to perform a complete

installation of a site

– You first have to install YAIM (see instructions) – http://www.cern.ch/grid-deployment/gis/yaim/

slide-13
SLIDE 13

13

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

YAIM: Installation utility

  • YAIM has several installation targets (meta-packages):

– Complete list can be found at http://grid- deployment.web.cern.ch/grid- deployment/documentation/LCG2- Manual-Install/LCG2-Manual-Install.html (available meta- packages) – Examples:

  • LCG Computing Element (with Torque): lcg-CE_torque
  • Storage Element on local disk: glite-SE_classic
  • Monitoring-Box: glite-MON
  • Worker node: glite-WN glite-torque-client-config
  • User Interface: glite-UI
  • Install using

– /opt/glite/yaim/scripts/install_node <configuration> <meta-package>

slide-14
SLIDE 14

14

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

YAIM is more: configuration utility

  • YAIM is also a configuration utility:

– You have to adapt a few configuration files, the most important is site-info.def – You specify the configuration target (different from installation target)

  • Compute Element: CE_torque
  • Storage Element: SE_classic
  • Monitoring Box: MON
  • Worker Node: WN_torque
  • User Interface: UI

– Everything is configured for you

Configure using

– /opt/glite/yaim/scripts/configure_node <configuration> <meta-package>

slide-15
SLIDE 15

15

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Configuration files

  • During this course, you will adapt four files
  • site-info.def

– Contains description of your site in general – Describes the components you use at your site – Definition and infos about the Virtual Organizations you support

  • users.conf

– List of generic mapping users the configuration should create – Might be left empty if users are imported through LDAP&NFS e.g

  • groups.conf

– Infos about different user groups – Details accounts for special roles (Softwaremanager:SGM,….)

  • wn-list.conf

– Lists all worker nodes connected to the CE – Left empty if the CE is not the PBS server

slide-16
SLIDE 16

16

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Location of these files

  • Example files are contained in the directory

/opt/glite/yaim/examples/

  • You can work in this directory (although it is called

examples)

  • Attention:

– A source of mistakes are configuration files with different content

  • n different nodes

– Make sure you distribute a file to all other nodes after you made a change in it! – (Hint for later at home: Either export this directory via NFS from a central location or use CVS)

slide-17
SLIDE 17

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

EGEE and gLite are registered trademarks

www.eu-egee.org

Information services

slide-18
SLIDE 18

18

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

GRISs, local BDII and BDII

Each site can run a BDII. It collects the information given by the local BDIIs At each site, a *local* BDII collects the information given by the GRISs

Local GRISes run on CEs and SEs at each site and report

dynamic and static information Abbreviations:

BDII: Berkeley DataBase Information Index GIIS: Grid Index Information Server GRIS: Grid Resource Information Server

Slide from Diego Scardaci

slide-19
SLIDE 19

19

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

The IS in gLite

RB Local GRIS SE Local GRIS CE Local GRIS

BDII-A BDII-B

SE Local GRIS SE Local GRIS CE Local GRIS SE Local GRIS

BDII-C

CE Local GRIS

CE Site BDII

CE Local GRIS

CE Site BDII

CE Local GRIS

CE Site BDII

Site 1 Site 2 Site 3

Slide from Diego Scardaci

slide-20
SLIDE 20

20

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Publishing additional CE& SE

SE Local GRIS CE Local GRIS

CE Site BDII

Site 1

On your machines:

  • Your CE (iwrgks-X-1.fzk.de) has to be

entered into the VOs BDII in order to publish information to the VO

  • It is publishing the information on all

nodes listed in the file /opt/bdii/etc/bdii-update.conf

  • If you want to provide information about

more than one CE and SE on your site, just add the contact strings

  • Do not forget to restart the bdii after

changes /etc/init.d/bdii restart

slide-21
SLIDE 21

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

EGEE and gLite are registered trademarks

www.eu-egee.org

Certificates:

How they work? How to get them?

slide-22
SLIDE 22

22

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Review of the basics

  • The Grid uses public key or asymmetric encryption for authentication of

users, resources and services.

  • According to the basics of public-key cryptography, each resources on the

GRID has a key pair, a public and a private key.

  • The public key is made public while the private key must be kept secret.
  • Encryption is performed using the public key while decryption and digital

signature is performed with the private key.

  • It is important to notice that generating a key pair does not automatically

provide you access to the Grid resources.

  • A trusted authority of the Grid, called the Certificate Authority (CA) needs to

sign your public key, this way confirming your identity. This signing procedure of the CA is often referred as “issuing a certificate”.

slide-23
SLIDE 23

23

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Files

  • userkey.pem / hostkey.pem:

– private key encrypted with your password / without password

  • usercert.pem / hostcert.pem:

– Certificate file, contains the public key together with additional important informations such as the subject name of the holder /full hostname of the certificate, the name of the signing CA, and the digital signature of the CA.

  • The important role of the CA is to establish a trustful

connection between the identity of the user/host and the public key in the certificate file.

  • The digital signature of the CA in the user's/host’s

certificate file officially declares that the public key in the file belongs to the specific user / specific hostname

slide-24
SLIDE 24

24

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Request

  • In order to obtain a valid passport to the Grid you need

to create a key pair and submit your public key to the CA (this process is called as a certificate request) for a signature.

  • The CA will follow its certificate policy and upon

successful evaluation of your request your public key will be signed and posted back to you.

  • To do so: On a configured UI:

grid-cert-request -dir destdir -host mynewressource.mysite.de

  • Three files:
  • rw-r--r-- … 0 … hostcert.pem For the future certificate
  • rw-r--r-- … 1255 … hostcert_request.pem The request to be sent to the CA
  • r-------- … 887 … hostkey.pem The private key
slide-25
SLIDE 25

25

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Closer look at one host certificate

Signature Algorithm: sha1WithRSAEncryption Issuer: C=DE, O=GermanGrid, CN=GridKa-CA Validity Not Before: Aug 1 15:07:24 2006 GMT Not After : Aug 31 15:07:24 2007 GMT Subject: O=GermanGrid, OU=FZK, CN=host/iwrgks-1-1.fzk.de Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): 00:a8:5f:44:2a:86:31:12:eb:1f:63:f2:91:6c:25: X509v3 extensions: X509v3 Subject Key Identifier: 1C:1E:C8:16:BC:E7:CA:51:B3:31:1C:06:98:A8:22:B5:9C:8F:93:AF X509v3 Authority Key Identifier:

keyid:C6:75:C9:28:AC:D1:0B:FC:3C:FF:B9:B5:1E:D3:5F:3B:80:62

DirName:/C=DE/O=GermanGrid/CN=GridKa-CA X509v3 Subject Alternative Name: DNS:iwrgks-1-1.fzk.de X509v3 Issuer Alternative Name: email:gridka-ca@iwr.fzk.de X509v3 Certificate Policies: Policy: 1.3.6.1.4.1.2614.5548.1.1.1.3 Signature Algorithm: sha1WithRSAEncryption a8:32:76:dd:4a:61:5d:10:85:f7:bd:b3:2a:4e:8a:2e:dd:3d:

  • ----BEGIN CERTIFICATE-----

MIIFHjCCBAagAwIBAgICCPkwDQYJKoZIhvcNAQEFBQAwNjELMAhMCREUx

  • ----END CERTIFICATE-----
slide-26
SLIDE 26

26

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Host Certificates

  • Which resources do need certificates?:

– CE, SE, MON need to communicate over the net without user interaction – UI does not need one as the user takes over its own authentication – WN are authenticated to their PBS-Server via other mechanisms

  • Certificates bound to hostname
  • Certificates have been requested in advance

– Find them in /root/iwrgks-X-Y.fzk.de on the relevant machines

slide-27
SLIDE 27

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

EGEE and gLite are registered trademarks

www.eu-egee.org

The PBS queue

slide-28
SLIDE 28

28

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Maui/Torque

  • The default queueing system in gLite is Torque, based
  • n PBS or PBS2.0

– Often only called PBS

  • The scheduler is Maui
  • The default PBS server is the CE
  • The WN run the pbs_mom
slide-29
SLIDE 29

29

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Get the server configuration

  • On the CE: (excerpt of the output)

# qmgr -c "print server” create queue dgtest set queue dgtest queue_type = Execution set queue dgtest acl_group_enable = True set queue dgtest acl_groups = dgtest set queue dgtest enabled = True set queue dgtest started = True … set server scheduling = True set server acl_host_enable = False set server default_queue = dteam set server default_node = lcgpro …

slide-30
SLIDE 30

30

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Test nodes

  • Status of the nodes: On CE:

# pbsnodes -a ekpwn036.physik.uni-karlsruhe.de state = free np = 1 properties = lcgpro ntype = cluster status = arch=linux,uname=Linux ekpwn036.physik.uni-karlsruhe.de 2.6.16-xenU #1 SMP Thu Apr 20 16:57:21 CEST 2006 i686,sessions=? 0,nsessions=?0,nusers=0,idletime=280452,totmem=1484924kb,availmem =3559472kb,physmem=460932kb,ncpus=1,loadave=0.00,rectime=115809 8139 ekpwn037.physik.uni-karlsruhe.de state = down (Node down? Pbs Mom troubled?) np = 1 properties = lcgpro …..

slide-31
SLIDE 31

31

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Test job submission

  • Become a normal user on CE:

– su - dgtestXY – qsub -I qsub: No default queue specified – qsub -I -q dgtest [dgtest011@ekpwn043 dgtest011]$ Job submission works – echo “sleep 5” | qsub -q dgtest (check status with qstat) you should find the files STDIN.[e,o]<JOBID> in your home directory, if not: host-based communication between CE and WN is disturbed

slide-32
SLIDE 32

32

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

  • And now: Your turn…

Start opening the installation instructions at: http://www-ekp.physik.uni-karlsruhe.de/~kemp/gks06/

slide-33
SLIDE 33

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

EGEE and gLite are registered trademarks

www.eu-egee.org

Network, Firewall, Ports

slide-34
SLIDE 34

34

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Communication over Networks

  • Server listening on

ports

  • Official port list:

– Not exhaustive, only basic services

  • Worker node:

– In general no firewall rules needed: Private IP – Behind a NAT

slide-35
SLIDE 35

35

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Testing the firewall

  • Firewall not an issue during this course

– Needed ports open in FZK firewall

  • One example: IEKP at the University of Karlsruhe

– Central University firewall: Completely open for EKP gLite machines… (hard fight) – We have an own firewall between our cluster and the university network: Only opens the ports detailed on the previous slide

  • Testing the firewall(s)

– You should have access to your cluster from the outside with no firewall at remote site (at home with DSL e.g.) – ping, traceroute – Portscan (nmap e.g.): Check for differences between

  • utside/inside firewall

– telnet to a specific port – Firewall logs (if you have access to them :-))