SELECTED TOPICS IN DISTRIBUTED AND PARALLEL SYSTEMS Grid - - PowerPoint PPT Presentation

SELECTED TOPICS IN DISTRIBUTED AND PARALLEL SYSTEMS

Grid Technologies - Practical Lab

1

Setting up the Grid Environment

2

Grid Environment Setup (Lab)

3  “Install“ your certificate

 Copy XXXX.pem => ~/.globus/usercert.pem

 Setup Environment:

 Add “/usr/site/grid/globus/current/bin” to $PATH  Add “/usr/site/grid/globus/current/lib” to $LD_LIBRARY_PATH  Run “source /usr/site/grid/etc/profile.d/globus.sh”

 Should export GLOBUS_TCP_PORT_RANGE=40000,40500

 => you might wanna add everything to .bashrc

 Download CA certificate into ~/.globus

 http://ca.austriangridca.at/root.php  ”wget https://ca.austriangridca.at/certs/6e3b436b.0” => ~/.globus

 --no-check-certificate … if security error is raised

Setting up Thunderbird

4  Convert certificate and key to the PKCS12 format

 openssl pkcs12 –inkey .globus/userkey.pem –in .globus/usercert.pem –

• ut KEYNAME.p12 –export –name “John Doe, AustrianGrid“

 PKCS12: container for certificate + private key, encrypted using a

symmetric key

 Setting up Thunderbird:

 Edit>Preferences>Advanced>Certificates>View Certificates

 Your Certificate => import KEYNAME.p12  Authorities => import CA certificate (6e3b436b.0)

 Edit>Account Settings>$account$>Security

 Pick certificate for digitally signing messages

 Send a signed Test-mail to yourself, check signature (little letter)  Finally confirm CA mail by a signed response

Using the Grid (Part A)

5

Getting Started

6

 Login to the Grid  grid-proxy-init … creates a user proxy

 No remote service involved  Proxy created in e.g. /tmp/x509up_u1000  New public/private key pair, certificate signed by user

 grid-proxy-info … get proxy information  grid-proxy-destroy … delete proxy

 Corresponds to logout  Does not invalidate sub-proxies or stop running jobs (!)

CA User Proxy Proxy

sign sign sign …

Austrian Grid Parallel Machines

7

Cluster Master

• Loc. Manager

Architecture CPU karwendel karwendel.dps.uibk.ac.at SGE COW AMD Opteron 80 altix-uibk altix1.uibk.ac.at Torque SMP NUMA SGI Altix 350 16 altix-jku altix1.jku.austriangrid.at PBS SMP NUMA SGI Altix 3k 64 hydra.jku hydra.gup.uni-linz.ac.at PBS COW AMD 2000 16 lilli lilli.edvz.uni-linz.ac.at PBS SMP NUMA Altix 64 FHV blades.labs.fhv.at Maui COW Intel Xeon 12 alex alex.jku.austriangrid.at PBS Altix ICE 8200 Intel Xeon 768

http://agrid.uibk.ac.at/austriangrid/

Login to a Parallel Machine

8

 Log into grid-node: glogin

 e.g. “glogin alex.jku.austriangrid.at”  Provides a remote shell through Job Submission  Details:

http://www.gup.jku.at/research/projects/glogin

 Should only be necessary for debugging

 Observing the job queue, logs and temporary files

Using the local batch queuing system

Running Jobs

9

PBS/Torque (1)

10

 Getting System Information:  qstat … lists own active jobs  qstat –Q … lists queue information (including names)  More details shown with –f

 => see man pages  Handle Jobs

 qsub … submit a new job to the queuing system  qstat –u user job_id … track a job  qdel … kills a running job

http://www.clusterresources.com/torquedocs21/?id=torque:torque_wiki

PBS/Torque (2)

11

 Sample Job submission: "qsub job.pbs”

http://www.clusterresources.com/torquedocs21/?id=torque:torque_wiki # resources to allocate and execution time #PBS -l walltime=12:00:00,nodes=4:ppn=2 # the queue #PBS -q lva.q # stdout and stderr files #PBS -o std.out #PBS -e std.err # job name #PBS -N my_job # commands to execute cd $PBS_O_WORKDIR mpirun –np 8 a.out Job Parameters (see qsub man pages) arbitrary script + some PBS variables

SGE – Sun Grid Engine (1)

13

 Getting System Information:

 qstat … lists all active jobs  qstat –f … lists jobs an queue information  qstat -g c … queue information

 Handle Jobs

 qsub … submit a new job to the queuing system  qstat –u user –j job_id … track a job  qdel … kills a running job http://www.clusterresources.com/torquedocs21/?id=torque:torque_wiki

SGE - Sun Grid Engine (2)

14

 Sample Job submission: "qsub job.sge”

http://unix-docu.uibk.ac.at/zid/fremddoc/sge/ # resources to allocate #$ -pe mpich-2perhost 4 # the queue #$ -q workq # stdout and stderr files #$ -o std.out #$ -e std.err # current directory for working #$ -cwd # commands to execute mpirun –np $NSLOTS a.out Job Parameters (see qsub man pages) arbitrary script + some SGE variables