SELECTED TOPICS IN DISTRIBUTED AND PARALLEL SYSTEMS Grid - PowerPoint PPT Presentation

1 SELECTED TOPICS IN DISTRIBUTED AND PARALLEL SYSTEMS Grid Technologies - Practical Lab

Setting up the Grid Environment 2

Grid Environment Setup (Lab) 3  “Install“ your certificate  Copy XXXX.pem => ~/.globus/usercert.pem  Setup Environment:  Add “/ usr/site/grid/globus /current/bin” to $PATH  Add “/ usr/site/grid/globus /current/lib” to $LD_LIBRARY_PATH  Run “source / usr/site/grid/etc/profile.d /globus.sh”  Should export GLOBUS_TCP_PORT_RANGE=40000,40500  => you might wanna add everything to .bashrc  Download CA certificate into ~/.globus  http://ca.austriangridca.at/root.php  ” wget https://ca.austriangridca.at/certs/6e3b436b.0 ” => ~/. globus  --no-check- certificate … if security error is raised

Setting up Thunderbird 4  Convert certificate and key to the PKCS12 format  openssl pkcs12 – inkey .globus/userkey.pem – in .globus/usercert.pem – out KEYNAME.p12 – export –name “John Doe, AustrianGrid “  PKCS12: container for certificate + private key, encrypted using a symmetric key  Setting up Thunderbird:  Edit>Preferences>Advanced>Certificates>View Certificates  Your Certificate => import KEYNAME.p12  Authorities => import CA certificate (6e3b436b.0)  Edit>Account Settings>$account$>Security  Pick certificate for digitally signing messages  Send a signed Test-mail to yourself, check signature (little letter)  Finally confirm CA mail by a signed response

Using the Grid (Part A) 5

Getting Started 6  Login to the Grid  grid-proxy- init … creates a user proxy  No remote service involved  Proxy created in e.g. /tmp/x509up_u1000  New public/private key pair, certificate signed by user CA User Proxy Proxy … sign sign sign  grid-proxy- info … get proxy information  grid-proxy- destroy … delete proxy  Corresponds to logout  Does not invalidate sub-proxies or stop running jobs (!)

Austrian Grid Parallel Machines 7 Cluster Master Loc. Manager Architecture CPU karwendel karwendel.dps.uibk.ac.at SGE COW 80 AMD Opteron altix-uibk altix1.uibk.ac.at Torque SMP NUMA 16 SGI Altix 350 altix-jku altix1.jku.austriangrid.at PBS SMP NUMA 64 SGI Altix 3k hydra.jku hydra.gup.uni-linz.ac.at PBS COW 16 AMD 2000 lilli lilli.edvz.uni-linz.ac.at PBS SMP NUMA 64 Altix FHV blades.labs.fhv.at Maui COW 12 Intel Xeon alex alex.jku.austriangrid.at PBS Altix ICE 8200 768 Intel Xeon http://agrid.uibk.ac.at/austriangrid/

Login to a Parallel Machine 8  Log into grid-node: glogin  e.g. “ glogin alex.jku.austriangrid.at”  Provides a remote shell through Job Submission  Details: http://www.gup.jku.at/research/projects/glogin  Should only be necessary for debugging  Observing the job queue, logs and temporary files

Running Jobs 9 Using the local batch queuing system

PBS/Torque (1) 10  Getting System Information:  qstat … lists own active jobs  qstat –Q … lists queue information (including names)  More details shown with – f  => see man pages  Handle Jobs  qsub … submit a new job to the queuing system  qstat – u user job_id … track a job  qdel … kills a running job http://www.clusterresources.com/torquedocs21/?id=torque:torque_wiki

PBS/Torque (2) 11  Sample Job submission: "qsub job.pbs” # resources to allocate and execution time #PBS -l walltime=12:00:00,nodes=4:ppn=2 # the queue #PBS -q lva.q Job Parameters # stdout and stderr files (see qsub man pages) #PBS -o std.out #PBS -e std.err # job name #PBS -N my_job # commands to execute arbitrary script cd $PBS_O_WORKDIR mpirun – np 8 a.out + some PBS variables http://www.clusterresources.com/torquedocs21/?id=torque:torque_wiki

SGE – Sun Grid Engine (1) 13  Getting System Information:  qstat … lists all active jobs  qstat –f … lists jobs an queue information  qstat - g c … queue information  Handle Jobs  qsub … submit a new job to the queuing system  qstat – u user – j job_id … track a job  qdel … kills a running job http://www.clusterresources.com/torquedocs21/?id=torque:torque_wiki

SGE - Sun Grid Engine (2) 14  Sample Job submission: "qsub job.sge” # resources to allocate #$ -pe mpich-2perhost 4 # the queue #$ -q workq Job Parameters # stdout and stderr files (see qsub man pages) #$ -o std.out #$ -e std.err # current directory for working #$ -cwd # commands to execute arbitrary script mpirun – np $NSLOTS a.out + some SGE variables http://unix-docu.uibk.ac.at/zid/fremddoc/sge/