Access: bwUniCluster, bwForCluster, ForHLR Shamna Shamsudeen, SCC, - - PowerPoint PPT Presentation

▶
access bwunicluster bwforcluster forhlr
SMART_READER_LITE
LIVE PREVIEW

Access: bwUniCluster, bwForCluster, ForHLR Shamna Shamsudeen, SCC, - - PowerPoint PPT Presentation

Nov 04, 2023 313 likes •600 views

Access: bwUniCluster, bwForCluster, ForHLR Shamna Shamsudeen, SCC, KIT Steinbuch Centre for Computjng (SCC) www.bwhpc-c5.de Funding: Outline Introductjon Registratjon Processes bwUniCluster bwForCluster ForHLR Phase I & II First Steps

slide-1
SLIDE 1

Steinbuch Centre for Computjng (SCC) Funding:

www.bwhpc-c5.de

Access: bwUniCluster, bwForCluster, ForHLR

Shamna Shamsudeen, SCC, KIT

slide-2
SLIDE 2

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

2

Outline

Introductjon Registratjon Processes

bwUniCluster bwForCluster ForHLR Phase I & II

First Steps

Login File Transfer

Questjons

slide-3
SLIDE 3

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

3

  • 1. Introductjon
slide-4
SLIDE 4

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

4

Introductjon

bwUniCluster

Tier 3, Baden-Würtuemberg (BW) cluster for general purposes Simple registratjon process

bwForCluster (JUSTUS, MLS&WISO, NEMO, BinAC)

Tier 3, BW research clusters Architecture optjmized for certain scientjfjc communitjes Access process ensures using the suitable cluster and enhances user support

ForHLR I & II

Tier 2, natjonal research cluster Access process ensures that applicatjons fulfjll requirements of parallelizatjon

1.Introductjon 1.Introductjon

slide-5
SLIDE 5

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

5

  • 2. Registratjon
slide-6
SLIDE 6

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

6

Difgerent clusters → difgerent registratjon processes

bwUniCluster bwForCluster ForHLR Phase I & II Registratjon

  • 2. Access
  • 2. Access
slide-7
SLIDE 7

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

7

Registratjon Process - bwUniCluster

Access only for members of shareholder universitjes. Authentjcatjon with usual university account via bwIDM. Authorizatjon via bwUniCluster entjtlement issued by the universitjes.

  • 2. Access
  • 2. Access

Step A: Obtainment of bwUniCluster entjtlement

Each university has its own entjtlement grantjng policies!

Step B: Web registratjon at htups://bwidm.scc.kit.edu/

Login via bwIDM with your account at university

slide-8
SLIDE 8

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

9

Step 1: Registratjon at „Central Applicatjon Site (ZAS)“

Approval of Cluster Assignment Team (CAT)

Step 2: Obtainment of bwForCluster entjtlement Step 3: Web registratjon at bwForCluster site

e.g. htup://bwidm.rz.uni-ulm.de/, bwForCluster JUSTUS (Computatjonal Chemistry)

Registratjon Process - bwForClusters

Access for members of Baden-Würtuemberg's universitjes.

  • 2. Access
  • 2. Access
slide-9
SLIDE 9

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

11

Rechenvorhaben (RV)

RV = planned compute actjvitjes research objectjve, used methods & sofuware packages, ..

Registratjon Process: bwForClusters – Step 1

  • 2. Access
  • 2. Access

Resource requirements

CPU Hours, memory, disk space, ..

Team

managers, coworker

An RV approval is valid for

  • ne certain bwForCluster

all team members a period of one year afuer the approval Team

managers, coworker

RV responsible

= applicant

slide-10
SLIDE 10

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

12

Registratjon Process: bwForCluster – Step 1a

  • 2. Access
  • 2. Access

RV registratjon

ZAS

slide-11
SLIDE 11

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

13

Registratjon Process: bwForClusters – Step 1b

  • 2. Access
  • 2. Access

acronym / password CAT (Cluster Assignment Team) ZAS RV responsible ? ? ? ?

RV applicatjon

slide-12
SLIDE 12

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

14

Registratjon Process: bwForClusters Step 1c

  • 2. Access
  • 2. Access

CAT (Cluster Assignment Team) ZAS RV responsible acronym / password ! ! ! ! RV approval

slide-13
SLIDE 13

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

15

Registratjon Process: bwForClusters – Step 1a

  • 2. Access
  • 2. Access

ZAS RV responsible Collaboratjon Registratjon via acronym / password „ “

slide-14
SLIDE 14

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

16

Registratjon Process: bwForClusters – Step 3

  • 2. Access
  • 2. Access

ZAS

(Approval of RV)

RV team ?

✓

web registratjon / login bwForCluster: Registratjon server Home University

(bwForCluster entjtlement)

?

Automatjc Check

slide-15
SLIDE 15

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

17

Step 1: Completjon of the „Online Proposal Form“. (htup://www.scc.kit.edu/forschung/4971.php) (Resources to be granted are CPU-tjme and permanent disk space) Step 2: Writjng the extended project descriptjon (3-5 pages). PDF-fjle must be send to forhlr- projects@lists.kit.edu Step 3: Fill out of the ForHLR entjtlement form for each project collaborator. Form must be send to SCC-Service desk or scanned to haefner@kit.edu Step 4: Personal registratjon for ForHLR I & II on website htups://bwidm.scc.kit.edu

Registratjon Process – ForHLR I & II

  • 2. Access
  • 2. Access
slide-16
SLIDE 16

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

18

  • 3. First Steps - Login
slide-17
SLIDE 17

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

19

Login (1)

  • 3. First Steps
  • 3. First Steps

General

Connectjon by ssh Username

[<prefix>_]<username> ab1234 (KIT) ho_anfuchs (Hohenheim)

Host:

uc1.scc.kit.edu justus.uni-ulm.de fh1.scc.kit.edu

Linux / Mac OS Windows (via PuTTY)

$ ssh ab1234@uc1.scc.kit.edu ab1234@uc1.scc.kit.edu's password:

slide-18
SLIDE 18

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

20

Login (2)

Add host key .. and welcome!

  • 3. First Steps
  • 3. First Steps
slide-19
SLIDE 19

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

21

Basic commands

  • 3. First Steps
  • 3. First Steps

$ pwd

show path of working directory

$ mkdir <dirname>

make directory

$ ls -l

list directory contents

$ cd

change directory

$ cp <sourcefile> <targetfile>

copy fjle

$ mv <sourcefile> <targetfile>

move fjle

$ rm <filename>

remove fjle

$ man <command>

show command's manual

$ vi

standard unix editor

slide-20
SLIDE 20

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

22

File transfer

slide-21
SLIDE 21

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

23

File transfer - Linux

Example

$ scp paket.tar ab1234@uc1.scc.kit.edu:dir/ ab1234@uc1.scc.kit.edu's password: $ sftp ab1234@uc1.scc.kit.edu:dir ab1234@uc1.scc.kit.edu's password: Connected to uc1.scc.kit.edu. Changing to: ${HOME}/dir sftp> put paket.tar

  • 3. First Steps
  • 3. First Steps

$ scp <sourcefile> <targetfile>

secure copy (remote fjle copy program)

$ scp -r <sourcedir> <targetdir> recursively copy entire directories $ sftp <targetdir> $ put|get <sourcefile> secure file transfer program upload/download file

slide-22
SLIDE 22

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

24

File transfer - Windows

Windows (via WinSCP)

source directory target directory drag&drop

  • 3. First Steps
  • 3. First Steps
slide-23
SLIDE 23

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

25

bwFileStorage

slide-24
SLIDE 24

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

26

bwFileStorage/LSDF-DIS

Central storage located at KIT 100GB disk space per user Requirements

bwFileStorage entjtlement web registratjon at htups://bwidm.scc.kit.edu

Hosts

Via web: htups://bwfjlestorage.lsdf.kit.edu Via ssh: bwfjlestorage-login.lsdf.kit.edu

Transfer tools

scp, sfup, rsync, htups, ….. rdata @ bwUniCluster, ForHLR

  • 3. First Steps
  • 3. First Steps

bwUniCluster ForHLR bwForCluster bwForCluster bwFile- Storage/ LSDF-DIS ... ...

slide-25
SLIDE 25

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

27

bwFileStorage: rdata

  • 3. First Steps
  • 3. First Steps

File system operatjons on „data mover“ nodes Supported commands:

cp, rm, ls, rsync, mv, mkdir, ...

Environment variables:

$BWFS=/bwfjlestorage/ka/ka_scc/ab1234/

Example:

$ scp fjle ka_ab1234@bwfjlestorage.lsdf.kit.edu: $ rdata cp fjle $BWFS Performance with a fjle size of 20000 MB: scp : 2min 24s (139 MB/s) rdata cp : 1min 5s (308 MB/s)

slide-26
SLIDE 26

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

28

FAQ

“I cannot login”, why? Login at registratjon server of your cluster e.g. bwUniCluster → htups://bwidm.scc.kit.edu @ bwUniCluster: Check if you’ve done the questjonnaire within 14 days afuer the registratjon. Errors may look like:

Check info given at: Registry Check info given at: Registry Info, Index, User Propertjes Info, Index, User Propertjes

slide-27
SLIDE 27

Access: bwUniCluster, bwForClusters, ForHLR / S. Shamsudeen 06/12/2016

29

  • 4. Questjons