Virtualizing the Philippine e-Science Grid International Symposium - - PowerPoint PPT Presentation

▶
virtualizing the philippine e science grid
SMART_READER_LITE
LIVE PREVIEW

Virtualizing the Philippine e-Science Grid International Symposium - - PowerPoint PPT Presentation

Oct 15, 2022 308 likes •530 views

Virtualizing the Philippine e-Science Grid International Symposium on Grids and Clouds 2011 25 March 2011 Rene Mendoza rene@asti.dost.gov.ph Advanced Science and Technology Institute Outline Who we are ASTI's initiatives on Cluster and

slide-1
SLIDE 1

Virtualizing the Philippine e-Science Grid

International Symposium on Grids and Clouds 2011 25 March 2011

Rene Mendoza rene@asti.dost.gov.ph Advanced Science and Technology Institute

slide-2
SLIDE 2

Outline

  • Who we are
  • ASTI's initiatives on Cluster and Grid Computing –

the PsciGrid

  • Problems and Headaches
  • Going the virtual way
  • Summary
  • Future plans
slide-3
SLIDE 3

The Advanced Science and Technology Institute (ASTI)

  • A research and dev’t institute under the

Philippine Government's Department of Science and Technology (DOST)

  • Our mandate –

Conduct R&D in the advanced fields of ICT and Electronics

slide-4
SLIDE 4

The Philippine e-Science Grid (PsciGrid)

Funding: Department of Science and Technology Grants-In-Aid (DOST- GIA) Program ~ (1M USD) Duration: 3 years (January 2008 – December 2010); Extended until June 2011 Objectives:

  • Establish a national e-Science grid infrastructure in the Philippines

that will enable collaborative research among local educational and research institutions.

  • Provide seamless access to high-performance computing resources

and applications for Life and Physical Sciences

slide-5
SLIDE 5

The PsciGrid Infrastructure

University of the Philippines Computational Science Research Center University of the Philippines Computational Science Research Center Ateneo De Manila University School of Science and Engineering Ateneo De Manila University School of Science and Engineering Advanced Science & Technology Inst.

  • Dept. of Science & Technology

Advanced Science & Technology Inst.

  • Dept. of Science & Technology

100 Mbps 100 Mbps 155 Mbps

slide-6
SLIDE 6

ASTI's HPC

Computing

  • 51 computing nodes (2 x 2.0 GHz Intel

Xeon), 408 cores

  • 300GB/500 GB of disk space and

16GB/24GB of RAM per node

  • 8 FPGA – based hardware accelerators

Storage

  • 6TB for raw data
  • 4TB for DNA and protein sequences (Bio-

Mirror)

  • 4TB for software mirror
  • Additional 16TB (usable 12TB)
slide-7
SLIDE 7

ASTI's HPC

2 nodes 5 nodes 7 nodes 8 nodes

slide-8
SLIDE 8

UP MSI (1) UP NIP (13) UP CSD (1)

Users and Applications

  • Users
  • Applications

– RegCM, WRF, MM5 – Bioinformatics apps – SPECFEM3D

slide-9
SLIDE 9

Problems and Headaches

  • J o b s c h e d u l i n g i n e f fi c i e n

→

  • Unbalanced utilization of compute nodes

b e t w e e n c l u s t e r s s o m e c → unutilized

  • H e t e r o g e n o u s c l u s t e r s e t

→

  • D i s k s p a c e s h o r t a g e u n h

→

  • Low uptake for targeted users of the HPC
slide-10
SLIDE 10

To charge or not to charge?

  • Government projects have very limited

funding

  • Must come up with a sustainability plan
  • Charge users based on some criteria?
slide-11
SLIDE 11

To virtualize or not to virtualize?

Two Options: 1.Setup one big physical cluster and provide access to all local users OR 2.Setup individual virtual clusters running on

  • ne common physical cluster

OUR CHOICE? 2

slide-12
SLIDE 12

Why Virtualization?

  • Dynamic or on-demand reallocation of

unused compute nodes to busy clusters.

  • Better user isolation and fine-grained control
  • Easier backup
  • Flexibility
slide-13
SLIDE 13

Cluster Usage Policy (aka “The Ten Commandments in Using the HPC Cluster”)

1.A user needs to apply for a valid login- account. 2.SSH access is allowed through key-based authentication only. 3.No sharing of login accounts. 4.User should report any suspicious activity regarding his/her account. 5.No running of jobs on the head node.

slide-14
SLIDE 14

The Ten Commandments in Using the HPC Cluster 6.Jobs should be run using Torque. 7.Paying users will be prioritized over non- paying users. 8.The maximum number of running jobs per user is 4. 9.No logging in on the compute nodes. 10.Repeated violations of these rules may lead to suspension of the user login account

slide-15
SLIDE 15

Storage Policy

1.Each user is given 3 GB of HD space in his home directory. 2.Files in the home directory will remain as long as the account is valid. 3.A work space will be provided under /work. 4.The work space has no quota, but when the filesystem becomes full, the work space is reduced to 50% (delete oldest first).

  • 5. Larger disk storage may be requested but is subject

to disk space availability.

slide-16
SLIDE 16

Experimental Virtual Clusters Setup

  • Two clusters

– Buhawi (meteorology cluster)

  • 16 nodes, 4 cores per node, 8 GB memory,

36 GB HD

– PRAGMA

  • 13 nodes, 2 cores per node, 4 GB memory,

36 GB HD

slide-17
SLIDE 17

Virtual Clusters Setup

Physical Cluster: Rocks 5.3 with Xen Roll

slide-18
SLIDE 18

Summary

  • ASTI's HPC is gradually moving to a full

virtual infrastructure (uphill battle)

  • Usage and storage policies are finally being

implemented.

  • Exploring the possibility of setting up a

sustainability plan

slide-19
SLIDE 19

Future Plans

  • Continue virtualization efforts (3-6 months)
  • Procurement of blade servers (next 2-3 years)
  • Diskless clusters (within 1 year)
  • IPv6 clusters (within 1 year)
  • Storage clusters (within 1 year)
  • Use of the HPC for the government's disaster

mitigation plans (within 1 year)

  • Integrate PsciGrid with government-wide data

center (within 1 year)

slide-20
SLIDE 20

Thank you!