The UCLA Grid Portal (UGP) and the University of California Grid - - PowerPoint PPT Presentation

▶
the ucla grid portal ugp and the university of california
SMART_READER_LITE
LIVE PREVIEW

The UCLA Grid Portal (UGP) and the University of California Grid - - PowerPoint PPT Presentation

Jan 13, 2024 89 likes •487 views

The UCLA Grid Portal (UGP) and the University of California Grid (UC Grid) Joan Slottow, Prakashan Korambath, Kejian Jin UCLA Academic Technology Services 1 The University of California UC Office of the President (UCOP) 10 autonomous

slide-1
SLIDE 1

1

The UCLA Grid Portal (UGP) and the University of California Grid (UC Grid)

Joan Slottow, Prakashan Korambath, Kejian Jin UCLA Academic Technology Services

slide-2
SLIDE 2

2

The University of California

  • UC Office of the President (UCOP)
  • 10 autonomous campuses
  • Medical Centers
  • Lawrence Berkeley National Laboratory

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2

slide-3
SLIDE 3

3

UC-Wide IT Policy and Planning

  • IT Leadership Council (ITLC)

– UC-wide IT Governing Body – Membership = CIOs and IT leaders from UC entities. – Strategic planning and priorities

  • IT Guidance Committee (ITGC) charged

with recommending strategic directions to guide IT investments

– Issued Report December 2007

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3

slide-4
SLIDE 4

4

ITGC Report

  • IT infrastructure is critical

– UC Grid is a key enabling technology

  • Scalable data center services
  • Sustainable funding models
  • Collaboration
  • Networking (CENIC)

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

4

slide-5
SLIDE 5

5

UC must operate as one and leverage the strengths of the 10 campuses

  • NEED Multi-campus initiatives and

coordination

  • UC Grid has to be implemented in

collaboration at the campus level

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

5

slide-6
SLIDE 6

6

Grid

  • Multiple, independent, geographically

dispersed compute resources -- networked together -- used cooperatively

  • Autonomous units under local control
  • Named after the electricity grid

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

6

slide-7
SLIDE 7

7

UC Grid

  • Started at UCLA and currently

implemented at UCLA, UCOP, UCSB, and UCI

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

7

slide-8
SLIDE 8

8 CENIC 08: Lightpath to the Stars • cenic08.cenic.org

8

slide-9
SLIDE 9

9

UCLA Grid Portal (UGP)

  • Joins computational clusters into a Grid.
  • Under development at UCLA since 2002.
  • Built on top of open source components.
  • A Globus Incubator Project

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

9

slide-10
SLIDE 10

10

Motivation

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 Hoffman Cluster Hosted Cluster1 Hosted Cluster2 Hosted Cluster3 Hosted Cluster4 Hosted Cluster5 Professor 1’s Cluster Professor 2’s Cluster Professor3’s Cluster Professor4’s Cluster Professor5’s Cluster MAC OS Myrinet InfiniBand

???

slide-11
SLIDE 11

11

Motivation

  • Common interface
  • Single sign-on
  • Work with/see all resources at once
  • Transfer files
  • Portal makes it easy for users to use

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 1

slide-12
SLIDE 12

12 CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 2

Head node

Cluster II

Head node

Cluster I

ION Visualization Server Storage Server MyProxy Server Grid Portal Grid Appliance Credential mapped to local ID Credential mapped to local ID Grid Appliance https Uniform browser based interface

Single Campus Architecture

slide-13
SLIDE 13

13

UC Grid Architecture

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 3

UC Grid Portal

UCLA Grid Portal

  • thers

C1 C2 C3 C4 C5 C6 …… …

  • User always applies from Campus Portal
  • One Certificate Authority(CA) throughout UC Grid
  • User CA is automatically created and pushed to myproxy server
  • Simple account creation process (approval/denial workflow)
  • Once the admin approves, the user is able to login to two portals instantly
  • Enabled by Register Web Service and Sync Web Service in UC Grid Portal

… …

UCSB Grid Portal UCI Grid Portal

slide-14
SLIDE 14

14

The UC Grid Portal

  • Each user of a Campus Grid is

automatically able to use the UC Grid Portal

  • From the UC Grid Portal, users can:

– work with clusters from all the campuses – transfer files across the campuses. – Use pooled resources across the campuses.

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 4

slide-15
SLIDE 15

15

UGP Services

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 5

slide-16
SLIDE 16

16

Resource Discovery

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 6

slide-17
SLIDE 17

17

Data Manager

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 7

slide-18
SLIDE 18

18

File Transfer

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 8

slide-19
SLIDE 19

19

Visualization

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

1 9

slide-20
SLIDE 20

20

Batch Jobs

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 Generic

slide-21
SLIDE 21

21

Batch Jobs

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 1 Applications and pool Jobs

slide-22
SLIDE 22

22

Pool Job Submission

  • A way to share/use unused cycles.
  • UGP:

– Chooses the target cluster for the job. – Submits the job to run under a guest login id there. – Stages the input files.

  • UGP sends jobs to local scheduler.

Decision for sharing resources made by the cluster owner.

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 2

slide-23
SLIDE 23

23

Application Input Forms

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 3

slide-24
SLIDE 24

24

Data Manager and Batch Jobs to Other Grids

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 4

slide-25
SLIDE 25

25

Interactive GUI Applications

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 5

slide-26
SLIDE 26

26

Xterm

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 6

slide-27
SLIDE 27

27

UC System Wide Mandate

  • The CIOs of the campuses have agreed

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 7

slide-28
SLIDE 28

28

Has to be Carried Out on each

  • f the 10 Campuses
  • Implementation/maintenance/development has

to happen on each of 10 campuses

  • People doing the work need:

– a connection with the researchers – time to do the work – resources

  • ATS at UCLA has the leadership role.

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 8

slide-29
SLIDE 29

29

Campus have very Distributed Research IT Infrastructures

  • Individual researchers acquire and

maintain their own HPC resources.

  • Most campuses do not have:

– central research computing organizations – HPC facilities/resources for the campus

  • This is not sustainable
  • Required:

– Data centers, HP Networks, support staffs

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

2 9

slide-30
SLIDE 30

30

  • UCLA is unique in that it has ATS

– Emphasis on research computing

  • HPC, Scientific Visualization, Statistics
  • Close relationship with researchers

– Started as the campus computer center, but:

  • Remained throughout the era of decentralization
  • Was on the forefront of: cluster building, cluster

hosting, grid

  • Has the ability to effect changes
  • UGP Development

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3

slide-31
SLIDE 31

31

Who to work with on the Campuses

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3 1

slide-32
SLIDE 32

32

UC Grid Implementers

  • Monthly Meetings (Skype, ichat, VNC)
  • Discussion/Mailing List
  • ATS Provides Support for implementers

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3 2

slide-33
SLIDE 33

33

The UC Grid Provides for the Sharing of System-wide Resources

  • The majority of campuses are planning

data center expansions

  • Research computing needs are growing
  • The grid provides a way to use virtual

(networked) resources no matter where they are.

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3 3

slide-34
SLIDE 34

34

January 2008 -- UCLA -> Shared Cluster Hosting Model

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3 4 New Hoffman2 Cluster

slide-35
SLIDE 35

35

Motivation for the Shared Cluster Hosting Model

  • Manpower to administer multiple hosted

clusters >> to administer 1 cluster

  • > 50% cycles are idle
  • Power use by idle clusters is not

significantly lower than for full operation

  • The ability for others to use your idle

cycles is the price you pay to join the shared cluster

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3 5

slide-36
SLIDE 36

36

The UCLA Shared Cluster Model scales to the UC

  • Cost of data center space/location
  • Cost of electricity/location
  • Redundant and wasted capacity
  • Great need

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3 6

Sharing must be attractive to the resource

  • wners. Only they can make the decision to

share.

slide-37
SLIDE 37

37

February 2008 -- UC Research Cyberinfrastructure Pilot -- Draft Document

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3 7

Phase I:

Provide computing resources via two moderate institutional clusters housed at LBNL and SDSC Network-based services via UC Grid

Phase II:

Expand and develop

slide-38
SLIDE 38

38

Learn more about UGP

  • UGP documentation, download and wiki

CENIC 08: Lightpath to the Stars • cenic08.cenic.org

3 8

http://www.ucgrid.org