Ibercloud: orchestrating services to provide virtualized access to - - PowerPoint PPT Presentation

▶
ibercloud orchestrating services to provide virtualized
SMART_READER_LITE
LIVE PREVIEW

Ibercloud: orchestrating services to provide virtualized access to - - PowerPoint PPT Presentation

Aug 29, 2022 149 likes •374 views

Ibercloud: orchestrating services to provide virtualized access to IberGrid C. Fernandez, A. Simn (CESGA) I. Campos, E. Fernndez, A. Lopez Garcia, J. Marco De Lucas, M.A. Nuez Vega (IFCA) C. Alfonso, I. Blanquer, M. Caballer, G. Molto

slide-1
SLIDE 1

Ibercloud: orchestrating services to provide virtualized access to IberGrid

  • C. Fernandez, A. Simón (CESGA)
  • I. Campos, E. Fernández, A. Lopez Garcia, J. Marco De

Lucas, M.A. Nuñez Vega (IFCA)

  • C. Alfonso, I. Blanquer, M. Caballer, G. Molto (GRyCAP)
  • G. Borges, M. David, J. Gomes (LIP)
slide-2
SLIDE 2

IberGrid

24,000 cores and 20PBytes storage available to Grid user community Protugal & Spain NGIs joint operations Wide usage of virtualization techniques

21/09/12 EGI Technical Forum 2012 2

slide-3
SLIDE 3

Ibercloud Objectives

— Investigate the requirements of scientific

users of cloud technologies

— Deploy a federated cloud IaaS testbed for

scientific computing within the Ibergrid collaboration

  • based on existing local deployments

— Provide a unique user friendly interface

for the services

21/09/12 EGI Technical Forum 2012 3

slide-4
SLIDE 4

Ibercloud Sites

21/09/12 EGI Technical Forum 2012 4

slide-5
SLIDE 5

Authorization

— Users should be able to use a single

identity at all sites

— Grid experience

  • VOMS J
  • User certificates L

— We want a working solution fast

  • working across cloud implementations
  • easy enough to be quickly deployable
  • Flexible for different models of federation (country,

site)

21/09/12 EGI Technical Forum 2012 5

slide-6
SLIDE 6

Architecture

— Start with centralized LDAP authentication:

  • 1. Cloud service portal adds users do main LDAP instance
  • 2. Sites can read LDAP records and authenticate against

LDAP server

LDAP SERVER (NCG) CLOUD PORTAL (IFCA) CLOUD SITES

Read only certain fields Read Write

21/09/12 EGI Technical Forum 2012 6

slide-7
SLIDE 7

Registration Portal (I)

— Web portal to add users to the

infrastructure

— http://cloud.ibergrid.eu

21/09/12 EGI Technical Forum 2012 7

slide-8
SLIDE 8

Registration Portal (II)

— Registration consists on filling a survey

with intended usage

  • Not needed if already part of IBERGRID

— Each request is evaluated and approved

independently

21/09/12 EGI Technical Forum 2012 8

slide-9
SLIDE 9

LDAP tree and namespaces (I)

— Tree with country and site branches

dc=ibergrid, dc=eu c=pt c=es

  • =cloud
  • u=roles

cn=readonly cn=…

  • u=users
  • u=users
  • u=lip
  • u=cesga
  • u=ifca
  • u=upv

general users general ES users LIP users CESGA users IFCA users UPV users

uid=aaa@xxx.pt, ou=users, c=pt,

  • =cloud,

dc=ibergrid, dc=eu uid=bbb@yyy.es, ou=users, c=es,

  • =cloud,

dc=ibergrid, dc=eu uid=ccc@cesga.es, ou=cesga, c=es,

  • =cloud,

dc=ibergrid, dc=eu

21/09/12 EGI Technical Forum 2012 9

slide-10
SLIDE 10

LDAP tree and namespaces (II)

  • Users are “uniquely” identified by e-mail with a

common suffix:

  • Internal remapping within the openldap server
  • All users remapped to o=cloud,dc=ibergrid,dc=eu
  • uid=xxxx@yyyy.pt is also a valid DN
  • We get the advantages of a hierarchical

namespace with the simplicity of a flat namespace

  • =cloud, dc=ibergrid, dc=eu

uid=xxxx@yyyy.pt,

21/09/12 EGI Technical Forum 2012 10

slide-11
SLIDE 11

LDAP Support

— OpenStack:

  • Authentication is performed by a dedicated service

named “keystone”

– Changed architecture while deploying our testbed – LDAP support required particular schema

  • IFCA has extended it for LDAP authentication

– LDAP + LDAPS support – No restrictions on DN or LDAP schema

— OpenNebula:

  • Common DN for all users à remapping at the LDAP

server

  • Secure LDAPS needed tweaks but worked
  • LDAP authentication with the APIs

– Does not work à major show-stopper for us !

21/09/12 EGI Technical Forum 2012 11

slide-12
SLIDE 12

VOMS AuthN

— IFCA+CNRS Started to develop

VOMS AuthN in OpenStack Keystone

  • Ibercloud will evaluate if it fits the deployment

21/09/12 EGI Technical Forum 2012 12

Keystone VOMS mapping HTTPD OpenStack Client HTTPS request with RFC proxy Maps VO & attributes to tenants and roles Checks proxy validity

Code on github: https://github.com/alvarolopez/keystone/tree/voms_auth Docs: http://keystone-voms.readthedocs.org/en/latest/voms.html

slide-13
SLIDE 13

OCCI XML-RPC OpenNebula EC2 EC2 OpenStack deltacloud libcloud Cloud Middleware Web Interfaces Compatibility Layer horizon hybridfox sunstone

Accessing the Resources

21/09/12 EGI Technical Forum 2012 13

slide-14
SLIDE 14

Site Capabilities (I)

Name m1.tiny m1.small m1.medium m1.large m1.xlarge Number of Cores 1 1 2 4 8 Memory (RAM) 512 2048 4096 8192 16384 Disk 20 40 80 160 Intranet Network GB Eth Public/Private IP VLAN and VPN per project, no public IPs currently Name small medium large small-kvm small-occi Number of Cores 1 4 8 1 1 Memory (RAM) 1024 4096 8192 1024 1024 Disk 40GB 60GB 80GB 40GB 40GB Intranet Network 10G Eth. Public/Private IP Pool of public IPs with a maximum of 254

CESGA IFCA

21/09/12 EGI Technical Forum 2012 14

slide-15
SLIDE 15

Site Capabilities (II)

Name tiny small medium large Number of Cores 1 1 2 4 Memory (RAM) 512 1024 2048 4096 Disk 20 40 80 80 Intranet Network GB Eth Public/Private IP Pool of public IPs with a maximum of 32 Name small medium large Number of Cores 1 2 4 Memory (RAM) 512 1024 4096 Disk 10 40 100 Intranet Network GB Eth Public/Private IP VLAN and VPN per project, no public IPs currently

LIP GRyCAP

21/09/12 EGI Technical Forum 2012 15

slide-16
SLIDE 16

Use case: MPI Applications

— Good I/O performance with PCI

Passthrough

21/09/12 EGI Technical Forum 2012 16

1 10 100 1000 10000 100000 1 10 100 1000 10000 100000 1e+06 1e+07 Time in Microseconds Number of Bytes Transferred Intel MPI Benchmark - Reduce Test (16 processes) Bare metal iband VM iband 1 10 100 1000 10000 1 10 100 1000 10000 100000 1e+06 1e+07 Time in Microseconds Number of Bytes Transferred Intel MPI Benchmark - PingPong Test (02 processes) Bare metal iband VM iband

slide-17
SLIDE 17

Use Case: PROOF as a Service (I)

— PROOF is a parallel mode for ROOT

(HEP analysis software)

— PROOF requires the deployment of a set

  • f services on the executing hosts
  • Not trivial for users
  • Dynamic demand of resources

— PaaS on top of the IaaS service

  • Builds PROOF cluster automatically from the

ROOT interface

21/09/12 EGI Technical Forum 2012 17

slide-18
SLIDE 18

Use Case: PROOF as a Service (II)

21/09/12 EGI Technical Forum 2012 18

Proof Master NFS Server Workers Cloud Volume (analysis data) attach Proof as a Service start PROOF PROOF session CLOUD RESOURCES instantiate (1) (II) (III) (IV)

slide-19
SLIDE 19

Use Case: Mathematica

— Used at IFCA for physics phenomenology

simulations

— Very specific machine configuration

  • not grid friendly
  • too heavy for desktops

— Researchers start

VMs with Mathematica as needed

  • hardware independent environment
  • ability to test and execute various software

configurations

  • better reliability and availability

21/09/12 EGI Technical Forum 2012 19

slide-20
SLIDE 20

Next steps…

— Continue working on federated identity

  • VOMS
  • SAML

— Investigate user interfaces/API compatibility

  • OCCI now also available in OpenStack

— Open infrastructure to pilot users

  • Get feedback and requirements

— VM Image Management

  • Image catalogues & repositories

— Monitoring & Accounting

  • following the EGI Cloud TF developments

21/09/12 EGI Technical Forum 2012 20

slide-21
SLIDE 21

Thanks

Questions?