New Storage and Cloud Computing Services Proposed by France Grilles - - PowerPoint PPT Presentation

New Storage and Cloud Computing Services Proposed by France Grilles

Jérôme PANSANEL <jerome.pansanel@iphc.cnrs.fr> Technical Director at France Grilles & Catherine BISCARAT, Cal LOOMIS, Gilles MATHIEU, Geneviève ROMIER

2

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 2 -

Contents

• Ongoing Projects at France Grilles
• FG-CLOUD
• FG-iRODS

3

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 3 -

Ongoing projects at France Grilles

France Grilles, overseer of French production grids and manager of the relationship with the user communities, aims at increasing the usage of distributed computing

• infrastructures. To achieve this objective, three projects are ongoing:
• FG-DIRAC: a national DIRAC instance whose objective is to simplify the access

to grid and cloud resources for users.

• FG-CLOUD: a project that aims to federate IaaS-Cloud resources in France.
• FG-iRODS: a storage space based on the iRODS software, which provides

virtualized storage.

4

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 4 -

FG-CLOUD: Objectives & Obstacles

• Objectives
• Set up a strategy to develop a French Federated IaaS-Cloud infrastructure
• Develop the national infrastructure
• Build a policy for the resource usage
• Define user services
• Obstacles
• Coexistence with existing initiatives (bottom-up approach)
• France Grilles strategies should be compliant with third-party strategies (sites,

guardianship, partners)

• Several solutions are currently used for providing Cloud resources
• Cloud resources should be transparently accessible
• Demonstrate the benefit of a federated Cloud

5

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 5 -

FG-CLOUD: Technical Group

• Group composition and functioning
• Members: site administrators, developers and advanced users
• Bimonthly meeting
• Guided by a mission statement
• Contact: cloud-tech-l@france-grilles.fr
• Ongoing
• Development around SlipStream for using OpenNebula
• VM-DIRAC configuration tests for working with OpenNebula and SlipStream
• Development of Nagios tests
• Objectives
• Do a proof of concept of the French federated Cloud with a real scientific

use-case (Cloud Challenge)

• Ensure that the Cloud is working in a production environment (service

redondancy, monitoring, service catalog, ...)

• Extend the available resources and user community

6

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 6 -

FG-CLOUD: Overview of the Sites

• Production sites
• OpenStack@CC-IN2P3 (*)
• CloudMip@IRIT
• Stratuslab@LAL (*)
• Pre-production sites
• OpenStack@IPHC (*)
• StratusLab@LLR (*)
• StratusLab@LUPM
• OpenStack@UNIV-LILLE
• Planned site
• StratusLab@LPSC (*)
• More information
• http://www.france-grilles.fr/6-Cloud

7

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 7 -

FG-CLOUD: Achievements and Perspectives

• Achievements
• Common authentication system
• Single marketplace hosted at LAL
• VM-DIRAC and SlipStream available for accessing the resources (OpenStack,

StratusLab)

• Funding call for helping sites joining EGI Federated Cloud (ongoing)
• AUPs are defined
• Perspectives
• Validate the use of VM-DIRAC and SlipStream with OpenNebula
• Development of rOCCI backend for StratusLab
• Launch the Cloud Challenge
• Deploy a monitoring solution to ensure infrastructure reliability
• Provide an expertise for Cloud deployment

8

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 8 -

FG-iRODS: iRODS in a few Words

iRODS

Disk MS Disk

• Project started in 2006 (based on SRB)
• Release under an Open Source license (BSD)
• Developed by the DICE group and several

collaborators

• Rule Engine applies user-defined policies and rules
• Integrate a descriptive metadata system to manipulate

data

• Data collections manageable over several sites and

heterogeneous hardware

• Logical organization of files is independent of its

physical implementation

• Enforcement of data consistency and homogeneity

http://irods.org/

9

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 9 -

FG-iRODS: French NGI Infrastructure

• Coordinated by France Grilles
• A single production instance:
• Federated resources and workforce
• Hosting users from any scientific domain
• Design for small and medium projects
• Open to new resource providers
• User support and training
• Contact:

irods-support-l@france-grilles.fr

9

40 TB 20 TB iCAT 40 TB

10

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 10 -

FG-iRODS: Achievements

• Federated set of resources for a total of 100 TB
• Real synergy between the administrators
• Reliable and highly available storage
• Usage policies is published
• First training has been performed (Clermont-Ferrand, February 2014)
• First users are currently hosted (proteomics and biological data)
• Web Interface available
• iRODS clients installed on all grid sites supporting the France Grilles VO
• iRODS packaging with GSI support (deb, rpm)
• VM appliance provided to access the computing Grid and iRODS

11

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 11 -

FG-iRODS: Perspectives

• Extend the storage pool with new resource providers
• Find new financial resources to ensure the sustainability of the

infrastructure

• Deploy a monitoring solution to ensure infrastructure reliability
• Test the S3 plugin
• Welcome more new users
• Share expertise regarding data management and user support with other

groups

12

• J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud
• 12 -

Thank you for your attention!