Co-funded by the European Union Fenix User Forum meeting Parallel - - PowerPoint PPT Presentation

▶
co funded by the european union fenix user forum meeting
SMART_READER_LITE
LIVE PREVIEW

Co-funded by the European Union Fenix User Forum meeting Parallel - - PowerPoint PPT Presentation

Oct 18, 2023 170 likes •491 views

Co-funded by the European Union Fenix User Forum meeting Parallel Session 1 Co-funded by the European Union Fenix User Forum meeting outline Welcome and introduction Anne Nahm, JUELICH ICEI services overview and update

slide-1
SLIDE 1

Co-funded by the European Union

slide-2
SLIDE 2

Co-funded by the European Union

Fenix User Forum meeting

Parallel Session 1

slide-3
SLIDE 3

Fenix User Forum meeting – outline

 Welcome and introduction

  • Anne Nahm, JUELICH

 ICEI services overview and update

  • Jacques-Charles Lafoucriere, CEA

 Examples for how to use ICEI

  • Alex Upton, CSCS

 Fenix User Forum

  • Javier Bartolomé, BSC

 Discussion and questions

  • Cristiano Padrin, CINECA
slide-4
SLIDE 4
  • BSC (Barcelona, ES)
  • CEA (Bruyeres-le-Chatel, FR)
  • CINECA (Bologna, IT)
  • CSCS (Lugano, CH)
  • JSC (Jülich, DE)

Introduction: Fenix & ICEI

5 supercomputing centres (PRACE Hosting Members)

  • Build the Fenix infrastructure
  • Project partners in ICEI*

* Interactive Computing E-Infrastructure for the Human Brain Project

  • > Specific Grant Agreement under the umbrella of the HBP Framework Partnership Agreement
slide-5
SLIDE 5

Introduction: Concept of Fenix

Layered approach for e-infrastructure service provisioning:

HBP Platform/EBRAINS Services that build on/use Fenix Infrastructure Services Fenix Infrastructure Services for HBP and European researchers

slide-6
SLIDE 6

Introduction: Access to resources and Fenix User Forum

  • Access to Fenix Infrastructure Services:

 25 % of available resources are reserved for HBP  Resources need to be requested

  • Light-weight review process for small projects
  • The Fenix User Forum = Opportunity for you as a user to engage

✉ icei-coord@fz-juelich.de

fenix-ri.eu

🌑

slide-7
SLIDE 7

ICEI services overview and update

Jacques-Charles Lafoucriere (CEA)

slide-8
SLIDE 8

Fenix infrastructure – overview

  • Federated computing and data services for European researchers
  • Data storage and scalable computing resources in close proximity to each
  • ther and tightly integrated
  • Service-oriented provisioning of resources, aiming to
  • Meet the requirements of various science communities
  • Form a basis for the development and operation of community-specific platform

tools and services

  • Federation of infrastructure services to optimize for data locality, enhance

availability and broaden variety of services

slide-9
SLIDE 9

Fenix infrastructure – User resources (1)

  • Scalable Computing Services
  • Access to Europe's largest HPC systems
  • Large parallel simulations or data analysis
  • Interactive Computing Services
  • No delay access to single servers
  • Interactive use
  • Virtual Machine Services
  • VM deployment service
  • User defined VM allocation time
slide-10
SLIDE 10

Fenix infrastructure – User resources (2)

  • Active Data Repositories
  • Site-local data store
  • File based interface (parallel file systems)
  • Object storage interface (OpenStack swift)
  • Archival Data Repositories
  • Federated data store for long-term storage
  • Object store interface (OpenStack swift)
slide-11
SLIDE 11

Fenix infrastructure – Infrastructure services (1)

  • Authentication and Authorization Infrastructure (AAI)
  • Central user identification and authentication
  • Used to access Fenix/ICEI resources
  • Fenix User and Resource Management Service (FURMS)
  • Central management of user roles and membership
  • Community resource allocation and accounting
slide-12
SLIDE 12

Fenix infrastructure – Infrastructure services (2)

  • Data Location and Transfer Services
  • Enable Fenix users to locate their data
  • Enable Fenix users to move their data between centres
  • Data Mover Services
  • Enable Fenix users to move data inside centres between Data Repositories
  • Same way to move data in all centres
slide-13
SLIDE 13

Fenix infrastructure – Usages

  • Compute and data analysis
  • Interactive usage (“on demand”)
  • Store data with different classes of service
  • Data comes from simulation or from instruments
  • Run user-defined virtual machines
  • For permanent services
  • For some pre-defined time for a specific service or data processing
slide-14
SLIDE 14

Fenix infrastructure – Available resources

  • Resources at other centres will become available within the next months

See details at https://fenix-ri.eu/infrastructure/resources Service Component ICEI Total Allocation (100%) Available Resources (quarterly)1 Scalable Compute Piz Daint Multicore 250 nodes 465’375 node-hrs Scalable + Interactive Compute Piz Daint Hybrid 400 nodes 744’600 node-hrs Virtual Machines OpenStack IaaS 35 servers 35 servers Long-term Storage POSIX, Object and Tape 4 PB 4 PB Temporary Storage Low-Latency Storage Tier 80 TB 80 TB

1 Amount of available resources is subject to change

Fenix infrastructure services are currently available at CSCS:

slide-15
SLIDE 15

Examples for how to use ICEI

Alex Upton (CSCS)

slide-16
SLIDE 16

What is ICEI being used for?

  • 23 projects from HBP members that are making use of ICEI resources at CSCS
  • These are approximately categorised as follows:
  • 7 ‘small’ scalable compute projects (up to 4,999 hours per quarter)
  • 7 ‘medium’ scalable compute projects (between 5,000 and 19,999 node hours per

quarter)

  • 4 ‘large’ scalable compute projects (20,000+ node hours per quarter)
  • 5 ‘cloud compute’ projects (using Pollux OpenStack)
  • Wide variety of projects, ranging from neurorobotics to cerebellum modelling

to neuromorphic computing front-end

slide-17
SLIDE 17

Examples of running projects

  • The next slides, detail three examples of running projects
  • ich002: Full-scale hippocampus model (PI: M. Migliore)
  • ich006: NEST network construction and simulation (PI: H. E. Plesser)
  • ich009: Neuromorphic Computing Front-end Services (PI: A Davison)
  • The offering of ICEI/Fenix services supports variety of uses across multiple

fields of neuroscience

slide-18
SLIDE 18

ich002: Full-scale hippocampus model

  • Detailed model of rat Hippocampus CA1 region with 460,000 neurons, includes

input projections from CA3 region

  • Studies conditions under which travelling waves of activity at theta rhythm

emerge and how they interact with external inputs to perform cognitive functions

  • Major and unique contribution to the field
  • Model requires a minimum of 25,000 cores (approx. 700 nodes) to run on the

Piz Daint multicore partition

slide-19
SLIDE 19

ich002: Full-scale hippocampus model

In simulation above from the HBP Hippocampus Team, 0.5 secs of simulation time generated 1Tb of data and required 2hr15min to run

slide-20
SLIDE 20

ich006: NEST network construction and simulation

  • Simulations are essential in neuroscience, specifically the NEST simulator is a

key tool for network simulation in HBP

  • This project tests, benchmarks and optimises NEST network construction and

simulation for the benefit of users of large scale cellular level simulation

  • Comprehensive set of benchmarks with a focus on connectivity generation,

automated using the JUBE tool

  • Performance issues in NEST 2.16.0 identified and addressed, which have been

subsequently removed in NEST 2.18.0

slide-21
SLIDE 21

ich006: NEST network construction and simulation

Above plot shows the performance of different versions of NEST on Piz Daint using different numbers of virtual processes

slide-22
SLIDE 22

ich009: Neuromorphic Computing Front-end Services

  • Migration of neuromorphic computing servers from commercial provider to ICEI

resources

  • Amongst others, the following is running on the ICEI VMs:
  • Job Queue REST API
  • Resource Manager REST API
  • Benchmarking REST API
  • Collaboratory apps inc. Job Manager, PyNN Network Builder, Benchmark results

viewer and Job statistics dashboard

  • Before end of SGA2, plan to deploy load balancer in front of the Job Queue API,

and to deploy a centralised logging/monitoring server (based on the ELK stack)

slide-23
SLIDE 23

ich009: Neuromorphic Computing Front-end Services

slide-24
SLIDE 24

Fenix User Forum

Javier Bartolomé (BSC)

slide-25
SLIDE 25

Fenix User Forum

  • An online forum and set of relevant engagement activities directed to users

Objectives:

  • To establish a communication channel between Fenix resource providers and Fenix

users on:  Current and future user needs  Experiences and best practices for using the Fenix infrastructure  User satisfaction

slide-26
SLIDE 26

Fenix User Forum

  • An online forum and set of relevant engagement activities directed to users

Members:

  • Fenix users (including current, potential and rejected users)
  • Fenix resource providers
  • EBRAINS Infrastructure Allocation Committee (IAC) and other members of resource

allocation committees

  • Other people interested in using ICEI/Fenix infrastructure services
slide-27
SLIDE 27

Fenix User Forum: Using the platform

Post content, ask questions, make comments, participate in discussions Access documentation and presentations Join events like webinars and Fenix User Forum meetings

slide-28
SLIDE 28

Fenix User Forum: Signing up

  • Sign up by submitting a form
  • n the Fenix webpage

https://fenix-ri.eu/infrastructure/fenix-user-forum

slide-29
SLIDE 29

Discussion and questions

Cristiano Padrin (CINECA)

@Fenix_RI_eu fenix-ri.eu

🌑

icei-coord@fz-juelich.de ✉

Contact:

slide-30
SLIDE 30

Co-funded by the European Union