Outline The Phosphorus project (FP6) Network resource abstraction - - PowerPoint PPT Presentation
FP6 Phosphorus and FP7 Federica projects Network resources for grid applications and Europe-wide experimental open infrastructure Joan Antoni Garca Espn Network Engineer i2CAT Foundation , Barcelona (Catalonia, Spain) 7 th NREN and Grids
7th NREN and Grids workshop, Dublin (I reland), September 2008
Network Engineer
i2CAT Foundation, Barcelona (Catalonia, Spain)
7th NREN and Grids workshop, Dublin (I reland), September 2008
7th NREN and Grids workshop, Dublin (I reland), September 2008
7th NREN and Grids workshop, Dublin (I reland), September 2008
What: 6th FP project in the area “Research networking test-beds” 5.1 M€ EC contribution, 6.9 M€ budget 20 partners, 814 Person Months When: 1st October 2007 – 30th March 2009 (30 months) More: http://www.ist-phosphorus.eu
7th NREN and Grids workshop, Dublin (I reland), September 2008
NRENs & RON:
CESNET Poznan Supercomputing and Networking Center SURFnet MCNC
Manufacturers:
ADVA Optical Networking Hitachi Europe Ltd. NORTEL
SMEs:
NextWorks
HPC centers, Universities and Research Institutions:
Communication Research Centre Fraunhofer–Gesellschaft Fundació I2CAT (with UPC as Third party) Forschungszentrum Jülich Interdisciplinair instituut voor BreedBand Technologies Research Academic Computer Technology Institute Research and Education Society in Information Technology SARA Computing and Networking Services University of Bonn University of Amsterdam University of Essex University of Wales Swansea / University of Leeds
7th NREN and Grids workshop, Dublin (I reland), September 2008
Integration between application middleware and transport networks, based on three planes:
– Service plane:
reservations with those resources
infrastructure, allowing both network resource owners and applications to have a stake in the decision to allocate specific network resources
– Network Resource Provisioning plane:
interoperability with Phosphorus’ resource reservation system
– Control plane:
network resources as first-class Grid resource
interoperability between G2MPLS and Argia/ UCLP, DRAC and ARGON
7th NREN and Grids workshop, Dublin (I reland), September 2008
NBI: It receives the reservation requests from the Grid Middleware. It is used to indicate to the NSP which are the resources under control (NRPSs, endpoints links). EWI: It is in charge of the communication between NRPSs. SBI: It handles the communication between the NRPSs and the lower layers (GMPLS or transport layer). EI: It provides intero-perability between the NSP and the G2MPLS CP or other projects.
GRID APLICATIONS & MIDDLEWARE
OTHER PROJECTS
Network Resource Provisioning Systems Transport Network Transport Network Request Handler Reservation Handler Scheduler TOPOLOGICAL CONFIGURATION
G
2MPLS
Transport
Network
PHASE 2 DB Network Service Plane Path Computer NRPS Broker
ARGON UCLP DRAC
GMPLS
NBI NBI SBI SBI EWI
NBI: North-Bound Interface SBI: South-Bound Interface EWI: East-West Interface EI: External Interface
EI EI
Key points:
connections using resources from several domains
authentication
Grid applications can program fixed, deferrable or malleable resource reservations with one or more connections
t bw t bw t bw t bw t bw t bw
(a) fixed reservation (b) deferrable reservation (c) malleable reservation
Type of advance reservations
Network Resource Provisioning Systems (NRPSs)
ARGON
The Allocation and Reservation in Grid-enabled Optic Networks system was developed to manage resources of advanced network equipment as it is present in the German VIOLA test-
the MPLS level. It guarantees a certain QoS for applications for the requested time interval. This feature enables a Meta-Scheduling Service to seamlessly integrate the network resources into a Grid environment.
DRAC
The Dynamic Resource Allocation Controller system was developed by NORTEL and it is a commercial-grade network abstraction and mediation middleware platform, acting as an agent for network clients (users, applications, compute resource managers) to negotiate and reserve appropriate network resources on their behalf. DRAC uses client's QoS requirements and pre- defined policies to negotiate end-to-end connectivity across heterogeneous in support of just-in- time or scheduled computing workflows.
UCLP (ARGIA)
The User Controlled LightPaths is provided by CRC, Inocybe and i2CAT. It provides a network virtualization framework upon which communities of users can build their own services or
can be considered as a next generation Virtual Private Network where a user can create a complex, multi-domain topology by binding together network resources, time slices, switching nodes and virtual/real routing services.
7th NREN and Grids workshop, Dublin (I reland), September 2008
Key points:
architecture for the Network Service Plane
for the adaptation layer and the network service plane
composed by independent entities (Inter Domain Brokers)
information of each domain they control
tested over the new virtual testbed
10
Provided by the partners Implemented in Phosphorus Network Service Plane (NSP) Middleware (WP3)
ARGON (NRPS)
NRPS Adapter
DRAC (NRPS)
NRPS Adapter
UCLP (NRPS)
Thin NRPS
GMPLS drive
NRPS Adapter ‘zoom’
G²MPLS G²MPLS G-Lambda G-Lambda GÉANT2 (JRA3) GÉANT2 (JRA3)
Reservation-WS Reservation-WS
NSP NSP
Network Service Plane
Administrator
Topology client
Middleware
MSS
User
Application WS-Modules
Topology-WS
Reservation-WS Notification-WS Java MySQL
DB
Req handler Database NRPS manager Resv setup
Resv operation
Authn Topology Validation Path comp
ARGON Domain ARGON Domain UCLP Domain UCLP Domain DRAC Domain DRAC Domain GMPLS Domain GMPLS Domain
ARGON
Reservation-WS
UCLP
Reservation-WS
DRAC
Reservation-WS
GMPLS thin
Reservation-WS ARGON Adapter UCLP Adapter DRAC Adapter
Key Points
using resources from several domains
applications can program fixed or malleable resource reservations with one or more connections
7th NREN and Grids workshop, Dublin (I reland), September 2008
NRPS1
endpoints
in parallel in order to shorten the request processing time.
D H F E G B C A A D E G B C F H
NRPS2 NRPSn NRPS Controllers
NRPS1 Controller NRPS2 Controller NRPSn Controller
ARGON DRAC UCLP Network virtualization Advanced reservations AAA
Partly
(under further development)
Restoration / Protection Point to point connections Point to multipoint connections Interface Type WS WS WS
Partly
(under further development)
7th NREN and Grids workshop, Dublin (I reland), September 2008
NRPSs systems but development of interfaces for NRPSs interoperability through the NSP
act as a wrapper translator
each NRPS uses its own specific interface operations and own specific data types:
– Reservation interface: reservation of resources in e2e path provisioning – Topology interface: updating NSP with topology information
service interface descriptions, defined within WSDL
7th NREN and Grids workshop, Dublin (I reland), September 2008
Interoperability layer Network Service Plane Adapter ARGON driver UCLP driver DRAC driver Topology Service Path Computer Grid Middleware Database NRPS Manager
Reservation service Topology client Reservation service client
Adapter
Reservation service Topology client
Adapter
Reservation service Topology client ARGON domain UCLP domain DRAC domain Topology information Resource reservations Reservation service interface Topology service interface Legend: East / West interfaces Request for end-to-end resource provisioning NRPS address information
Reservation WS:
Topology WS:
Current NRPS adapter functionalities
7th NREN and Grids workshop, Dublin (I reland), September 2008
Middleware NSP NRPS Adapter UCLP NRPS Adapter ARGON THIN NRPS NRPS Adapter DRAC NRPS UCLP NRPS ARGON GMPLS Driver NRPS DRAC UCLP controlled network ARGON controlled network GMPLS controlled network DRAC controlled network
ARGON
Specific NRPS Driver
Common NRPS Driver
DRAC
Specific NRPS Driver
Common NRPS Driver
UCLP
Specific NRPS Driver
Common NRPS Driver
ARGON
Specific NRPS Adapter
DRAC
Specific NRPS Adapter
Common NRPS Adapter
UCLP
Specific NRPS Adapter
NRPS
Specific NRPS Driver Common NRPS Driver
WS Interface
NRPS
Specific NRPS Adapter
Common NRPS Adapter
WS Interface
Common NRPS Adapter Common NRPS Adapter
7th NREN and Grids workshop, Dublin (I reland), September 2008
Thin NRPS: is a network resource provisioning system for domains with a GMPLS control plane. It is a NRPS with restricted functionality
reserve, create and delete network connections via the GMPLS driver
(checking end points availability and possible overlapping reservations)
interface
manager WS of the NSP
7th NREN and Grids workshop, Dublin (I reland), September 2008
GMPLS driver: an interface between NRPS and the GMPLS CP. It is a general WS to create, delete and monitor paths for different GMPLS implementations, provides a WEB interface for testing the WEB service, Internal data base containing topology, path and status information, modules for accessing vendor specific GMPLS control planes (e.g. Alcatel-Lucent or Nortel and G(²)MPLS, dummy interface to GMPLS
GMPLS driver services:
notification
notification
7th NREN and Grids workshop, Dublin (I reland), September 2008
NRPS IDB (top) NRPS NRPS TN TN TN NRPS IDB NRPS NRPS
. . .
NRPS IDB NRPS NRPS
. . .
IDB NRPS NRPS
. . .
Phase 1 : Centralized Architecture (flat or hierarchical) Phase 2 : Distributed-heriarchical Architecture (peering multi-level)
IDB NRPS NRPS
. . . . . .
IDB
. . .
NRPS NRPS TN TN
. . . . . .
7th NREN and Grids workshop, Dublin (I reland), September 2008
Enhancements to the GMPLS Control Plane for Grid Network Services (GNS)
Extensions to the GMPLS CP for automatic and single–step setup of Grid & network resources G2 G2 G2 G2
G.I–NNI G.E–NNI G.O–UNI
G2MPLS NRPS Grid site A Grid site B Grid site C
G.O–UNI
– seamless coexistence with NRPS (UCLP, DRAC and ARGON) & Grid MW – Grid–aware network reference points (G.O–UNI, G.E–NNI, G.I–NNI) – CBR algorithms for recovery and TE – Integration with AAA system
allocation of Grid and network resources, because of
– faster dynamics for service setup – adoption of well–established procedures for traffic engineering, resiliency and crankback – uniform interface for the Grid–user to trigger Grid & network transactions
22
7th NREN and Grids workshop, Dublin (I reland), September 2008
– Open source GMPLS Control Plane prototype for:
Transport Plane (e.g. TL1, SNMP)
– Basic modules for routing and signalling in single control domain delivered on Q3–2007 – Recovery strategies and inter–domain expected by Q1–2008
– Grid–GMPLS enhancements to the Control Plane prototype, in terms of
Grid sites
– Coordination, co–allocation and configuration of grid and network resources associated with a Grid job – Recovery of the installed network services and possible escalation of procedures – Advance reservations of Grid and network resources
connections
7th NREN and Grids workshop, Dublin (I reland), September 2008
ADAPTER
Transport Network
Network Service Plane MIDDLEWARE or DEMONSTRATOR CLIENT NRPSs ARGON DRAC UCLP GMPLS
Transport Network
TOPOLOGY CLIENT G2MPLS
AdvResv capable OUNI-N OUNI-C
THIN NRPS
AdvResv capable ADAPTER ADAPTER ADAPTER
Transport Network
E-NNI ADAPTER
OUNI WRAPPER
OUNI-N
Transport Network
Grid resource B Grid resource A Connect A to B OIF signalling (propagate network info, NO Grid info)
OUNI-C
7th NREN and Grids workshop, Dublin (I reland), September 2008
Service (NRPS)
– Can be called from NRPS domain controller, or – Can drive reservation/provisioning process
– Push, pull, and agent
multidomain network resource provisioning:
– Polling, Relaying and Agent
– Multiple policy combination and/or mapping – Creates and supports dynamic user and resource associations – Supports different AuthZ frameworks interoperation
– Using AuthZ tickets and tokens at Service and Control planes – Integration of token based enforcement mechanism inside GMPLS control plane using RFC2750 policy data object
7th NREN and Grids workshop, Dublin (I reland), September 2008
reservation.
network domain/provider and each consecutive domain provides path to the next domain.
Agent that will do all necessary negotiations with all involved domains
PDP AAA PEP
User Client
NRPS NE PDP AAA PEP NRPS NE PDP AAA PEP NRPS NE Agent P R A
Service (AAA) plane Control plane
Dest Host Appli– cation
Network plane
AAA
TVS – Token Validation Service DRAM – Dynamic Resource Allocation and Mngnt PDP – Policy Decision Point PEP – Policy Enforcement Point
7th NREN and Grids workshop, Dublin (I reland), September 2008
– Adapt and extend Grid middleware to present PHOSPHORUS services to applications
network resources
selection
– Adapt and extend network–based applications to evaluate and demonstrate the PHOSPHORUS developments in the test–bed
7th NREN and Grids workshop, Dublin (I reland), September 2008
– services for user–driven or application–driven set–up of execution environments with dedicated capabilities & performance
– WI SDOM: Wide in silicio docking on Malaria – KoDaVis: collaborative, distributed visualization of huge data sets – TOPS: Streaming of ultra high resolution data sets over lambda networks – DDSS: Distributed Data Storage System – I NCA: Intelligent Network Caching Architecture
7th NREN and Grids workshop, Dublin (I reland), September 2008
Job routing & scheduling algorithms Network & resource management Job routing & scheduling algorithms Network & resource management Simulation environment Simulation environment Control plane design Control plane design
NRPS activity Control Plane activity Middleware and Applications activity
7th NREN and Grids workshop, Dublin (I reland), September 2008
– Job demand models – Grid job routing algorithms
– QoS–aware resource scheduling – Support for advance reservations
– Java, no dependencies, discrete event – Modeling network and Grid resources – Dynamic OCS path set–up and tear–down – Flexible job models (based on Markov states) – GUI to define network topology and traffic models – Code available on svn: http://phosphorus.atlantis.ugent.be/phosphorus
7th NREN and Grids workshop, Dublin (I reland), September 2008
– Requirements analysis and design of the test–bed – Construction of the test–bed and configuration of all related software components, middleware and applications – Tests of project’s developments – Demonstration of project’s results – Real scientific application and real life scenarios used to verify project developments
– Several local test–bed supporting different technologies (optical, Lambdas, Ethernet, SDH switching), GRID & HPC resources and applications – Lightpaths to interconnect local test–beds
– Most resources in local test–beds available for applications, almost all interconnection in place and NPRSes installed – Verification of the test–bed – Installation of middleware and applications
7th NREN and Grids workshop, Dublin (I reland), September 2008
7th NREN and Grids workshop, Dublin (I reland), September 2008
7th NREN and Grids workshop, Dublin (I reland), September 2008
BELIEF, Argia/UCLP, G-Lambda, DRAGON/OSCARS, FEDERI CA, …
7th NREN and Grids workshop, Dublin (I reland), September 2008
What: 7th FP project in the area “Capacities - Research Infrastructures” 3.7 M€ EC contribution, 5.2 M€ budget 20 partners, 461 Person Months When: 1st January 2008 - 30 June 2010 (30 months) Virtualization infrastructure, a “Network Factory” to provide “slices” to researchers in Future Internet, where a slice is a mix of network circuits and computing elements. Built using resources (Gb Ethernet circuits) from GÉANT2 and NRENs as contributions to the project. Open to interconnect other Infrastructures Connected to Internet (through NRENs)
7th NREN and Grids workshop, Dublin (I reland), September 2008
—Act as a forum and support for researchers/projects on “Future Internet”.
Support of experimental activities to validate theoretical concepts, scenarios, architectures, control and management solutions. Users have full control of their slice
—Provide on European scale network and system agnostic e-infrastructure
to be deployed in phases. Provide its operation, maintenance and on- demand configuration
—Validate and gather experimental information for the next generation of
research networking also through basic tool validation
—Dissemination and cooperation between NRENs and researchers’
community
—Contribution to standards in form of requirements and experience
In scope
—Internal extended research, e.g. advanced optical technology —Development and support of Grid applications —Offer raw computing power —Offer transit capacity
Out of scope
7th NREN and Grids workshop, Dublin (I reland), September 2008
Scope
network and computing systems, providing virtualized networks/facilities for end-users, allowing disruptive emulations.
monitoring, including user oriented control (IaaS philosophy).
How
NRENs and GEANT2 (using the GN2+ service)
selected FEDERICA PoPs
infrastructure itself
7th NREN and Grids workshop, Dublin (I reland), September 2008
7th NREN and Grids workshop, Dublin (I reland), September 2008
networks as mix of circuits and computing elements. A slice is as transparent and agnostic from specific technologies as possible.
topology) also between “real” and “virtual” networks
host any application/service
virtual infrastructures
Phase 1
networks) and proceed developing automated tools
7th NREN and Grids workshop, Dublin (I reland), September 2008
– Initially started with PC virtualization (VMware, Virtual Iron, VirtualPC, VirtualBox, and others) – Amazon and BlueLock pioneer the IaaS service by renting hardware using proprietary solutions
– Users pay to use shared infrastructures – Monthly fees or Pay per use – Long term exchanges compared to on-demand services – Users control/own the (virtual) infrastructure – Don’t buy an optical switch, just get some ports of it for a certain period of time!
virtualization solution for optical networks, starting in 2001
– UCLP initial goal was to provide e2e to end paths across domains (Cisco was a sponsor of the program) – UCLPv2 goals were to create reusable and configurable network blocks
(P2V) products and R&D projects that are built on the IaaS Framework:
– Argia -> Product for Optical Networks – Ether -> R&D for Ethernet and MPLS Networks – MANTICORE -> R&D for (logical) IP Networks – GRIM -> R&D for Instruments and Sensors
RMC MANTICORE ETHER GRIM CHRONOS
7th NREN and Grids workshop, Dublin (I reland), September 2008
IaaS Engine (Driver Architecture)
Physical Devices
Transient Information Resource Representations / Service Interfaces (Java) Persistent Information Persistance Layer
DB
Data Sources
LDAP File System
Application Container Security Framework Business Logic
Capability Capability Capability Capability
Presentation
(SOAP WS)
Presentation
(REST WS)
Presentation
(Web GUI)
7th NREN and Grids workshop, Dublin (I reland), September 2008
– Owner of the physical infrastructure (e.g. optical switches and links) decides which parts of the infrastructure can be accessed by what user (e.g. User A can access device X, ports X.1, X.2, … or ALL users can access device Y). – Makes sure the infrastructure is in a good shape (e.g. if a card stops working he must substitute it)
– Asks for resources to physical network administrators or other APN Administrators (e.g. Get device X from network A, port X.1, and also device Y from network B, …) – Integrates all the resources under a single control/management domain (its
– Provides services to the End User (e.g. deploys a circuit reservation service) – Can handoff the control of the resources it has to other APN Admins.
– Typical end user, requests a network service (such as bandwidth on demand, circuit reservations, …)
7th NREN and Grids workshop, Dublin (I reland), September 2008
User A Provider 1 User B Provider 2 User C
Resource List Resource List Resource List Resource List
7th NREN and Grids workshop, Dublin (I reland), September 2008
48
7th NREN and Grids workshop, Dublin (I reland), September 2008
Device Controller Resources Device Partition Resources
Optical Switch Resources Router Resources Resource Broker Resources
End User Services
Ethernet Switch Resources
IP Network Resources TDM Timeslot Resources Ethernet Port Resources WDM Resources Connections Resources VLAN Resources Physical Network Resources Resource List Resources
Resource List and PN Resources
WS MANTICORE (IP Research Project) WS Framework (Open Source) WS Ether™ (Product) (Ethernet Networks) WS Argia™ (Product) (Optical Networks)
Resource Management Centre and User Web Portal.
WS GRIM (Virtual Instruments Research) GRIM Resources Instrument Resources GUI DB Resources Support Services User Workspace Resources
Unless specified otherwise the development is being performed in partnership by i2CAT, CRC and Inocybe Technologies.
RCP WEB (Development)
. . .
7th NREN and Grids workshop, Dublin (I reland), September 2008
Joan Antoni García Espín
joan.antoni.garcia@i2cat.net
I 2CAT Foundation, I nternet and Digital I nnovation in Catalonia
Phone: (+ 34) 93 553 2518 Fax: (+ 34) 93 553 2520
Thanks to
Sergi Figuerola and Eduard Grasa from i2CAT and both Phosphorus and Federica consortiums