Open Issues for Dynamic Hybrid Network Services Jerry Sobieski - - PowerPoint PPT Presentation

NORDUnet

Nordic I nfrastructure for Research & Education

Open Issues for Dynamic Hybrid Network Services

Jerry Sobieski Director, International Research Initiatives NORDUnet Presented to TERENA End-to-End Workshop December 1 & 2, 2009 Amsterdam, NL

NORDUnet

Nordic I nfrastructure for Research & Education

Motivation and Caveats

• Hybrid networks for Research and Education have

become an important new means for advanced applications to acquire the types of predictable and deterministic performance they desire.

• Dynamic hybrid network services will introduce

both dramatic new flexibility for distributed application teams as well as simplified operations for the R&E networks themselves.

• The observations expressed here are not critiques -

just observations of issues that I believe are important to get right at the outset in order to insure adoption and growth

NORDUnet

Nordic I nfrastructure for Research & Education

What are we talking about?

• Dynamic “circuit” provisioning
• Currently Layer2 Ethernet services
• Layer “2.5” IP/MPLS in some networks
• Global reach
• What is required to accomplish this?
• A documented architecture that describes how it

all will work five years out:

• When we have 1000+ networks offering dynamic circuit

provisioning (some well managed, some not…)

• When we have 10,000+ users globally
• When we have 100,000 network elements participating at

different transport layers

• When we are suporting application specific networks for

1000s of different projects

• When we need to interoperate with non-academic

networks

• When we need to manage multiple transport protocols

NORDUnet

Nordic I nfrastructure for Research & Education

• Keep in mind what the user must do to

utilize these services:

• Simple processes to install necessary end system

software

• Simple and clear interfaces for using the services
• Clear articulated architecture for building out the

reach

• Into the regional networks
• Into campus environments
• Into labs and end systems
• Smaller networks need this type of documentation to

guide them over the next few years as they go thru infrastructure upgrades

What do we need?

NORDUnet

Nordic I nfrastructure for Research & Education

A few areas to work on:

• Simplified [user] installation requirements
• Simplified software complexities
• Clearly defined services
• For performance verification and debugging
• Scalable services
• Control plane protocols
• Authorization mechanisms
• Federated and generalize cyber-resource

management structure

• Including user tools for managing resources and

application specific distributed environments

NORDUnet

Nordic I nfrastructure for Research & Education

KI SS

• Simplicity will be the single most important

characteristic that will promote adoption within the end-user and near-end user environments

• The control plane software must be
• Brain dead easy for users to install !!!
• Includes UNI software, routing (INNI)software (if necessary),

authorization rights, etc.

• Provide a clear and simple programmatic API
• Automated control/data plane configuration
• “DHCP” for hybrid end-systems - easy user configuration of TE

addresses and layer specific TNA registration (LMP-like functions)

• Simplicity aids the network operator as well
• Means that the hybrid networking control software (broadly

construed) does not impose large scale dependencies within existing software/hardware

• The protocols are standards compliant (I.e. supported by

vendors) and interoperate reliably (validated and tested.)

NORDUnet

Nordic I nfrastructure for Research & Education

Reduced Software Complexity

• Minimize control plane software dependencies:
• Specific versions of scripting languages, interpretive

languages/tools, bloated libraries meant for specialized server environments or rapid development.

• These dependencies create complexity and excessive cost to

adoption

• Minimize built-up Layering:
• Layer software only where it makes sense * functionally* - not

simply because some other software lies between your function and lower layers.

• The latter may be useful in a proof of concept, but does not work

well wide scale adoption

• Ex: Developing provisioning interfaces for vendor specific NMSs rather

than implementing a standard protocol native to the device

• Ex: Web Services require XML, SOAP, Tomcat, PHP, X509, … could we
• ffer a user programatic interface that does not require such overhead

(Certainly YES!)

• How easy is it for Joe User to write a “hello world”

application?

NORDUnet

Nordic I nfrastructure for Research & Education

Common Service Definitions

• Minor differences in service provisioning among/between networks can

create unexpected incompatibilities or sub-par performance

• The “Service Definition” specifies the service characteristics offered by

the provider… and experienced by the end user.

• It does not specify how the service is engineered or provisioned, it simply

specifies what is delivered to the end user. How the service provider(s) decides to construct the service infrastructure is not part of the Service Definition.

• A well defined service should be predictable :
• The user knows in advance what to expect from the service in terms of

performance or other characteristics

• A well defined service is verifiable :
• It can be measured at the service delivery point(s) and found to conform to

the predicted (or committed) service characteristics

• A well defined service is repeatable :
• It behaves exactly same way every time it is invoked - but only with respect to

the defined characteristics

• A well defined [circuit] service is end to end :
• It should not matter which provider(s) or which networks are involved in

delivering the service for the end users – they will all result in conforming capabilities as experienced by the end user.

NORDUnet

Nordic I nfrastructure for Research & Education

Common Service Definitions

• Service Definitions are arbitrary.
• A Service Definition is a specific set of service parameters and allowed

values/defaults for those parameters, that form the complete set of measurable service characteristics.

• E.g. MTU < = 9000 Bytes; or
• SpanningTreeProtocol := notTransported;
• Any service characteristic that is not explicitly defined in the service

definition is thereby explicitly undefined

• i.e. the user can neither expect it to be present nor expect it to be absent.
• Service Definitions are flexible
• They can be modified, augmented, enhanced, refined, etc from time to

time as the community sees fit.

• In the near term, the GLIF should adopt several basic broad and inclusive

definitions that allow the process of define-deploy-review-refine to begin.

• There may be multiple Service Definitions: “basic

Ethernet”, “hicap Ethernet”, “TDM”, “Infiniband”, “Packet”)

• Differentiating services is somewhat arbitrary, but should reflect

fundamentally different network service capabilities.

• It is conceivable that some services may be subsets of others, or offer a

base set of characteristics inherited by an entire family of service dialects.

NORDUnet

Nordic I nfrastructure for Research & Education

Common Service Definitions

• A Common Service Definition should have a

formalized specification document…could be a BNF style spec or XML spec

• An example:
• This could as easily be implemented asan

XML document…

• Service_Name

Service_Name := := Ethernet_LightPath Ethernet_LightPath; ;

• Version

Version := 2006.01.10_1.1beta; := 2006.01.10_1.1beta;

• Description

Description := The := The “ “Ethernet_LightPath Ethernet_LightPath” ” service is a service is a point to point service that is to be used by large capacity point to point service that is to be used by large capacity users with certain performance criteria. ; users with certain performance criteria. ;

• Framing :=

Framing := IEEE_802.3; // plain IEEE_802.3; // plain ol

• l’

’ ethernet ethernet

• Frame Size <=

Frame Size <= 9252 Bytes ; 9252 Bytes ;

• Data Rate

Data Rate := 1 Mbs to 10 := 1 Mbs to 10 Gbs Gbs by 1 Mbs; by 1 Mbs;

• Access mode :

Access mode := { 1GE | 1GE_tag | 10GE | 10GE_tag }; = { 1GE | 1GE_tag | 10GE | 10GE_tag };

• Pacing Window :=

Pacing Window := .1 second; // used to constrain .1 second; // used to constrain buf req buf req

• Frame Loss Rate

Frame Loss Rate <= 1E <= 1E-

• 8; // BER=1E

8; // BER=1E-

• 12 and 9000B

12 and 9000B datagrams datagrams

• VLAN_transport

VLAN_transport = False ; = False ; // no VLAN stacks supported // no VLAN stacks supported

• SpanningTree

SpanningTree = False ; // no STP on the = False ; // no STP on the vlan/port vlan/port

NORDUnet

Nordic I nfrastructure for Research & Education

I nter-domain Service Matching

• Since two peering networks may have slightly

different service definitions,or limited available resources at the moment, there is a need to “compare” service requests for compatibility:

• For instance:
• NSP1 offers ethernet with 9258B mtu at NSP2 border
• NSP2 offers ethernet with 1500B mtu at NSP1 border
• A service request for 1500 Byte mtu can be

provisioned either direction…

• But a service request with 9000 B mtu will work across

NSP1 only.

• A Service Definition can be conceived of as a multi-

dimensional volume in n-space.

• End-to-End services can be viewed as intersections of the

services along the proposed path (constraint based path computation)

• Service requests that project inside this n-dimensional

service space can be provisioned. Service requests that lie outside cannot be established. MTU Frame Rate Frame Loss Rate

NORDUnet

Nordic I nfrastructure for Research & Education

• Given agreement on how a service might be

defined and formalized (and the consensus that this is needed!)…

• Find consensus among self-selecting service

providers to define equivalent (“common”) service(s):

• E.g. GEANT, NORDUnet, Internet2, JGN2, AARnet…all

agree that a “Basic Ethernet Circuit” has * specifically*

• A) Framing= IEEE 802.1q VLAN
• B) Capacity= (1 to 19)x 50Mbps xor (1 to 10)x 1

Gbps

• C) MTU= 9000 bytes payload
• (for example…)
• And all networks - that elect to offer compatible

services - insure that their networks in fact support and can deliver the full range of specified service performance characteristics

Common Service Definitions

NORDUnet

Nordic I nfrastructure for Research & Education

• Dynamic state update for inter-domain

topology distribution

• A well designed architecture here will afford the most

flexibility in the future for implementing different types of internal routing & provisioning protocols, security, and multi-domain peering

• Consider that growth of participating networks will create

a substantial increase in path alternatives - and as more paths are available, the more likely paths will be changing state

• Scalable authorization scheme
• Must allow each organization to decide how to allocate

their resources

• Must be able to support extensive growth in networks

and user community without imposing added work for each network

• Bi-lateral agreements - probably the only scalable

means to enable large scale adoption of dynamic circuit services

Dynamic Topology Distribution

NORDUnet

Nordic I nfrastructure for Research & Education

• Authorization should incorporate a “delegated

trust” model in order to scale and to promote adoption

• Particularly important outside the academic community
• “delegated trust” means:
• network A and network B construct bi-lateral authorization

policies between them - say x1, x2, and x3 - each with certain permissions and limits

• Likewise, network B and network C construct separate bi-

lateral policies between them - say y1, y2, and y3

• Network A can request service over B as long as A presents

a valid policy the B recognizes and against which the request can be satisfied: x1 or x2 or x3

• B can forward the request to C for service if B uses a policy

that C recognizes: y1, y2, or y3

• It is up to B to translate authorization: for example: { x1|x2

= > y1, x3= > y3, y2= > x2}

Scalable Authorization Policy

NORDUnet

Nordic I nfrastructure for Research & Education

Delegated Trust

• Why is delegated trust so important?
• Think of it as “I only trust that which I can postively

control” (everything else is potentially dangerous)

• I can control only internal services, and external links to users

and peering networks

• Without trust, every network will need to know and

authenticate every user for every setup req.

• And then users will neccessarily have to specify their own path

(how would the next hop network know which paths will be authorized?)

• Users don’t know how their circuits will be routed - and most

don’t care. Few even know what network their endpoints belong to…and you don’t want users doing your resource allocation(!)

• We need a distributed authorization mechanisms in
• rder to scale
• And scaling is * always* an issue when you want to

promote adoption of a new technology.

NORDUnet

Nordic I nfrastructure for Research & Education

Application I ntegration

• We need to encourage the development of

network resources as an equal component in the larger cyber-resource pool

• I.e. A Generic Resource Management Service

(GeRMS)

• GeRMS would assert a generalized architecture

that would:

• Allow e-resources to be described and advertised
• Allow users to acquire access to these resources based on

user specified constraints (ASTs)

• Provide a user interface that will manage these resources

to initialize them, monitor them, and release them

• Work on this topic is emerging in several areas:

Grid middleware, PlanetLab, GENI, OGF, …

• We need to find a way to bring these perspectives together

into a Grand Unified Theory of E-Resource Management

NORDUnet

Nordic I nfrastructure for Research & Education Application Specific Topologies and

GeRMS Architecture

AST_Master (Tool) AST Description File AST DB (XML) AST minions AST minions AST “Minions” Well known URL(s) User topology description Instantitated Topology state info file(s) Resource specific [embeded?] control/mgmt agents Topology Instantiation Protocol (init, query, release, …) Resource allocation Protocol(s) AST Description File (XML) User defined Application GUI/API/Portal/… ASTDL CLI global resource pool

Resource Specific Brokers/Mappers/Schedulers (agents)

Resource Descriptors (XML files)

User space [private] tools/files Public files/agents/etc..

NORDUnet

Nordic I nfrastructure for Research & Education

Application Specific Topologies “AST”s

• Synopsis:
• AST protocols and middleware allow the user to dynamically

establish customized hybrid network topologies that are dedicated to the user’s application

• ASTs consist of formalized XML (text file) descriptions of distributed

applications:

• A < topology> xml block consists of a set of < resource> blocks
• < resource> blocks describe specific network resources, computational

resources, storage resources, etc required by the application

• < resources> are extensible.

Custom resource schemas can be [could be] defined that include- for instance- sensors such as radio telescopes, or other facilities (e.g. MRI equipment, microscopes, etc.), or functional components such as a database (e.g. human genome db, a national census db. Etc) or a workflow task such as a datamining query, correlation function, simulation, etc .

• ASTs can support e-science processes (e.g. E-VLBI, UltraGrid HD)

and are being studied to support virtual organizations, resilient network architectures, video/visualization content distribution networks, etc.

• Work continues extending the protocols and middleware to support

real-time topology reconfiguration, AST monitoring and management across multiple domains, hierarchical (Object Oriented) specifications, work flow models, and “grid” integration.

NORDUnet

Nordic I nfrastructure for Research & Education

Master D

minion

E

dragond

F

minion

B

dragond

A

dragond

C

dragond

Application Specific Topologies “AST”

C C

minion

B B

minion

A A

minion

E E

minion * RB

OK ! OK ! OK ! OK ! Begin Begin Begin Begin

NORDUnet

Nordic I nfrastructure for Research & Education

Application Specific Topologies using XML

<topology> <resource> <resource_type> eVLBI.Mark5a </resource_type> <name> Haystack.muk1 </name> <ip_addr> muk1.haystack.mit.edu </ip_addr> <te_addr> muk1-ge0.haystack.mit.edu </te_addr> <appl> /usr/local/evlbi_script </appl> </resource> <resource> <resource_type> eVLBI.Mark5a </resource_type> <name> Westford1 </name> <ip_addr> wstf1.haystack.mit.edu </ip_addr> <te_addr> wstf1-ge0.haystack.mit.edu </te_addr> <appl> /usr/local/evlbi_script </appl> </resource> <resource> <resource_type> EtherPipeBasic </resource_type> <src> Haystack.muk1 </src> <dest> Westford.muk1 </dest> <datarate> 1 Gbs </datarate> </resource> </topology>

A B C

A B C

NORDUnet

Nordic I nfrastructure for Research & Education

Applications Specific Topologies

• Live demonstration at Internet2 Spring Member Meeting (April

2006, Washington DC)

• See www.internet2.edu for webcast of “HOPI update”

presentation.

• Set up global multi-link topologies
• Less than 30 seconds (!)

NORDUnet

Nordic I nfrastructure for Research & EducationApplications Specific Topologies

• Live demonstration at E-VLBI Workshop (Sydney, Aus. July

2007)

CHIN NY ARLG ~1000 km ~300 km

NORDUnet

Nordic I nfrastructure for Research & Education

Dynamic ASTs

Node A Link C Node B Link E Node D

Hierarchical / OO ASTs Hierarchical / OO ASTs

NORDUnet

Nordic I nfrastructure for Research & Education Private I P network configuration

• Given a user defined network
• a single light path or (more likely) a set of network links and

switches/routers…

• How does the user configure the upper layer service(s)?
• Almost all such application specific networks are looking for

dedicated and predictable performance among collaborating locations

• They typically still want the convenience of IP layer within
• Auto-configuration of IP network:
• Addressing - where does the addr space come from? How is it

assigned to network elements, end systems, etc.

• Where are services configured? DNS, access control, etc
• Which Routing protocols should be used inside the ASN?
• Peering arrangements - where should this ASN touch external

networks? Which external nets? What routes should be exported?

• Projects like MANTICORE are exploring these IP

architectural and configuration processes

NORDUnet

Nordic I nfrastructure for Research & Education

Business Model

• Basic issue: cost recovery
• How do we allocate the cost of these

[dedicated] resource services

• Fee for Gbps x Hour x km seems complex
• Many users have a fixed budget and would

prefer a subscription model

• What should transit networks charge? To

Whom?

• How do we normalize costs in a global multi-

national service footprint

• Reliability, Security…
• Evolution and maturation…

NORDUnet

Nordic I nfrastructure for Research & Education

Summary

• We need to keep in mind the * USER*

perspective - and minimize the effort they must exert to utilize these services

• We have to make sure we enable the

networks to build reliable and scalable services - not just build a working control plane

• We have to understand what “application”

means in terms of the 21st century global information grid - these will inform the types

• f tools and service models our networks

and middleware will need to address. The End