Tlcom SudParis Djamal Zeghlache Professor Dpartement Rseaux et - - PowerPoint PPT Presentation

Télécom SudParis

Djamal Zeghlache Professor Département Réseaux et Services Multimédia Mobiles Resource Management Group (in wireless, fixed and computer networks)

Optimisation Combinatoire, Apprentissage et Traitement Distribué de l’Information Réseaux Emergents (IoT, capteurs, Ad Hoc, LTE, SF) Test et Validation de Protocoles et de Services Gestion de ressources, programmabilité et configuration des réseaux Architecture de Services et Services M ultimédia M obiles

Monitoring Optimisation Apprentissage Prédiction Analyse Architecture Modèles Méthodes Réseaux Mobiles & Gestion de Ressources Services Vérification de Propriété, Protocoles IoT Méthodes, modèles et optimisation Architectures e de Réseaux et de Services

Département RS2M

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Equipe (personnes) concernée (s) Cadre

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

Gestion de ressources, programmabilité et configuration 2 E/ C, 6 doctorants, 1 Post Doc, 1 CIFRE Collaborations avec d’autres E/ C (RS2M , RST, INF et UM R CNRS 5157/ Samovar

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Inter-cloud networking framework

Work by Marouen Mechtri - Doctoral Student

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

4

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Cloud Broker Framework

2 3 3

VLAN VLAN

4 6 5 5 1

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

5

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Cloud Networking Gateway Manager

• M. Mechtri - Doctorant

 The CNG Manager has:

• Northbound

interface towards client requesting connectivity based on the OCCI specification and service model.

• The CNG Manager Core selects

the appropriate drivers, in line with user expressed networking requirements.

• Southbound interface interacting

with transport technologies through specific drivers.

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

6

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Cloud Networking Gateway and Gateway Manager implementations (M. Mechtri)

• Code available with readme at
• https://github.com/MarouenMechtri/CNG-Manager
• You can also find links to the following

(Mechtri et Ghribi)

• Installation of Openstack IceHouse release with Flat and

Neutron networking,

• Installation of Heat,
• Deployment of Dockers

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

7

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Managed Resources and Service Instances

MAPE

Users / Consumers / Tenants End users services and Applications Service and SLA descriptions and requests (e.g. WS-Agreement and USDL) Service Manager

(reconfiguration decisions)

SLA & Performance Monitoring of Service Instances

(analysis)

Alerts/notifications SLA violations, degradations, anomalies Service Life-Cycle Management QoS & QoE feedback Monitoring configuration & Probe Deployment

.

Configurable substrate network Cloud Manager SDN Controler & Network API Networking technologies drivers

.

Extended OCCI Interface

VLAN or Networking Controler

.

Cloud Manager Extended OCCI Interface for IaaS & Paas

Networking Controler (Nox/Pox/SDN/Any other)

Network Providers

Heterogeneous Networks & Networking technologies

. Cloud Provider (Heterogeneous PaaS & IaaS)

OCNI Interface OCNI Interface

e.g. OpenVSwitch e.g. Linux Bridge, VLANs, VXLANs... e.g. OpenStack Neutron

Infrastructure Resources/services provisioning (Includes scheduling and smart placement) Cloud services provisioning Network services provisioning SLA & Performance Monitoring of Service Instances (+analysis) Alerts/notifications SLA violations, degradations, anomalies Monitoring configuration & Probe Deployment OCNI Interface

OpenNebula.org (4.2)

Service Requests Reconfiguration Manager {StrategySet} Analyzer {RuleSet} Analyzer {RuleSet} Autonomic Manager (SLA injection & Mngt)

Notifications/Message Exchange

Cloud Provider (Heterogeneous PaaS & IaaS)

In-network services

Orchestration Automatisation (avec Dpt INF-TSP)

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

8

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Smart placement

(Collaborations: M. Hadji à l’ISx depuis et doctorants de l’équipe)

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

9

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

10

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Energy based VM placement algorithm

Objective, conditions & constraints

Bound on the number of used servers

 Objective

• initial VM placement leading to minimum number of used servers

(or containers)  Mathematical Programming Formulation

• modelled as a bin packing problem with a minimum power

consumption objective

Variable comment

m Number of servers Pj,Max Server power consumption limit Pj, current Current power consumption ej Boolean = 1 if j hosts VM xij Boolean = 1 if VM I assigned to server j n Number of requested VMs

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

11

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

VM placement algorithm

Model variables

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

12

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

 Objective

• Optimize data center power

consumption using dynamic VM consolidation  Mathematical Programming Formulation

• Based on linear integer

programming formulation

• Zijk = 1 if VM k migrated from server i to j
• yi = 1 if server i idle and = 0 if at least one

VM is active

• m’ = number of non idle servers m’< m
• P’k = power cost when migrating VM k
• qi is the total number of VMs hosted on

server i and candidate for migration into destination servers, especially server j in equation

VM Migration algorithm (Energy centric)

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

13

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

VM Migration algorithm

Maximize number of empty servers to shut them down by migrating VM to achieve consolidation Destination VM power budget limit has to be respected Ensuing migrations forbidden if a VMk is migrated from a server i (source) to a server j (destination), it can not be migrated to any other server l (l  j). Non idle servers candidate for migration have to be entirely emptied Equivalent total number of empty servers Do not migrate a VM whose job is about to end….

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

14

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

VM Migration algorithm

 A server candidate to a migration should not migrate its own VMs  A VM can not be migrated to many servers at the same time

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

15

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Exact energy efficient graph precoloring Algorithm

Variable comment

G

Global graph of requests.

V

Vertex set of G.

E

The set of all edges in the graph G.

E’

The set of intra VM edges.

C j,id

A virtual resource unit VRU or color where j is the server to which it belongs and where id is its associated id.

C j

Cluster of colors containing colors that belong to the same server j.

w j

Performance per watt (PPW) of the server j.

Z c

Boolean = 1 if color c is used and 0 otherwise.

x u c

Boolean = 1 if node u is reserved to color c and 0 otherwise.

yj

Boolean = 1 if at least one color belonging to Cj is used and 0 otherwise.

n

Total number of nodes in the graph G.

m

Number of servers of the data center.

Notations

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

16

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Exact energy efficient graph precoloring Algorithm

 Objective:

Advanced resource reservation (with minimum number of resources) while maximizing energy efficiency.

 Mathematical Programming Formulation:

Minimize the number of used resources while maximizing energy efficiency Ensure that each node is associated to one and only one color Any two nodes connected by an edge must have different colors Ensure that zc is equal to 1 if the color c is assigned to a node u Nodes belonging to the same VM have to be associated to colors of the same color cluster

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

17

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Exact

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

18

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

• To handle and integrate new request

arrivals, we use pre-coloring which is more general than classical graph coloring.

• Some vertices of the graph have

already assigned colors.

• The aim is to color uncolored vertices

such that all the graph is colored to respect all time overlap and node constraints

Exact

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

19

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Energy efficient graph precoloring heuristic

The EEGP algorithm uses the following steps to find a solution:

1)

Find the color cluster Cj with the highest PPW and with free colors.

2)

Determine the neighboring RRUs (or graph vertices) directly connected to VMi RRUs.

3)

Construct the list (colj,VMi) of colors Cj that are not assigned to Vmi neighboring RRUs.

4)

Finally, the algorithm can assign to each RRU  Vmi a different color from the colj,VMi

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

20

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Joint node and link mapping Models

(mapping/selection/placement) M. Mechtri - Doctorant

Exact algorithm (Joint node & link mapping)

Heuristic algorithm consists of using network patterns to build a pattern based representation of the reference and the input graph. Use the 2 graphs to solve the problem using a bipartite graph matching technique. Heuristic improves convergence by a factor of in convergence time

10

4

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

21

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Mathematical Models Exact Virtual Network Mapping

 Joint node and link selection  Mathematical Programming Formulation Inter domain Path Intra domain Path

VT is the set of vertices and EP the set of edges of the physical or reference or target graph, d(i, k), xij and yij, kn, km are Boolean variables indicating if a virtual resource is mapped on a physical one (nodes & links)

and

Placement

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

22

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

 Node mapping  Limited storage  Link mapping  Node & link mapping  Latency  Localisation

• r

Additional Constraints

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

23

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Additional Constraints

Example of resources localization constraints :

• virtual resources 0 and 4 in same node
• virtual resources 1, 2 & 3 in another node

Input Graph (IG) Reference Graph (RG)

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

24

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Heuristic approach

 Based on graph patterns and bipartite request and reference graph mappings

IG Request Virtual resources RG Infrastructure Physical resources

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

25

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Performance Results

Convergence speed comparison

(from hours to ms , a 104 improvement factor)

Scalability of Heuristic Algorithm

(as a function of input and reference graph sizes)

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

26

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Fedération

• S. Rebai - Doctorante

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

27

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Fedération – Quelques résultats

• S. Rebai - Doctorante

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

28

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Réseaux Logiciels

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

29

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes. Specialized Packet Forwarding Hardware

S ervice S ervice S ervice

Specialized Packet Forwarding Hardware

S ervice S ervice S ervice

Specialized Packet Forwarding Hardware

S ervice S ervice S ervice

Specialized Packet Forwarding Hardware

S ervice S ervice S ervice

Specialized Packet Forwarding Hardware Operating System Operating System Operating System Operating System Operating System

S ervice S ervice S ervice

Network Operating System

LB service FW service IP routing service

Réseaux Logiciels or Software Defined Networking An old Principle that is gradually materializing

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

30

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

CMBS - Cloud Message Brokering Service

Work by Houssem Medhioub, 3rd year graduate student and RA

• Exchanging information between cloud

providers can have several patterns and can

• ccur in different ways like sending

information to one or many providers, or sending information related to a specific topic.

• To satisfy the Cloud networking

requirements of the information exchange, CMBS is based on a message exchange pattern that is structured in five layers

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

31

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

CMBS - Cloud Message Brokering Service

Work by Houssem Medhioub, 3rd year graduate student and RA

• layer 1 - cloud providers discovery space: The role of this

layer is simply to exchange information about the type and characteristics of cloud providers. Cloud providers that want to be members of the CloNe DCP should declare themselves through this layer, in essence they advertise or indicate their desire to join.

• layer 2 - broadcast space: via this layer, a provider can send

an information to all known members of the DCP.

• layer 3.1 - for provider X: via this layer, a provider can send

an information to a particular provider (X).

• layer 3.2 - about topic Y: via this layer, a provider can send

an information about a specific topic (Y) to all providers that are subscribed to this topic (concerned about this topic).

• layer 4 - for provider X about topic Y: via this layer, a provider

can send an information about a particular topic (Y) to a specific provider (X).

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

32

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

OCNI – Open Cloud Networking Interface – extension of OCCI for cloud networking

Work by Houssem Medhioub, 3rd year graduate student and RA

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

33

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

L2 Switching OSPF Mobility Management Load Balancing

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

An Overview of NOX

What is NOX…

• talks to OpenFlow switches
• provides libraries for common
• perations
• connects to GUI

(communicates with NOX by way of JSON messages)

• has an active user community
• runs a network out of box

Core written in C/C++

• event-based
• cooperative threading
• use of STL and Boost

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

35

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

OpenFlow packet format (v1.0)

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

36

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

OpenFlow

Link Provisioning

OpenFlow Switch 1 OpenFlow Switch 3 OpenFlow Switch 2

OpenFlow Protocol

Header Action Statistics Header Action Statistics B Action B 2 Statistics B B Action B 3 Statistics B

VN View

B Action B 1 Statistics B

B 12 BA 1 Link Provisioning

Header Action Statistics

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

37

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

Resource Management Group (in wireless, fixed and computer networks) - D. Zeghlache

Journée Cloud sur Nantes 18-19 Septembre 2014, Mines Nantes.

https://wiki.openstack.org/wiki/Quantum-Linux-Bridge-Plugin

Cloud Computing – Mastère RS -2014-Djamal Zeghlache

39