Jean-Louis Pazat IRISA/INSA Rennes, FRANCE MYRIADS Project Team - - PowerPoint PPT Presentation

▶
jean louis pazat
SMART_READER_LITE
LIVE PREVIEW

Jean-Louis Pazat IRISA/INSA Rennes, FRANCE MYRIADS Project Team - - PowerPoint PPT Presentation

Feb 09, 2023 228 likes •540 views

Towards the Magic Green Broker Jean-Louis Pazat IRISA 1/29 Jean-Louis Pazat IRISA/INSA Rennes, FRANCE MYRIADS Project Team Towards the Magic Green

slide-1
SLIDE 1

Towards the Magic Green Broker Jean-Louis Pazat IRISA 1/29

Jean-Louis Pazat

IRISA/INSA Rennes, FRANCE MYRIADS Project Team

slide-2
SLIDE 2

Towards the Magic Green Broker Jean-Louis Pazat IRISA 2/29

OUTLINE

 Clouds and SOA: A green perspective  The Magic Green Broker  Conclusion

slide-3
SLIDE 3

Towards the Magic Green Broker Jean-Louis Pazat IRISA 3/29

CLOUDS & SOA a green perspective

slide-4
SLIDE 4

Towards the Magic Green Broker Jean-Louis Pazat IRISA 4/29

CLOUDS: An “old” idea ?

GRID fundations: Computing as an "utility"

Novembre 1998

slide-5
SLIDE 5

Towards the Magic Green Broker Jean-Louis Pazat IRISA 5/29

CLOUDS vs GRIDS

  • GRID is about technology
  • Tools for using distributed resources in a seamless

way

  • CLOUD is about business
  • Pay-per-use model for data & computation
  • The technical basis of Clouds:
  • Virtualization
  • Clouds are today Data/computer centers
  • Federations of Clouds/Grids in Clouds/Clouds in

Grids are for tomorrow

slide-6
SLIDE 6

Towards the Magic Green Broker Jean-Louis Pazat IRISA 6/29

CLOUDS

  • Two types of usage
  • Computing
  • Data Storage
  • More than one concept
  • IaaS: Infrastructure as a Service
  • you get a "virtual computer”
  • PaaS: Platform as a Service
  • includes OS and software tools to build your (web) application
  • SaaS: Software as a service
  • Software is installed, running. You can call a “service”
slide-7
SLIDE 7

Towards the Magic Green Broker Jean-Louis Pazat IRISA 7/29

Green Challenges for CLOUDS

  • Should be seen in a business perspective
  • CLOUD users
  • don’t care about energy used in Data centers
  • their main concern is price
  • CLOUD providers
  • Care about energy
  • Because of the cost
  • Because of their reputation
  • How to enforce a green perspective ?
  • Taxes ?
slide-8
SLIDE 8

Towards the Magic Green Broker Jean-Louis Pazat IRISA 8/29

SERVICES & SOA

  • In the SOA model applications are hidden
  • Everything is a “service” or a “process”
  • No more application view
  • A service does something for you
  • Could be a piece of software or a human
  • Described with an interface, can be dynamically discovered and invoked
  • One should not worry about “who” does the work (it may change)
  • Most services are stateless
  • A Business Process describes
  • How to achieve some goal
  • Uses services
  • New interest in SOA due to Cloud usage
slide-9
SLIDE 9

Towards the Magic Green Broker Jean-Louis Pazat IRISA 9/29

SOA: A green perspective ?

  • Green IT ?
  • Green workflows, green services, green SLAs
  • Green Services ?
  • Services can expose some QoS parameters
  • => Need for green parameters
  • Energy, CO2
  • Usual difficult problem:
  • How to enforce/guarantee SLA ?
  • Even more difficult with energy
slide-10
SLIDE 10

Towards the Magic Green Broker Jean-Louis Pazat IRISA 10/29

Cloud computing & Services

  • Layered Architecture
  • Final User
  • chooses a service provider
  • calls services
  • Service provider
  • chooses a platform provider
  • provides service implementation
  • Platform provider
  • chooses infrastructure provider
  • deploys/configure service
  • Infrastructure provider
  • Mainly Provides (V)Ms

Service user Service provider Infrastructure provider Platform provider

SaaS PaaS IaaS

slide-11
SLIDE 11

Towards the Magic Green Broker Jean-Louis Pazat IRISA 11/29

Cloud Computing & Services

User vs Provider Requirements

  • Users’ view: Price (including energy) & Quality (including CO2)
  • Final user (SaaS user) needs
  • garantees on service execution (Performance, availability)
  • SaaS provider (SaaS & PaaS user)
  • garantees on single service execution (Performance, availability)
  • Easily deployment of distributed applications
  • PaaS provider (IaaS user)
  • Easily deployment of distributed applications
  • Elastic resources
  • Providers’ view: Profit & Reputation
  • Efficient use of resources (including energy consumption)
  • Includes « virtual resources » such as Services
  • High resource utilization
slide-12
SLIDE 12

Towards the Magic Green Broker Jean-Louis Pazat IRISA 12/29

Cloud Computing & Services

a limited choice …

Final user Infrastructure provider Service provider Platform provider EC2 EMR SES Google App Engine Google Compute Engine Apps

slide-13
SLIDE 13

Towards the Magic Green Broker Jean-Louis Pazat IRISA 13/29

Cloud Computing & Services

I want to choose

slide-14
SLIDE 14

Towards the Magic Green Broker Jean-Louis Pazat IRISA 14/29

Cloud Computing & Services

I want to choose

  • I want to choose everything
  • According to « green » and many other garantees
  • I want to change dynamically my providers
  • Today
  • this is not possible
  • However it would be a nightmare…

Service provider Infrastructure provider Implementation Provider

SaaS IaaS

Platform provider

PaaS SaaS

Service user

slide-15
SLIDE 15

Towards the Magic Green Broker Jean-Louis Pazat IRISA 15/29

Cloud Computing & Services

A bigger Picture

Service provider Infrastructure provider Implementation Provider

SaaS IaaS

Platform provider

PaaS SaaS

  • This is a business organization for tomorrow’s CLOUDs
  • This is not a technical layered architecture
  • Many different players
  • Need for a broker to help users
  • A «Magic Broker » could help…

Service Composition provider

The key player is the Service composition provider

slide-16
SLIDE 16

Towards the Magic Green Broker Jean-Louis Pazat IRISA 16/29

The Magic Green Broker

slide-17
SLIDE 17

Towards the Magic Green Broker Jean-Louis Pazat IRISA 17/29

The Magic Green Broker

  • I need a « service/cloud advisor »
  • Let’s call it a Magic Green Broker
  • Allows to choose the best providers, the best configurations
  • Is able to dynamically adapt sotware & resources
  • Negociates/controls quality
  • Is the best way to get end-to-end energy control ?
  • But still one of my dreams…
slide-18
SLIDE 18

Towards the Magic Green Broker Jean-Louis Pazat IRISA 18/29

Building a Magic Green Broker

  • Adaptation at the IaaS level: Snooze
  • Multilevel adaptation: SAFDIS
  • Brokering at the PaaS level: QU4DS
slide-19
SLIDE 19

Towards the Magic Green Broker Jean-Louis Pazat IRISA 19/29

Adaptation at the IaaS level

SNOOZE: an Autonomous & Energy Aware VM Management System

  • Context & Motivation
  • Huge data centers (dozen of thousands of servers)
  • Scalable management of large numbers of VM
  • Energy-hungry data centers
  • Energy conservation during periods of low utilization
  • Approach
  • Scalability
  • Distributed VM management
  • Dynamic hierarchical management system
  • Self-organizing & self-healing hierarchy
  • Energy conservation
  • Idle nodes in power-saving mode
  • Holistic approach
  • 19
slide-20
SLIDE 20

Towards the Magic Green Broker Jean-Louis Pazat IRISA 20/29

  • Scalable VM Management System
  • Hierarchical algorithm for VM placement
  • Scalable monitoring
  • Heartbeats, multicast groups and election algorithm for self-organization

and self-healing

  • Holistic approach for Energy conservation
  • Underload/overload mitigation
  • Dynamic consolidation
  • Modified version of the SERCON algorithm
  • Power management
  • Evaluation on Grid’5000 on a 144 node cluster with realistic scientific and

elastic web applications

  • Fault tolerance features do not impact application performance
  • Negligible cost is involved in performing distributed VM management
  • System remains highly scalable with increasing number of VMs and hosts
  • Significant energy savings for elastic web applications
  • 20
slide-21
SLIDE 21

Towards the Magic Green Broker Jean-Louis Pazat IRISA 21/29

Multilevel Adaptation: SAFDIS

  • The SAFDIS experimental framework
  • Generic framework based on the MAPE-K model
  • Distributed & decentralized ; OSGi implementation
  • Focus on Distributed Adaptation & Planning

Monitoring Analysis Planning Execution

Execution Platform Single Service Service Composition

slide-22
SLIDE 22

Towards the Magic Green Broker Jean-Louis Pazat IRISA 22/29

Adaptation at Any Level

Adaptation Framework Monitor Decision Planning Execution Event Manager Service-Based Application Service-Oriented Platform Infrastructure (OS & Hardware)

Guide Policy

Probe Monitor Monitor Probe Probe Effector Effector Effector

Framework internals

slide-23
SLIDE 23

Towards the Magic Green Broker Jean-Louis Pazat IRISA 23/29

Monitoring

 Goal

 To provide a dynamic view of the system

 How

 Probes → Monitors → Event Manager  Pull & push probing  Composite events

Adaptation Framework Service-Based Application Service-Oriented Platform Infrastructure (OS & Hardware) Probe Probe Probe Monitor

Event Manager

Monitor Monitor

slide-24
SLIDE 24

Towards the Magic Green Broker Jean-Louis Pazat IRISA 24/29

Results

  • Prototype
  • partly integrated in a component based environment
  • Large scale experiments still missing
  • Used in an « Internet of Things » Application
  • Some lessons
  • Decision is one of the most important part
  • Early bad decisions can lead to worse configurations
  • Late decisions cannot solve problems
  • Planning is difficult
  • Efficiency ?
  • Avoid « hazardous states »
slide-25
SLIDE 25

Towards the Magic Green Broker Jean-Louis Pazat IRISA 25/29

A PaaS level Broker: Qu4DS

 Contract types

deployment and service execution

Fixed (or negociated) time slot

Multiple service calls

SLA templates used for negotiation

Created by the service provider

Only Labels (no values)

 Pricing model

Pay-per-use

Function of expenses

Fines

No auctions

slide-26
SLIDE 26

Towards the Magic Green Broker Jean-Louis Pazat IRISA 26/29

 Work done

An approach for specifying and enforcing SLAs

Creation of SLA templates + Pricing model

Enforcement: QoS assurance mechanisms

Performance & Fault tolerance

Validation

Prevention of losses for distinct fine costs

 Lessons learned

 PaaS as a third party ?

Allows to define better SLAs

IaaS do not provide realistic SLAs for business use

Works like an insurance mechanism for final users

Allows to enforce fault tolerance

Could also use more than one IaaS provider

Could generate profits

By underprovisionning resources (=overbooking)

slide-27
SLIDE 27

Towards the Magic Green Broker Jean-Louis Pazat IRISA 27/29

Conclusion

slide-28
SLIDE 28

Towards the Magic Green Broker Jean-Louis Pazat IRISA 28/29

Conclusion

  • What is so new in these designs ?
  • There should be a contract between any piece of

software from Application to OS

  • These contracts are a place of choice for green

parameters

  • We still have a lot of work to design a

Magic Green Broker

  • We need to re-think OS level software design
slide-29
SLIDE 29

Towards the Magic Green Broker Jean-Louis Pazat IRISA 29/29

Thanks for your attention Questions ?