Dynamic request allocation and scheduling for context aware - - PowerPoint PPT Presentation

Dynamic request allocation and scheduling for context aware applications subject to a percentile response time SLA in a distributed cloud Keerthana Boloor∗, Rada Chirkova⋆⋄, Tiia Salo⋄ and Yannis Viniotis∗⋄

∗Department of Electrical and Computer Engineering ⋆Department of Computer Science

North Carolina State University

⋄IBM Software Group

Research Triangle Park

1 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Agenda

Agenda Problem description Dynamic request allocation and scheduling scheme Comparison with static allocation and FIFO/Weighted Round Robin scheduling scheme Conclusion

2 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Problem description

Problem description More web applications are designed to be context aware. Most context aware applications are built on SOA principles. Cloud computing systems - the most preferred platform for deployment. Service Level Agreements (SLA) - terms of service and pricing model. What is this presentation about?

3 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Problem description Geographically distributed cloud computing system

Geographically distributed cloud computing system

Clients Data center hosting K context-aware applications Data center hosting K context-aware applications Data center hosting K context-aware applications Data center hosting K context-aware applications 4 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Problem description Context aware applications

SOA based context aware application

Contextaware SOA applications End servers Contextdata stores

Gateway

• 2. Client request allocated to

and scheduled at end-server

• 3. Load required

service-endpoint

• 4. Load required

contextdata DATA CENTER

• 1. Client request with

context-id Internet Updates to contexts at contextdata stores

5 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Problem description Model of an end-server

An end-server serving multiple user classes

Server ‘j’ at data center ‘i’

Class 1 Class 2 Class K

Each context aware application services multiple classes of users Each user class is guaranteed different quality of service based on economic considerations SLA specifies different service levels and service charges for the different user classes

6 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Problem description Percentile Service Level Agreements

Percentile Service Level Agreements

P X 100

Profit Conformance(%)

X% - the fraction of requests of a particular user class which need to have a response time less than r seconds $P - The profit charged by the cloud, if the percentile of requests that have response time less than r seconds is greater than or equal to X%

7 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Problem description Problem statement

Problem statement Allocate and schedule service requests locally at the end-servers so as to globally:

max

• 1≤j≤K

profitj (1)

where profitj is the profit charged for conformance of the requests from users of class j.

8 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Problem description Problem statement

Problem statement Allocate and schedule service requests locally at the end-servers so as to globally:

max

• 1≤j≤K

profitj (1)

where profitj is the profit charged for conformance of the requests from users of class j. This problem is NP-hard!!

8 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Solution Management scheme description

Heuristic-based data-oriented request management scheme

Periodic allocation and adaptation at each datacenter.

Allocation phase Allocation phase Adaptation phase Adaptation phase Observation interval (T) subinterval Allocation phase Allocation phase Allocation phase Adaptation phase Adaptation phase Adaptation phase

9 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Solution Management scheme description

Heuristic-based data-oriented request management scheme

Periodic allocation and adaptation at each datacenter.

Allocation phase Allocation phase Adaptation phase Adaptation phase Observation interval (T) subinterval Allocation phase Allocation phase Allocation phase Adaptation phase Adaptation phase Adaptation phase

Adaptation phase Datacenters exchange conformance levels. Allocation phase Rank-based request allocation and gi-FIFO scheduling. Aim at increasing global profit. 9 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Solution Rank-based allocation and gi-FIFO scheduling

Rank-based allocation and gi-FIFO scheduling Profit-score calculation

Profit: pk Required global conformance: ck Current global conformance: cck If cck < ck Profit-score = pk/(ck − cck) Else Profit-score = 0

10 20 30 40 50 60 70 80 90 100 500 1000 1500 2000 Current conformance of class 1 (%) Profit−score assigned to each arriving request of class 1 ($) Class 1 SLA − Profit of 2000$ on conformance of 75%

10 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Solution Rank-based allocation and gi-FIFO scheduling

Rank-based request allocation

1

Query hash-based lookup table ([context-id,machine-id] or [service-id,machine-id])

2

Rank-based compatibility test

1

The arriving request is assigned a rank based on its profit-score and deadline.

2

Does the arriving request meet its deadline? - Machine compatible!!!

3

Compatible machine not found? - Choose least loaded closest to context DB

11 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Solution Rank-based allocation and gi-FIFO scheduling

gi-FIFO scheduling

Choose the request of user class with the highest current profit-score Choose one with maximum waiting time but which results in a response time less than

• r equal to r

If no such request exists, choose the request with higher waiting time resulting in a response time greater than r

gi-FIFO has been proven to be the most suitable for percentile SLAs for a single server serving multiple classes.

12 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Evaluation

Evaluation

Dynamic scheme vs static schemes

5 10 15 20 25 30 35 40 45 50 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Request rate Profit incurred ($) Dynamic rank based allocation with gi−FIFO scheduling Static allocation with WRR scheduling Static allocation with FIFO scheduling 13 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Evaluation

Dynamic rank based allocation vs static allocation scheme

50 100 150 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Request rate Profit incurred ($) Static allocation with gi−FIFO scheduling Dynamic rank based allocation with gi−FIFO scheduling 14 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Evaluation

Variation in subinterval length

50 100 150 200 250 300 350 400 450 500 2000 4000 6000 8000 10000 12000 14000 16000 18000 Subinterval period Profit obtained($) Uniform distribution of classes, stringent SLA Uniform distribution of classes, relaxed SLA Non−uniform distribution of classes, stringent SLA Non−uniform distribution of classes, relaxed SLA

Variation in context update interval

20 40 60 80 100 120 140 160 180 200 2000 4000 6000 8000 10000 12000 14000 16000 18000 Contextdata update interval Profit obtained ($) Low contextdata load times High contextdata load times Medium Contextdata load times

15 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Conclusion

Conclusion

Identified the need for dynamic request scheduling and allocation for context aware applications in a distributed cloud. Proposed a novel rank-based request allocation and gi-FIFO scheduling scheme for managing percentile SLAs with an aim to maximize profit obtained by the cloud.

16 / 17

Cloudcom 2010, Indianapolis, Indiana, USA

Questions??

17 / 17

Cloudcom 2010, Indianapolis, Indiana, USA