On the use of Fuzzy Logic Controllers to Comply with Virtualized - - PowerPoint PPT Presentation

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud

Kyriakos M. Deliparaschos Cyprus University of Technology k.deliparaschos@cut.ac.cy Evangelia Kalyvianaki City University London evangelia.kalyvianaki.1@city.ac.uk. Themistoklis Charalambous Chalmers University of Technology, Sweden themistoklis.charalambous@chalmers.se Christos Makarounas Cyprus University of Technology christos.makarounas@cut.ac.cy.

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 2

Outline of the presentation

❖ Introduction ❖ Motivation ❖ System Model:

• mRT model
• System Usage and Allocation

❖ Adaptive Neuro-Fuzzy Inference

• The Adaptive Neuro-Fuzzy controller
• Training on Kalman and H∞ controllers

❖ Performance Evaluation and comparison with linear

controllers

• Gradual Workload Changes
• Saw-tooth demand for CPU usage

❖ Conclusions and Future Work

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 3

Introduction

Modern omnipresent server applications are complex programs that provide diverse services to thousands of users, e.g., Amazon and Google services. Traditionally,

• disjoints sets of machines are dedicated to different applications.
• over-provision applications - machines to cope with their demanding workloads.

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 4

Motivation

• Over-provisioning causes machine under-utilization in modern data centers as

shown by several reports: servers typically run at 15-20% of their capacity.

• Maintaining under-used machines at scales of contemporary data centers, has

severe financial consequences in capital expenses and additional power costs.

• To alleviate these issues, current data centers employ server consolidation -

multiple applications running on the same machines - to run fewer but more utilized machines.

• To fully capitalize on these mechanisms, methods for adaptive resource allocation

subject to applications' resource demands are required.

• If each application is properly provisioned, unused resources can be allocated
• therwise, e.g., to run additional applications.
• However, the application demands are difficult to estimate in advance since their

characteristics typically change over time.

! " ! "

!"#"$%&'()*+(#,

• ./#0$%&'()*+(#,

#++(%#"$(/&1 #++(%#"$(/&2 #++(%#"$(/&1 #++(%#"$(/&2

! "

• ./#0$%&'()*+(#,

#++(%#"$(/&1 #++(%#"$(/&2

Resource management example in virtualized applications: the resource allocation needs to adapt to the new resource demands

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 5

System model

• A. mRT model

!"#$"#

%&'( )& *& 0.2 0.4 0.6 0.8 1 0.5 1 1.5 2 2.5 3 CPU Demand/CPU allocation mRT (s)

mRT model with respect to CPU usage. Model of the demand D in a server.

mRT is given as a function of the ratio between workload demand D and allocation a and it is depicted in the following figure: A metric for measuring server performance is the client mean request response times (mRT). A graphical illustration of the demand model in a server is given in the figure:

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 6

System model

Utilization (uk) modeled as a random walk

uk = c × ak

where c<1 is a pre-specified constant that shows the desired gap between allocation and usage.

• B. The CPU Usage and Allocation
• rk : the utilization between successive intervals caused by workload changes, 


e.g., requests being added, doing work from previous intervals, or leaving the server.

uk+1 = uk + rk

Aim: to maintain good server performance in the presence of workload changes by adjusting the allocation (αk) to values above the utilization, i.e.,

Problem targeted by aiming to minimize the MMSE (Kalman filter), and to minimize the maximum error (H∞ filter)

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 7

The Fuzzy Controller

A1 A2 Layer 1

• Layer 2

N N

Layer 3 f1 f2 Layer 4

• Layer 5

x µA1(x) µA2(x) w1 w1 w2 w2 w1 w2 w1f1 w2f2 y

Layer 1 is the fuzzification layer. Each neuron in this layer represents the fuzzy sets expressing the linguistic terms in the antecedents of the fuzzy rules. A 5-layer SISO neuro-fuzzy model with a zero-order T-S inference structure is used. Layer 2 forms the rule layer. Each neuron is analogous to a single Sugeno type fuzzy rule. Layer 3 is the normalization layer where each neuron receives inputs from Layer 2 and determines the contribution wi Layer 4 represents the defuzzification layer, where each neuron calculates a weighted consequent value of a given rule

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 8

The Fuzzy Controller

A1 A2 Layer 1

• Layer 2

N N

Layer 3 f1 f2 Layer 4

• Layer 5

x µA1(x) µA2(x) w1 w1 w2 w2 w1 w2 w1f1 w2f2 y

A 5-layer SISO neuro-fuzzy model with a zero-order T-S inference structure is used. Layer 5 sums all outputs from the defuzzification neurons and produces the overall ANFIS defuzzified crisp output y as a weighted average of the contribution of each rule:

y =

n

X

i=1

yi = Pn

i=1 wifi

Pn

i=1 wi

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 9

Training

% CPU demand on which the controllers have been trained and tested. The range of values of % CPU demand is [0, 105], wide enough to include most of the range of values of % CPU demanded.

20 40 60 80 100 120 10 20 30 40 50 60 70 80 90 100 time interval CPU demand (%)

Demand: training Demand: 2 Demand: 3 Demand: 4

• Trained on representative data sets formed by the observed allocation and

utilization with Kalman and H∞ filters

• The membership function parameters of the fuzzy inference system were trained

for 20 epochs to adopt to the training data

• The chosen scheme includes most of the range of values of % CPU demanded and

with different rates of changes

• The SISO fuzzy estimator maintains a low

complexity with,

• 5 triangular type MFs on the aggregate part
• 5 singletons on the consequence part
• 5 rules
• Training algorithm uses the least squares

method in the forward pass to identify the consequent parameters on Layer 4, while in the backwards pass the errors are propagated and the antecedents parameters are updated by gradient descent.

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 10

Performance Evaluation

RMSE AND mRT VALUES FOR WORKLOAD DEMAND SCHEMES SIMILAR TO THE TRAINING SCHEME.

Gradual workload changes: The CPU allocation and mRT response of the fuzzy controller when evaluated with data (dataset 2) that is different than the training data, but follow the same % CPU demand.

Τhe fuzzy controller was evaluated against data that fluctuate around the training data. The Kalman and the H∞ filters were not able to follow the required demand (abrupt changes), increasing the mRT considerably. The neuro-fuzzy controller is trained on the response of the Kalman and H∞ filter and

• utperforms both on the same exact data.
• A. Gradual workload changes

The neuro-fuzzy controller responds a bit better by allocating slightly more resources than the other controllers, thus reducing the mRT overshoot.

10 20 30 40 50 60 70 80 90 100 20 40 60 80 100 time interval CPU (%) Fuzzy controller H∞ filter Kalman filter 10 20 30 40 50 60 70 80 90 100 20 40 60 80 100 120 time interval mRT (s) Fuzzy controller H∞ filter Kalman filter

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 11

Performance Evaluation

Saw-tooth workload changes: The CPU allocation and mRT response

• f the proposed controller when evaluated with data (dataset 4) that is

different than the % CPU demand in the training data.

The utilization changes rapidly from very large to very small values. In this case it is important for the controller to adapt the allocations in a timely fashion. To achieve good performance the error during contention should be minimized. The neuro-fuzzy controller is able to capture some nonlinearities that the Kalman and H∞ filters cannot due to their linear nature.

• B. Saw-tooth demand for CPU usage

The neuro-fuzzy controller has worse RMSE, but

• utperforms both the Kalman and H∞ filters when

comparing the mRT, which is the main metric in server consolidation, even if the noise is normally distributed, due to the good adaptation in sudden changes.

10 20 30 40 50 60 70 80 90 100 5 10 15 20 25 30 time interval mRT (s) Fuzzy controller H∞ filter Kalman filter 10 20 30 40 50 60 70 80 90 100 20 30 40 50 60 70 80 90 time interval CPU (%) Fuzzy controller H∞ filter Kalman filter

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 12

Conclusions and Future Work

Conclusions:

• Important to build controllers that adjust the allocation and avoid saturation of

the available resources.

• Neuro-fuzzy controller
• Better control performance than the conventional Kalman and H∞ filters that

work well only if the model assumptions of the system hold.

• No assumption on noise characteristics, hence it is more robust.
• No precise mathematical model required.
• Works well for complex applications provided the neuro-fuzzy controller is

properly trained. Future work/Ongoing research:

• Build a neuro-fuzzy controller that will consider the coupling between the

workload of different servers.

• Controller evaluation against complex consolidation scenarios in modern multi

core systems with the latest resource schedulers

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 13

Thank you! Questions?

On the use of Fuzzy Logic Controllers to Comply with Virtualized Application Demands in the Cloud 14

The H∞ Controller

We formulate the allocation problem as a state estimation problem. H∞ filters minimize the worst-case estimation error and can be used to incorporate more robustness into the state estimation problem. The cost function for our formulation is given by:

J = PN−1

k=0 kak ˆ

akk2

2

ka0 ˆ a0k2

P −1

+ PN−1

k=0

⇣ kwkk2

Q−1

k

+ kvkk2

R−1

k

⌘

Kk = Pk[I − θPk + CT R−1

k CPk]−1CT R−1 k

ˆ ak+1 = ˆ ak + Kk(uk − Cˆ ak) Pk+1 = Pk[I − θPk + CT R−1

k CPk]−1 + Qk

where Kk is the gain matrix and Pk is the error covariance matrix.

P0 ∈ RN×N, Qk ∈ RN×N and Rk ∈ RN×N

For achieving a mRT less than some pre-specified value (hence J<1/θ), the H∞ filter is given by: where are symmetric, positive definite matrices defined by the problem specifications.