Optimisation of virtual machine garbage collection policies ASMTA11 - - PowerPoint PPT Presentation

Optimisation of virtual machine garbage collection policies

ASMTA’11 Simonetta Balsamo Gian-Luca Dei Rossi Andrea Marin

Dipartimento di Scienze Ambientali, Informatica e Statistica Universit` a Ca’ Foscari, Venezia

June 22, 2011

Outline

• Context
• Prerequisites
• Model
• Examples
• Heuristic
• Conclusions

Optimisation of virtual machine garbage collection policies 2 of 20

Memory management in HLLs

Automatic memory management (Garbage Collection)

• Easier
• Safer
• Performance issues

Many different technologies

• Algorithms
• Number of phases
• Blocking activities (stop-the-world approach)
• Activation timings

Performance optimisation strategies:

• Changing algorithms or implementations
• Reducing blocking phases
• Tuning the activation timing
• Service time degradation vs. blocking activities

Optimisation of virtual machine garbage collection policies 3 of 20

Model: Assumptions

• Customers (reqs) arrival poisson process, parameter λ
• Scheduling discipline: Processor Sharing
• Memory divided in B blocks
• At each customer arrival, b blocks are allocated, according to a

discrete random variable probability distribution.

• Service rate µi depends on the number i of allocated memory blocks.
• Garbage collector is activated periodically (rate αi) or when the

memory is full.

• The garbage collector frees unused allocated memory blocks with

rate γi.

• During the garbage collection phase all services are suspended
• The garbage collector stops unconditionally after a random delay,

with rate βi.

• When the system is empty (no customer), the memory is freed

instantaneously.

Optimisation of virtual machine garbage collection policies 4 of 20

Model: states space

State: a triplet (c, i, g), where

• c is the number of customers in the system
• i is the number of allocated memory blocks
• g is the state of the GC: ON (active) or OFF (non active)

When there is no customer in the system, i.e., c = 0, memory is always completely unallocated and the garbage collector is inactive, i.e., i = 0, g = OFF. Formally E = (0, 0, OFF) ∪ {(c, i, g)|c ∈ N>0, i ∈ {1 . . . B}, g ∈ {ON, OFF}}. The model is a Quasi-Birth-Death Process and is solvable using a Matrix Geometric method [3, 2].

Optimisation of virtual machine garbage collection policies 5 of 20

Quasi-Birth-Death Processes

1 2 . . . λ′ λ λ µ′ µ µ

Q =           B00 B01 . . . B10 A1 A2 . . . A0 A1 A2 . . . A0 A1 A2 . . . ... ... ... . . . . . . . . . . . . . . . . . . . . .          

• States are grouped in levels
• Transitions are permitted only between states in the same level or in

adjacent levels.

• Levels can be represented by square matrices
• Transitions between levels are also represented by matrices
• After an optional initial phase, all levels and transitions have an

identical structure.

Optimisation of virtual machine garbage collection policies 6 of 20

Model: initial state

(1, 1, OFF) (1, 2, OFF) (1, B − 1, OFF) (1, B, OFF) (1, 1, ON) (1, 2, ON) (1, B − 1, ON) (1, B, ON) α1 α2 αB−1 αB βB−1 β2 β1 γ2 γB . . . γB−1 γ3 . . . . . . . . . βB (0, 0, OFF) µ1 µ2 µB−1 µB λ

Optimisation of virtual machine garbage collection policies 7 of 20

Model: regular block of states

(n, 1, OFF) (n, 2, OFF) (n, B − 1, OFF) (n, B, OFF) (n, 1, ON) (n, 2, ON) (n, B − 1, ON) (n − 1, 1, ON) (n − 1, 2, ON) (n − 1, B − 1, ON) (n − 1, B, ON) (n − 1, 1, OFF) (n − 1, 2, OFF) (n − 1, B − 1, OFF) (n − 1, B, OFF) (n + 1, 1, OFF) (n + 1, 2, OFF) (n + 1, B − 1, OFF) (n + 1, B, OFF) (n, B, ON) (n + 1, 1, ON) (n + 1, 2, ON) (n + 1, B − 1, ON) (n + 1, B, ON) α1 α2 αB−1 αB βB−1 β2 β1 γ2 γB . . . γB−1 γ3 . . . . . . . . . λ λ λ λ λ λ λ λ λ µB µ1 µ2 µB−1 βB . . . . . .

Optimisation of virtual machine garbage collection policies 8 of 20

Model: matrix representation

B0(i) =

• µ i+1

2

if i is odd

• therwise

B1(1) = −λ B2(j) = λ if j = 1

• therwise

A0(i, j) =

• µ i+1

2

if i = j and i, j are odd

• therwise

A1(i, j) =                  α i+1

2

if j = i + 1 ∧ i is odd β i

2

if j = i − 1 ∧ i is even γ i

2

if j = i − 2 ∧ i is even −

• ∀k=i

(A0(i, k) + A1(i, k) + A2(i, k)) if i = j

• therwise

(1) A2(i, j) =    λ if j = i + 2 λ if (i = 2B ∨ i = 2B − 1) ∧ j = 2B

• therwise

Optimisation of virtual machine garbage collection policies 9 of 20

Model: performance analysis

The model is a QBD processes

• Matrix Analytic Methods for steady state probabilities
• Closed forms for E[N] and E[R]
• More performance indices, e.g., GC overhead, using iteration.

Performance indices in function of a variable parameter

• How a performance index, e.g., the average response time, vary over

the GC activation rate?

• To simplify the examples, we assume αi = α, βi = β ∀i ∈ {1 . . . B}
• R(α): mean response time of the model as function of α
• Numerical search for a minimum

Optimisation of virtual machine garbage collection policies 10 of 20

Model: minimum search

Where to search for a minimum? Proposition If the optimisation problem α∗ = argminαR(α) admits a solution, then the following inequality holds: 0 < α∗ < (β + γ)(µ+ − λ) λ , where µ+ = maxi(µi), 0 ≤ i ≤ B. ATM no proof for minimum existence or uniqueness.

• Experimental evidence seems to suggest so

Optimisation of virtual machine garbage collection policies 11 of 20

Numerical Examples

Where not stated otherwise, the parameters are the following Parameter Name Value B 50 λ 3.0 βi 3 ∀i ∈ {1 . . . B} γi 25 ∀i ∈ {1 . . . B} µi 5.0 for 1 ≤ i ≤ B/2, 0.05 for B/2 < i ≤ B

Table: Parameter Values in Numerical Examples

Optimisation of virtual machine garbage collection policies 12 of 20

Numerical examples: R

0.8 0.82 0.84 0.86 0.88 0.9 0.92 0.94 0.96 0.98 1 1.78 1.8 1.82 1.84 1.86 1.88 1.9 1.92 1.94 1.96

Garbage Collector Activation Rate Mean Response Time

Figure: An example of the function R

Optimisation of virtual machine garbage collection policies 13 of 20

Numerical examples: effects of increasing customer arrivals

2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 1.05

Customer Arrival Rate Optimal Garbage Collector Activation Rate

Figure: Optimal α value in function of λ

Optimisation of virtual machine garbage collection policies 14 of 20

Numerical examples: utilisation

0.6 0.7 0.8 0.9 1 1.1 1.2 0.76 0.77 0.78 0.79 0.8 0.81 0.82 0.83 0.84 0.85

Garbage Collector Activation Rate Utilisation

Figure: ρ value in function of α

Optimisation of virtual machine garbage collection policies 15 of 20

A Heuristic for optimising GC Policies

Use the model to determine on flight the optimum garbage collector activation rates αi for a real system with GC in order to minimise the average response time. Suppose that we can measure:

• the number of available memory blocks ¯

B

• the number of free memory blocks f
• the average customer arrival rate ¯

λ

• the average service times ¯

µi

• the average rates at which the garbage collector can free a block of

memory ¯ γi

• the average rate at which the garbage collector returns control to

the program ¯ βi Where i = ¯ B − f is the number of allocated memory blocks. Easily measurable stats, e.g., using the

• verbose:gc option of the Sun

JVM.

Optimisation of virtual machine garbage collection policies 16 of 20

Heuristic: optimisation of α

Parameter α is unbound.

• We can solve the optimisation problem α∗ = argminαR(α)
• We can also solve the problem over a function of α that determines

the values of αi ∀i

• Other performance indices can be optimised using stationary

probabilities. Once determined the optimum α value, this is set as the activation rate for the GC of the real system. What if some measured values changes, e.g., ¯ λ, or more data is recovered, e.g., for new values of i?

• The model is parametrised again and new values for αi are

generated.

Optimisation of virtual machine garbage collection policies 17 of 20

Conclusions

• We have proposed a queueing model for systems with garbage

collection

• We have shown that the solution is numerically tractable
• We have proposed a heuristic for the optimisation of garbage

collection activation rates

• Easy to implement

Future works:

• Better validation of the model with experimental data
• Results in [1] seem to be coherent with results from our model.
• Comparison with traditional policies for garbage collection activation
• Energy-aware optimisation of the activation rate

Optimisation of virtual machine garbage collection policies 18 of 20

References

[1] Matthew Hertz and Emery D. Berger. Quantifying the performance of garbage collection vs. explicit memory management. SIGPLAN Not., 40(10):313–326, October 2005. [2] G. Latouche and V. Ramaswami. Introduction to Matrix Anlytic Methods in Stochastic Modeling. Statistics and applied probability. ASA-SIAM, Philadelphia, PA, 1999. [3] M. F. Neuts. Matrix Geometric Solutions in Stochastic Models. John Hopkins, Baltimore, Md, 1981.

Optimisation of virtual machine garbage collection policies 19 of 20

Any question?

Optimisation of virtual machine garbage collection policies 20 of 20

Stationary distribution of the model

The minimum of the utilisation ρ is different from the minimum of the average response time R. This behaviour can be explained by the fact that, for this model, given two distribution π and π′, after a certain k, π(k + i) < π′(k + i) for i ∈ N even if π(0) > π′(0).

Optimisation of virtual machine garbage collection policies 21 of 20

1 2 3 4 5 6 7 8 9 0.05 0.1 0.15 0.2 0.25

Number of Customers Stationary Probability α = 0.7 α = 0.9

Figure: Vector π for two different α values and λ = 3

Optimisation of virtual machine garbage collection policies 22 of 20