A Game-Theoretic Approach for Runtime Capacity Allocation in - - PowerPoint PPT Presentation

A Game-Theoretic Approach for Runtime Capacity Allocation in MapReduce

Eugenio Gianniti *, Danilo Ardagna *, Michele Ciavotta *, Mauro Passacantando * *

*Politecnico di Milano **Università di Pisa POLITECNICO DI MILANO Slide 1

Goals

POLITECNICO DI MILANO

Cost-effective deployment configuration for MapReduce systems hosted on private Clouds Efficient solution algorithm to support run-time cluster management Distributed approach leveraging on local knowledge of problem parameters

Slide 2

Reference System

POLITECNICO DI MILANO

Hadoop YARN

Slide 3

M M R R M

Preliminary Formulation

POLITECNICO DI MILANO

min

r,h,sM,sR N

X

i=1

¯ ρri +

N

X

i=1

Pi (hi)

N

X

i=1

ri ≤ R Hlow

i

≤ hi ≤ Hup

i ,

∀i ∈ A Aihi sM

i

+ Bihi sR

i

+ Ei ≤ 0, ∀i ∈ A sM

i

cM

i

+ sR

i

cR

i

≤ ri, ∀i ∈ A ri ∈ N0, ∀i ∈ A sM

i

∈ N0, ∀i ∈ A sR

i ∈ N0,

∀i ∈ A hi ∈ N0, ∀i ∈ A

subject to:

Integer Nonlinear Problem Non-convex constraints DECISION VARIABLES hi — concurrency level ri — virtual machines siM — Map slots siR — Reduce slots Continuous relaxation hi à (𝜔i)-1

Slide 4

Centralized Problem

POLITECNICO DI MILANO

Continuous Nonlinear Problem Need to centralize information Convex constraints KKT conditions are necessary and sufficient for optimality DECISION VARIABLES 𝜔i — reciprocal concurrency level ri — virtual machines siM — Map slots siR — Reduce slots

Slide 5

min

r,ψ,sM,sR N

X

i=1

¯ ρri +

N

X

i=1

(αiψi − βi)

subject to:

N

X

i=1

ri ≤ R ψlow

i

≤ ψi ≤ ψup

i ,

∀i ∈ A Ai sM

i ψi

+ Bi sR

i ψi

+ Ei ≤ 0, ∀i ∈ A sM

i

cM

i

+ sR

i

cR

i

≤ ri, ∀i ∈ A ri ∈ R+, ∀i ∈ A sM

i

∈ R+, ∀i ∈ A sR

i ∈ R+,

∀i ∈ A ψi ∈ R+, ∀i ∈ A

Optimal Configuration Formulae

POLITECNICO DI MILANO

In the optimal configuration it holds: sM

i

= cM

i

1 + r

Bi Ai cM

i

cR

i

ri, ∀i ∈ A sR

i =

cR

i

1 + r

Ai Bi cR

i

cM

i

ri, ∀i ∈ A ψi = − ⇣q

Ai cM

i

+ q

Bi cR

i

⌘2 Ei r−1

i

, ∀i ∈ A

PROPOSITION

rlow

i

= − ⇣q

Ai cM

i

+ q

Bi cR

i

⌘2 Eiψup

i

, ∀i ∈ A rup

i

= − ⇣q

Ai cM

i

+ q

Bi cR

i

⌘2 Eiψlow

i

, ∀i ∈ A Exact bounds on resources requirement Full knowledge of the problem parameters

Slide 6

¯ ρ ≤ ρa

i ≤ ρup i

ψlow

i

≤ ψi ≤ ψup

i

Ai sM

i ψi

+ Bi sR

i ψi

+ Ei ≤ 0 sM

i

cM

i

+ sR

i

cR

i

≤ ri sM

i

∈ R+ sR

i ∈ R+

ψi ∈ R+ ρa

i ∈ R+

min

ψi,ρa

i ,sM i ,sR i

αiψi − βi

subject to:

Application Masters Problems

POLITECNICO DI MILANO

One instance per AM Continuous Nonlinear Problem Only application-specific information DECISION VARIABLES 𝜔i — reciprocal concurrency level 𝜍ia — bid for VMs siM — Map slots siR — Reduce slots

Slide 7

max

r,y,ρ N

X

i=1

(ρ − ¯ ρ) ˜ ri −

N

X

i=1

pi (rup

i

− ri)

subject to:

N

X

i=1

ri ≤ R ri ≥ rlow

i

, ∀i ∈ A ri ≤

• rup

i

− rlow

i

• yi + rlow

i

, ∀i ∈ A ¯ ρ ≤ ρ ≤ ˜ ρ ρ − ρa

i ≤ M (1 − yi) ,

∀i ∈ A ρa

i − ρ ≤ Myi,

∀i ∈ A ri ∈ R+, ∀i ∈ A yi ∈ {0, 1}, ∀i ∈ A ρ ∈ R+

Resource Manager Problem

POLITECNICO DI MILANO

DECISION VARIABLES 𝜍 — price of VMs ri — virtual machines yi — AM i offers more than price One instance for the whole cluster Mixed Integer Nonlinear Problem Takes care of resource management only Two alternatives: and

˜ ri = ri ˜ ri = ri − rlow

i

Slide 8

Generalized Nash Equilibrium Problems

POLITECNICO DI MILANO

Not jointly convex Lack of theoretical guarantees

NEP JC-NEP GNEP

Slide 9

A set of players, N, each with utility function Θi Solution concept: ¯ x ∈ X is equilibrium ⇔ ∀i ∈ N, ∀xi ∈ Xi (¯ x−i) , Θi (¯ x) ≤ Θi (xi, ¯ x−i) Feasible set of player i: Xi = Xi (x−i)

Optimal Configuration Formulae

POLITECNICO DI MILANO

sM

i

= cM

i

1 + r

Bi Ai cM

i

cR

i

ri sR

i =

cR

i

1 + r

Ai Bi cR

i

cM

i

ri ψi = − ⇣q

Ai cM

i

+ q

Bi cR

i

⌘2 Ei r−1

i

The optimal configuration for every AM, given an amount of resources ri, is given by the following relations:

PROPOSITION

Each AM problem reduces to a quick algebraic update Local knowledge of application- specific parameters

Slide 10

Best Reply Algorithm

POLITECNICO DI MILANO

Each iteration is performed in parallel by the RM and AMs Continuous equilibrium configuration

1: ri ← rlow

i

, ∀i ∈ A

2: sM

i

← sM, low

i

, ∀i ∈ A

3: sR

i ← sR, low i

, ∀i ∈ A

4: ψi ← ψup

i , ∀i ∈ A

5: ρa

i ← ¯

ρ, ∀i ∈ A

6: repeat 7:

rold

i

← ri, ∀i ∈ A

8:

solve RM problem

9:

for all i ∈ A do

10:

solve AM i problem

11:

if ψi > ψlow

i

then

12:

ρa

i ← max {ρa i , ρ} + λρup i

13:

end if

14:

end for

15:

ε ← PN

i=1

|ri−rold

i |

rold

i

16: until ε < ¯

ε

Slide 11

Experimental Overview

POLITECNICO DI MILANO

PRELIMINARY ANALYSIS

Ensure the model behavior is consistent with intuition when applied to simple problems

SCALABILITY ANALYSIS

Verify the feasibility of solving problem instances at a realistic scale in production environments

STOPPING CRITERION TOLERANCE ANALYSIS

Check the sensitivity of the distributed algorithm with respect to the tolerance on the relative increment

VALIDATION WITH YARN SLS

Compare model solutions and timings measured on the official simulator

Slide 12

Preliminary Analysis

POLITECNICO DI MILANO

100 AMs 1,000 AMs ˜ ri = ri − rlow

i

Decreasing cluster capacity experiment,

Slide 13

Alternative Virtual Gain Terms

POLITECNICO DI MILANO

˜ ri = ri − rlow

i

˜ ri = ri

Increasing concurrency level experiment, 10 AMs

Slide 14

Scalability Analysis

POLITECNICO DI MILANO

Execution time [s]

˜ ri = ri − rlow

i

˜ ri = ri

Slide 15

Stopping Criterion Tolerance Analysis

POLITECNICO DI MILANO

Relative error with respect to centralized solutions

Slide 16

Validation with YARN SLS

POLITECNICO DI MILANO

Relative error absolute values average: 16.999 %

Users R Di [s] S [s] η [%] (20, 10, 10) 256 3090 2487.54 24.2191 (20, 10, 10) 512 1694 1142.3 48.2977 (20, 10, 20) 256 3380 2948.78 14.6237 (20, 10, 20) 512 1835 1299.59 41.1987 (20, 15, 10) 256 3390 2849.85 18.9536 (20, 15, 10) 512 1845 1409.88 30.8625 (20, 20, 10) 256 3700 3339.28 10.8023 (20, 20, 10) 512 1995 1512 31.9444 (20, 20, 15) 256 3845 3694.03 4.08677 (20, 20, 15) 512 2063 1745.9 18.1626 (20, 20, 20) 256 3987 4041.44

• 1.34704

(20, 20, 20) 512 2140 1877.4 13.9874 (25, 15, 25) 256 4300 3893.71 10.4346 (25, 15, 25) 512 2290 1773.53 29.1213 (25, 25, 25) 512 2597 2653.64

• 2.13455

Users R Di [s] S [s] η [%] (10, 15, 20) 256 2740 2767.58

• 0.996658

(10, 15, 20) 512 1508 1447.12 4.20698 (10, 20, 15) 256 2900 2708.3 7.07837 (10, 20, 15) 512 1589 1379.58 15.18 (15, 10, 10) 256 2575 2257.81 14.0487 (15, 10, 10) 512 1456 980.087 48.5583 (15, 15, 15) 256 3070 3183.02

• 3.55061

(15, 15, 15) 512 1678 1456.93 15.174 (15, 15, 20) 256 3210 3088.88 3.92116 (15, 15, 20) 512 1748 1547.92 12.9255 (15, 20, 10) 256 3220 3070.87 4.85639 (15, 20, 10) 512 1755 1328.58 32.0956 (15, 20, 20) 256 3512 3951.58

• 11.1242

(15, 20, 20) 512 1899 1574.1 20.6404 (15, 25, 10) 256 3520 3276.66 7.42646 (15, 25, 10) 512 1906 1524.83 24.9975

Slide 17

Conclusions and Future Work

POLITECNICO DI MILANO

The distributed approach yields accurate approximations of optimal solutions, even without theoretical guarantees Extend the formulation to Apache Tez and Spark (based on DAGs) Couple the proposed algorithm with a local search method based on Colored Petri Nets simulations

Slide 18

Thanks for your attention…

POLITECNICO DI MILANO

…any questions?

Slide 19

Solution Rounding Algorithm

POLITECNICO DI MILANO

1: sort A according to increasing αi 2: ri dˆ

rie , 8i 2 A

3: for all j 2 A do 4:

if PN

i=1 ri > R then

5:

rj rj 1

6:

end if

7: end for 8: sM

i

⌃ ˆ sM

i

⌥ , 8i 2 A

9: sR

i

⌃ ˆ sR

i

⌥ , 8i 2 A

10: for all j 2 A do 11:

while sM

j /cM j + sR j /cR j > rj do

12:

sR

j sR j 1

13:

if sM

j /cM j + sR j /cR j > rj then

14:

sM

j

sM

j 1

15:

end if

16:

end while

17: end for

The RM runs an O(N) loop Each AM runs an O(1) loop, concurrently Sorting is O(N log N), but can be done once and cached

Slide 20

Preliminary Analysis

POLITECNICO DI MILANO

˜ ri = ri − rlow

i

Decreasing deadlines experiment,

100 AMs 1,000 AMs

Slide 21

Scalability Analysis

POLITECNICO DI MILANO

Total cost and penalties

˜ ri = ri − rlow

i

˜ ri = ri

Slide 22

Obtained Concurrency Levels

POLITECNICO DI MILANO

Centralized model Closed form model

Decreasing cluster capacity experiment

Slide 23