Combinatorial Auction-Based Allocation of Virtual Machine Instances - - PowerPoint PPT Presentation

Combinatorial Auction-Based Allocation of Virtual Machine Instances in Clouds

Sharrukh Zaman and Daniel Grosu

Department of Computer Science Wayne State University Detroit, Michigan 48202, USA sharrukh@wayne.edu, dgrosu@cs.wayne.edu

CloudCom 2010

December 01, 2010

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Outline

1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Outline

1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Fixed-price Mechanisms (e.g., Windows Azure)

Pay fixed price for resource usage Not economically efficient

Cannot guarantee that the user who values a bundle of VM instances the most, gets it

Cannot guarantee optimal revenue

What if some users want to pay more? Or less?

Cannot distribute loads evenly over time

You pay the same price at 9am and at 2am!

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Auctions in Clouds

Example: Amazon EC2 Spot Instances Employed to sell unused resources All winning bidders pay the same (per unit) price Not a combinatorial auction

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Combinatorial Auctions in Practice

Combinatorial auctions: Auctions where users bid for bundles of items and bundles are considered as units of allocations. Enable bidders to express their valuations in a more meaningful way Proved more efficient where bundles are involved Successful implementations: selling wireless spectrum, transportation procurement

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Combinatorial Auctions in Clouds

Different types of resources (virtual machines) available Users request for combination (bundle) of VM instances Bidding for bundle is more realistic Available combinatorial auction mechanisms do not apply directly to VM instance allocation problem Our goal: Design suitable combinatorial auction-based mechanisms for allocating VM instances in clouds

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Outline

1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Model

Resources to allocate m virtual machine types, VM1, . . . , VMm k instances of each VM type w1, . . . , wm: computing power of VMs (relative) Assumption: w1 = 1, w1 ≤ . . . ≤ wm

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Model (Cont.)

Auction participants n users, u1, . . . , un User uj bids Bj = (rj

1, . . . , rj m, vj)

rj

i : Number of VMi instances requested by uj

vj: user uj’s valuation for the bundle (rj

1, ..., rj m)

Maximum willingness to pay for the bundle Value she ‘gets’ if the bundle is allocated

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Bidders

Assumption: The bidders are single-minded. Valuation function of a single-minded bidder for any bundle S: v(S) = vj if Sj ⊆ S

• therwise

Sj = bundle requested vj = valuation of Sj

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Virtual Machine Allocation Problem (VMAP)

Determine the set of winning users and the prices they need to pay W : Set of winners

uj ∈ W means user uj gets her requested bundle

p = (p1, . . . , pn): Price each user pays to auctioneer

0 ≤ pj ≤ vj, if uj ∈ W pj = 0, if uj / ∈ W

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Characteristics of a Good Solution

Efficient: Maximize social welfare max

• uj∈W

vj Truthful: Reporting true value maximizes user’s utility

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Characteristics of a Good Solution

Efficient: Maximize social welfare max

• uj∈W

vj Truthful: Reporting true value maximizes user’s utility

Utility: Net value obtained Uj(W , pj) =

• vj − pj,

if uj ∈ W 0,

• therwise

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Challenges

Winner determination in CA is NP-hard Not all approximate solutions guarantee truthfulness Existing CA mechanisms cannot solve VMAP directly Need to extend existing mechanisms or design new mechanisms for solving VMAP

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Outline

1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism

A linear programming-based approximation mechanism CA-LP: generalizes the mechanism proposed in [Archer et al 2005]

Archer et al mechanism: bidders can request at most one copy

• f each item

CA-LP: bidders can request multiple copies of each item

Truthful in expectation (as the original mechanism)

Reference

• A. Archer, C. Papadimitriou, K. Talwar, and E. Tardos, An

approximate truthful mechanism for combinatorial auctions with single parameter agents, Internet Mathematics, vol. 1, no. 2, pp. 129-15, 2005.

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 1

Collect bids Bj = (rj

1, . . . , rj m, vj) from users

j rj

1

rj

2

vj xj yj Winner Price 1 2 4 3.74 1.00 0.33 Y 3.05 2 4 7.46 1.00 0.65 Y 6.10 3 4 2 6.11 0.50 0.00 N 0.00 4 2 1 1.38 0.00 0.24 N 0.00 5 4 2 1.60 0.00 0.65 N 0.00 6 3 4 2.05 0.00 0.28 N 0.00

CA-LP Example

Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 1

Collect bids Bj = (rj

1, . . . , rj m, vj) from users

j rj

1

rj

2

vj xj yj Winner Price 1 2 4 3.74 1.00 0.33 Y 3.05 2 4 7.46 1.00 0.65 Y 6.10 3 4 2 6.11 0.50 0.00 N 0.00 4 2 1 1.38 0.00 0.24 N 0.00 5 4 2 1.60 0.00 0.65 N 0.00 6 3 4 2.05 0.00 0.28 N 0.00

CA-LP Example

Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 1

Collect bids Bj = (rj

1, . . . , rj m, vj) from users

j rj

1

rj

2

vj xj yj Winner Price 1 2 4 3.74 1.00 0.33 Y 3.05 2 4 7.46 1.00 0.65 Y 6.10 3 4 2 6.11 0.50 0.00 N 0.00 4 2 1 1.38 0.00 0.24 N 0.00 5 4 2 1.60 0.00 0.65 N 0.00 6 3 4 2.05 0.00 0.28 N 0.00

CA-LP Example

Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 2

Determine winners

Solve linear program max

n

• j=1

xjvj subject to

n

• j=1

xjr j

i ≤ k′

i = 1, . . . m 0 ≤ xj ≤ 1 j = 1, . . . n where k′ = (1 − ǫ)k

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 2

Determine winners

Solve linear program Sort users by descending order of xj

j rj

1

rj

2

vj xj yj Winner Price 1 2 4 3.74 1.00 0.33 Y 3.05 2 4 7.46 1.00 0.65 Y 6.10 3 4 2 6.11 0.50 0.00 N 0.00 4 2 1 1.38 0.00 0.24 N 0.00 5 4 2 1.60 0.00 0.65 N 0.00 6 3 4 2.05 0.00 0.28 N 0.00

CA-LP Example

Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 2

Determine winners

Solve linear program Sort users by descending order of xj Generate random numbers, yj

j rj

1

rj

2

vj xj yj Winner Price 1 2 4 3.74 1.00 0.33 Y 3.05 2 4 7.46 1.00 0.65 Y 6.10 3 4 2 6.11 0.50 0.00 N 0.00 4 2 1 1.38 0.00 0.24 N 0.00 5 4 2 1.60 0.00 0.65 N 0.00 6 3 4 2.05 0.00 0.28 N 0.00

CA-LP Example

Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 2

Determine winners

Solve linear program Sort users by descending order of xj Generate random numbers, yj uj is a winner if xj ≥ yj and resources exist

j rj

1

rj

2

vj xj yj Winner Price 1 2 4 3.74 1.00 0.33 Y 3.05 2 4 7.46 1.00 0.65 Y 6.10 3 4 2 6.11 0.50 0.00 N 0.00 4 2 1 1.38 0.00 0.24 N 0.00 5 4 2 1.60 0.00 0.65 N 0.00 6 3 4 2.05 0.00 0.28 N 0.00

CA-LP Example

Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 2

Determine winners

Solve linear program Sort users by descending order of xj Generate random numbers, yj uj is a winner if xj ≥ yj and resources exist

j rj

1

rj

2

vj xj yj Winner Price 1 2 4 3.74 1.00 0.33 Y 3.05 2 4 7.46 1.00 0.65 Y 6.10 3 4 2 6.11 0.50 0.00 N 0.00 4 2 1 1.38 0.00 0.24 N 0.00 5 4 2 1.60 0.00 0.65 N 0.00 6 3 4 2.05 0.00 0.28 N 0.00

CA-LP Example

Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-LP Mechanism: Step 3

Calculate payments

Find v ′

j for which xj < yj

pj = v ′

j if uj ∈ W , 0 otherwise

j rj

1

rj

2

vj xj yj Winner Price 1 2 4 3.74 1.00 0.33 Y 3.05 2 4 7.46 1.00 0.65 Y 6.10 3 4 2 6.11 0.50 0.00 N 0.00 4 2 1 1.38 0.00 0.24 N 0.00 5 4 2 1.60 0.00 0.65 N 0.00 6 3 4 2.05 0.00 0.28 N 0.00

CA-LP Example

Setting: two types of VM instances (m = 2), six users (n = 6), and eight instances of each type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Outline

1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-GREEDY Mechanism

A greedy-based approximation mechanism CA-GREEDY: generalizes the mechanism proposed in [Lehman et al 2002]

CA-GREEDY: considers the weighted total of number of VM instances when computing the allocation

Truthful Computes a √ M-approximate solution to VMAP, where M = k m

i=1 wi

Reference

• D. Lehmann, L. I. OCallaghan, and Y. Shoham, Truth revelation in

approximately efficient combinatorial auctions, Journal of the ACM,

• vol. 49, no. 5, pp. 577-602, 2002.

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-GREEDY Mechanism: Step 1

Collect bids Bj = (rj

1, . . . , rj m, vj) from users

j rj

1

rj

2

vj √sj vj/ √sj Winner Price 1 2 4 3.74 3.16 1.18 N 0.00 2 4 7.46 2.00 3.73 Y 2.36 3 4 2 6.11 2.82 2.16 Y 3.34 4 2 1 1.38 2.00 0.69 N 0.00 5 4 2 1.60 2.82 0.56 N 0.00 6 3 4 2.05 3.31 0.61 N 0.00

CA-GREEDY Example

Setting: two types of VM instances (m = 2) with one and two processors (w = (1, 2)), six users (n = 6), and eight instances of each VM type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-GREEDY Mechanism: Step 2

Determine winners

Weighted sum of VM of each bid: sj = m

i=1 r j i wi

Bid density: vj/ √sj

j rj

1

rj

2

vj √sj vj/ √sj Winner Price 1 2 4 3.74 3.16 1.18 N 0.00 2 4 7.46 2.00 3.73 Y 2.36 3 4 2 6.11 2.82 2.16 Y 3.34 4 2 1 1.38 2.00 0.69 N 0.00 5 4 2 1.60 2.82 0.56 N 0.00 6 3 4 2.05 3.31 0.61 N 0.00

CA-GREEDY Example

Setting: two types of VM instances (m = 2) with one and two processors (w = (1, 2)), six users (n = 6), and eight instances of each VM type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-GREEDY Mechanism: Step 2

Determine winners

Weighted sum of VM of each bid: sj = m

i=1 r j i wi

Bid density: vj/ √sj Sort bids by density, allocate highest to lowest, until resources exhaust

j rj

1

rj

2

vj √sj vj/ √sj Winner Price 2 4 7.46 2.00 3.73 Y 2.36 3 4 2 6.11 2.82 2.16 Y 3.34 1 2 4 3.74 3.16 1.18 N 0.00 4 2 1 1.38 2.00 0.69 N 0.00 6 3 4 2.05 3.31 0.61 N 0.00 5 4 2 1.60 2.82 0.56 N 0.00

CA-GREEDY Example

Setting: two types of VM instances (m = 2) with one and two processors (w = (1, 2)), six users (n = 6), and eight instances of each VM type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

CA-GREEDY Mechanism: Step 3

Calculate payments

Find uj′ that would win if uj would not participate pj = √sj ∗ (vj′/ √sj′), if uj ∈ W pj = 0 otherwise

j rj

1

rj

2

vj √sj vj/ √sj Winner Price 2 4 7.46 2.00 3.73 Y 2.36 3 4 2 6.11 2.82 2.16 Y 3.34 1 2 4 3.74 3.16 1.18 N 0.00 4 2 1 1.38 2.00 0.69 N 0.00 6 3 4 2.05 3.31 0.61 N 0.00 5 4 2 1.60 2.82 0.56 N 0.00

CA-GREEDY Example

Setting: two types of VM instances (m = 2) with one and two processors (w = (1, 2)), six users (n = 6), and eight instances of each VM type (k = 8)

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Outline

1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Fixed-Price Mechanism

Collect bids Bj = (rj

1, . . . , rj m, vj) from users

Allocate bundles

(f1, . . . , fm): Fixed prices for VM1, . . . , VMm Allocate bundle, on a first-come-first-served basis if valuation meets predetermined fixed price vj ≥

m

• i=1

r j

i fi

Calculate payments

User pays the predetermined fixed price if she gets the bundle pj =

m

• i=1

r j

i fi

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Outline

1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Simulation Parameters

Parameter Description / Values Total time Five simulation days Auction frequency Every hour Users Maximum of 100,000 users VM weights w = (1, 2, 4, 8) VM prices f = 0.12, 0.24, 0.48, 0.96 User types Type 1, 2, 3: high, medium, low valuation Bidding A user needs to win tj times to complete job

Simulation Parameters

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Comparison of Overall Performance

Served Revenue Utilization Time 1 2 3 4 5 6 7 User and System Parameters (log10 scale) FIXED-PRICE CA-LP CA-GREEDY 5.53% 8.01% 5.60% 116,750 272,556 221,098 54.63% 94.70% 91.86% 1 sec 837 sec 26 sec

Overall performance of the mechanisms

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Types of Users Completing Jobs

2 4 6 8 10 12 Type 1 Type 2 Type 3 Percent Served User Type Users Completing Tasks FIXED-PRICE CA-LP CA-GREEDY

Percentage of served users

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Varying Valuation Ranges - 1

100000 200000 300000 400000 500000 600000 700000 0-1 0-2 0-5 0-10 Revenue Range of Valuations (min-max) Revenue vs. Valuation Ranges FIXED-PRICE CA-LP CA-GREEDY

Generated revenue vs. ranges of valuation

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Varying Valuation Ranges - 2

20 40 60 80 100 0-1 0-2 0-5 0-10 VM Utilization (%) Range of Valuations (min-max) Resource Utilization vs. Valuation Ranges FIXED-PRICE CA-LP CA-GREEDY

Effect of valuation on machine utilization

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Outline

1 Motivation 2 Virtual Machine Allocation Problem 3 CA-LP: A Linear Programming-Based Mechanism 4 CA-GREEDY: A Greedy-Based Mechanism 5 A Fixed-Price Mechanism 6 Experimental Results 7 Conclusion Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Conclusion

Combinatorial auction mechanisms are more desirable than fixed-price mechanisms CA-LP has better performance over CA-GREEDY, but poor running time Use CA-GREEDY as a general solution - good running time, comparable performance

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds

Thank You! Questions?

Sharrukh Zaman, Wayne State University Combinatorial Auctions in Clouds