Market Driven Multi Resource Allocation Liran Funaro Under the - - PowerPoint PPT Presentation

▶
market driven multi resource allocation
SMART_READER_LITE
LIVE PREVIEW

Market Driven Multi Resource Allocation Liran Funaro Under the - - PowerPoint PPT Presentation

Jul 26, 2023 267 likes •654 views

Market Driven Multi Resource Allocation Liran Funaro Under the supervision of Prof. Assaf Schuster and Dr. Orna Agmon Ben-Yehuda Department of Computer Science Cost Efficient Scaling Jan. 26, 2020 L. Funaro (Technion) Market Driven Multi

slide-1
SLIDE 1

Market Driven Multi Resource Allocation

Liran Funaro

Under the supervision of Prof. Assaf Schuster and Dr. Orna Agmon Ben-Yehuda Department of Computer Science Cost Efficient Scaling

  • Jan. 26, 2020
  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 1 / 26

slide-2
SLIDE 2

Resource Allocation

◮ One of the main challenges of public and private cloud providers ◮ Needs to serve all the clients on each server according to their their service-level-agreement (SLA) ◮ Affects utilization

◮ Hence, affects the number of clients per server ◮ Thus, affects the provider’s operation cost per client

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 2 / 26

slide-3
SLIDE 3

Resource Allocation

◮ One of the main challenges of public and private cloud providers ◮ Needs to serve all the clients on each server according to their their service-level-agreement (SLA) ◮ Affects utilization

◮ Hence, affects the number of clients per server ◮ Thus, affects the provider’s operation cost per client

◮ Can reduce the provider’s operation costs ◮ Coupled with a fitting pricing scheme, it can increase the provider’s profits

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 2 / 26

slide-4
SLIDE 4

Our Goals

◮ Explore designs for such resource allocation schemes

◮ Increase the resource utilization ◮ Taking into account the financial needs of providers and clients

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 3 / 26

slide-5
SLIDE 5

Our Goals

◮ Explore designs for such resource allocation schemes

◮ Increase the resource utilization ◮ Taking into account the financial needs of providers and clients ◮ That is, design a market driven resource allocation scheme

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 3 / 26

slide-6
SLIDE 6

Our Goals

◮ Explore designs for such resource allocation schemes

◮ Increase the resource utilization ◮ Taking into account the financial needs of providers and clients ◮ That is, design a market driven resource allocation scheme

◮ What is the gap between the current resource utilization to an optimal one? ◮ What is the origin of the gap? ◮ What are the provider’s economic requirements? ◮ What are the clients’ economic requirements?

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 3 / 26

slide-7
SLIDE 7

The Problem: Fixed Resource Bundles

◮ Resources in the cloud are underutilized ◮ The main cause of resource underutilization is fixed performance bundles ◮ Clients rent the resources to sustain their highest workload

◮ But they do not use the resources all the time

◮ The provider guarantees with good probability that the clients will be able to use their rented resources at any given time ◮ It must reserve these resources

◮ It cannot resell them or use them to other purposes

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 4 / 26

slide-8
SLIDE 8

Our Approach

◮ Incentivizing clients to reduce their fixed reserved resource requirements

◮ With an option to add resources on the fly on demand

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 5 / 26

slide-9
SLIDE 9

Our Approach

◮ Incentivizing clients to reduce their fixed reserved resource requirements

◮ With an option to add resources on the fly on demand

◮ How? By designing an allocation mechanisms that incorporates a smart pricing scheme

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 5 / 26

slide-10
SLIDE 10

Our Mechanisms

Two different mechanisms, each is suitable for different goals ◮ Auction-based mechanism: optimizes the clients’ economic benefit

Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Ginseng: market-driven LLC allocation”. In: Proceedings of the 2016 USENIX Conference on Usenix Annual Technical Conference. USENIX Association. ACM, 2016, pp. 295–308

◮ Stochastic allocation mechanism: allocate a stochastic amount of resources alongside a fixed, reserved, amount

Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Stochastic Resource Allocation”. In: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’19). USENIX Association. Providence, RI, USA: ACM, 2019. ISBN: 978-1-4503-6020-3/19/04

Both mechanisms improve hardware utilization by using some kind of economic mechanism that incentivize clients to reduce the fixed, reserved, portion of their bundle

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 6 / 26

slide-11
SLIDE 11

Auction-Based Mechanism

◮ Auction mechanism that optimize the social welfare

◮ The aggregated value all the clients draw from the cloud

◮ Each client rents a base resource bundle that is reserved for it

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 7 / 26

slide-12
SLIDE 12

Auction-Based Mechanism

◮ Auction mechanism that optimize the social welfare

◮ The aggregated value all the clients draw from the cloud

◮ Each client rents a base resource bundle that is reserved for it ◮ Then...

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 7 / 26

slide-13
SLIDE 13

Auction Protocol

The host announces an auction every few seconds

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 8 / 26

slide-14
SLIDE 14

Auction Protocol

The host announces an auction every few seconds Each guest bids with a valuation for each quantity of additional resource — how much it is worth, subjectively

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 8 / 26

slide-15
SLIDE 15

Auction Protocol

The host announces an auction every few seconds Each guest bids with a valuation for each quantity of additional resource — how much it is worth, subjectively The host solves an optimization problem: finding the allocation that maximize the social welfare

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 8 / 26

slide-16
SLIDE 16

Auction Protocol

The host announces an auction every few seconds Each guest bids with a valuation for each quantity of additional resource — how much it is worth, subjectively The host solves an optimization problem: finding the allocation that maximize the social welfare The host informs the guests of their allocation and charges them according to the exclusion-compensation principle

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 8 / 26

slide-17
SLIDE 17

Exclusion-Compensation Principle

◮ Exclusion-Compensation Principle: Each guest pays for the damage it inflicted on the other guests in the system ◮ If the demand is low, clients can rent the additional resources in a very low price, which is financially beneficial to them ◮ It incentivizes clients to rent a smaller bundle because the client can bid for additional resources at a lower price on average compared to the reservation price

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 9 / 26

slide-18
SLIDE 18

Auction-Based Mechanism (2)

◮ First introduced by Orna Agmon Ben-Yehuda for RAM allocation

Orna Agmon Ben-Yehuda et al. “Ginseng: Market-driven Memory Allocation”. In: Proceedings of the 10th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE). vol. 49. 7. Salt Lake City, Utah, USA: ACM,

  • 2014. ISBN: 978-1-4503-2764-0

◮ We extended this mechanism to last-level-cache (LLC) allocation

Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Ginseng: market-driven LLC allocation”. In: Proceedings of the 2016 USENIX Conference on Usenix Annual Technical Conference. USENIX Association. ACM, 2016, pp. 295–308

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 10 / 26

slide-19
SLIDE 19

Auction-Based Mechanism (2)

◮ First introduced by Orna Agmon Ben-Yehuda for RAM allocation

Orna Agmon Ben-Yehuda et al. “Ginseng: Market-driven Memory Allocation”. In: Proceedings of the 10th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE). vol. 49. 7. Salt Lake City, Utah, USA: ACM,

  • 2014. ISBN: 978-1-4503-2764-0

◮ We extended this mechanism to last-level-cache (LLC) allocation

Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Ginseng: market-driven LLC allocation”. In: Proceedings of the 2016 USENIX Conference on Usenix Annual Technical Conference. USENIX Association. ACM, 2016, pp. 295–308

◮ Our mechanism can improve the aggregate benefit of the clients in a single physical machine ◮ Guests can utilize their cache fast enough to allow rapid changes in the allocation

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 10 / 26

slide-20
SLIDE 20

New Challenges

High computational complexity Memory elastic applications

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 11 / 26

slide-21
SLIDE 21

Efficient Auction Algorithm

◮ Finding the optimal allocation has a high computational complexity

◮ Forces a long time period between auctions—more than an hour ◮ For multi resource: RAM, LLC, CPU, BW, etc.

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 12 / 26

slide-22
SLIDE 22

Efficient Auction Algorithm

◮ Finding the optimal allocation has a high computational complexity

◮ Forces a long time period between auctions—more than an hour ◮ For multi resource: RAM, LLC, CPU, BW, etc.

◮ We introduced a new efficient multi-resource auction algorithm with a pseudo near linear complexity

Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Efficient Multi-Resource, Multi-Unit VCG Auction”. In: Proceedings of the 16th International Conference on Economics of Grids, Clouds, Systems, and Services (GECON ’19). Springer,

  • Sept. 2019

◮ Allow a multi-resource auction every two minutes for up to 256 clients

◮ On a single core ◮ Embarrassingly parallel

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 12 / 26

slide-23
SLIDE 23

New Challenges

High computational complexity Memory elastic applications

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 13 / 26

slide-24
SLIDE 24

Memory Elastic Applications

◮ Resource elastic applications performance is proportional to the resource availability ◮ For example, cache elasticity

◮ Looks similar for CPU and BW

2 4 6 8 10 12 14 16 18 Cache Way Allocation (~1.5 MB each) 1.0 1.5 2.0 2.5 3.0 Application Performance

Dense-LU FFT Composite BZIP2 H.264 PostgreSQL

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 14 / 26

slide-25
SLIDE 25

Memory Elastic Applications

◮ Resource elastic applications performance is proportional to the resource availability ◮ For example, cache elasticity

◮ Looks similar for CPU and BW

2 4 6 8 10 12 14 16 18 Cache Way Allocation (~1.5 MB each) 1.0 1.5 2.0 2.5 3.0 Application Performance

Dense-LU FFT Composite BZIP2 H.264 PostgreSQL

◮ Memory elastic applications are scarce ◮ Usually looks like this (thrashing)

400 500 600 700 800 Memory (MB) 0K 1K 2K 3K Hits per Second

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 14 / 26

slide-26
SLIDE 26

Where are the Memory Elastic Applications?

◮ Why do developers toil towards making performance scale nicely with the CPU and bandwidth, but neglect doing this for memory?

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 15 / 26

slide-27
SLIDE 27

Circular Dependency

Making applications memory elastic Developing memory elasticity systems E v a l u a t i

  • n

r e q u i r e s b e n c h m a r k s P u t t i n g e f f

  • r

t r e q u i r e s i n c e n t i v e ◮ A proof that memory-elastic applications exist or can be created is essential to break this circular dependency

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 16 / 26

slide-28
SLIDE 28

Applications with Resource Trade-off

◮ Mechanisms that were designed to allow trade-off between memory and other resources can be used to provide memory elasticity

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 17 / 26

slide-29
SLIDE 29

Memory as Cache

◮ Some applications use the RAM to cache computation results, network traffic, and so on (e.g., using memcached) ◮ They can seamlessly improve their performance when more memory is available to the operating system

1024 1536 2048 2560 3072 3584 Memory (MB) 0K 20K 40K 60K Items/second

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 18 / 26

slide-30
SLIDE 30

Intermediate Calculations

◮ Applications that handles data that is larger than the available memory

◮ E.g., databases, Spark (map/reduce), deep neural network

◮ Can use larger memory buffers to reduce secondary memory access and speed up temporarily data-heavy operations

◮ E.g., sorting, large matrix multiplication, deep neural network data propagation

1000 2000 3000 4000 5000 6000 Memory (MB) 500 1000 1500 Transactions per Second (TPS)

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 19 / 26

slide-31
SLIDE 31

Garbage Collected Memory

◮ Applications with automatic memory management (e.g., Java applications)

◮ May need fewer garbage-collection cycles with a larger heap, and improve their performance as depicted in Figure 3.

100 200 300 400 500 600 Maximal Heap Size (MB) 10 20 30 Executions per Minute

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 20 / 26

slide-32
SLIDE 32

Multiple Short-Lived Jobs

◮ Some applications have multiple short-lived jobs, each with different memory requirements

◮ For example, web servers might require a certain memory to handle each session ◮ They may be able to handle more concurrent sessions when more memory is available

◮ To deal with lack of memory, they can cap the number of concurrent sessions

◮ They trade off memory for latency and throughput

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 21 / 26

slide-33
SLIDE 33

New Challenges

High computational complexity Memory elastic applications

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 22 / 26

slide-34
SLIDE 34

Auction Drawbacks

◮ Requires a large and constant amount of computational power

◮ Could have been allocated to the clients

◮ Does not take the provider’s profits into account

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 23 / 26

slide-35
SLIDE 35

Stochastic Allocation (SA)

◮ Under the SA mechanism, the provider offers clients a combination:

◮ an amount of reserved resources ◮ with a choice of a stochastic allocation class

◮ The provider posts fixed unit-prices for both goods ◮ And periodically publishes statistics on resource availability for each SA class ◮ Each client may choose to rent reserved and/or stochastic resources

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 24 / 26

slide-36
SLIDE 36

Conclusions

◮ We developed new market-driven resource allocation schemes ◮ We showed how they can improve

◮ Financial properties: social welfare and profits ◮ Technical properties: resource utilization and the number of clients per server

◮ We designed, developed and implemented as open source frameworks

◮ Algorithms to support these mechanisms ◮ Rigorous evaluation methodologies

◮ Our work has demonstrated how sophisticated allocation and pricing mechanisms can improve hardware—and thus energy—efficiency in the cloud significantly ◮ We believe that more research on resource allocation mechanisms in the cloud would help cloud providers immensely

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 25 / 26

slide-37
SLIDE 37

◮ Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Ginseng: market-driven LLC allocation”. In: Proceedings of the

2016 USENIX Conference on Usenix Annual Technical Conference. USENIX Association. ACM, 2016, pp. 295–308

◮ Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Stochastic Resource Allocation”. In: Proceedings of the 15th ACM

SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’19). USENIX Association. Providence, RI, USA: ACM, 2019. ISBN: 978-1-4503-6020-3/19/04

◮ Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Efficient Multi-Resource, Multi-Unit VCG Auction”. In:

Proceedings of the 16th International Conference on Economics of Grids, Clouds, Systems, and Services (GECON ’19). Springer,

  • Sept. 2019

◮ Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Memory Elasticity Benchmark”. In: 2019

Questions?

Liran Funaro: funaro@cs.technion.ac.il

  • L. Funaro (Technion)

Market Driven Multi Resource Allocation Cost Efficient Scaling 26 / 26