Memory Elasticity Benchmark Orna Agmon Ben-Yehuda Assaf Schuster - - PowerPoint PPT Presentation

▶
memory elasticity benchmark
SMART_READER_LITE
LIVE PREVIEW

Memory Elasticity Benchmark Orna Agmon Ben-Yehuda Assaf Schuster - - PowerPoint PPT Presentation

Nov 16, 2023 590 likes •1.02k views

Memory Elasticity Benchmark Orna Agmon Ben-Yehuda Assaf Schuster Liran Funaro Department of Computer Science SYSTOR2020 L. Funaro , O. Agmon Ben-Yehuda, A. Schuster (Technion) Memory Elasticity Benchmark 1 / 26 Improve Utilization L.

slide-1
SLIDE 1

Memory Elasticity Benchmark

Liran Funaro Orna Agmon Ben-Yehuda Assaf Schuster

Department of Computer Science

SYSTOR’2020

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 1 / 26

slide-2
SLIDE 2

Improve Utilization

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 2 / 26

slide-3
SLIDE 3

Improve Utilization

◮ Cloud providers aim to make more money off the same hardware

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 2 / 26

slide-4
SLIDE 4

Improve Utilization

◮ Cloud providers aim to make more money off the same hardware ◮ Rigid allocation prevents optimal resource utilization

Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Stochastic Resource Allocation”. In: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’19). USENIX Association. Providence, RI, USA: ACM, 2019. ISBN: 978-1-4503-6020-3/19/04

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 2 / 26

slide-5
SLIDE 5

Elastic Allocation

Introducing Amazon EC2 T3 Instances

Posted On: Aug 21, 2018 Amazon Web Services (AWS) is introducing the next generation Amazon Elastic Compute Cloud (EC2) burstable general-purpose instances, T3. T3 instances

  • ff er a balance of compute, memory, and network

resources and are designed to provide a baseline level

  • f CPU performance with the ability to burst above the

baseline when needed. T3 instances are powered by the AWS Nitro System which includes a lightweight hardware-accelerated hypervisor, delivering practically

 

https://aws.amazon.com/about-aws/whats-new/2018/08/introducing-amazon-ec2-t3-instances

◮ Burstable performance offers CPU elasticity ◮ Clients can "burst" to a higher level when required

◮ Allow changing resource consumption on the fly ◮ Exploiting resources that are momentarily unused by others

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 3 / 26

slide-6
SLIDE 6

Elastic Allocation

Introducing Amazon EC2 T3 Instances

Posted On: Aug 21, 2018 Amazon Web Services (AWS) is introducing the next generation Amazon Elastic Compute Cloud (EC2) burstable general-purpose instances, T3. T3 instances

  • ff er a balance of compute, memory, and network

resources and are designed to provide a baseline level

  • f CPU performance with the ability to burst above the

baseline when needed. T3 instances are powered by the AWS Nitro System which includes a lightweight hardware-accelerated hypervisor, delivering practically

 

https://aws.amazon.com/about-aws/whats-new/2018/08/introducing-amazon-ec2-t3-instances

◮ Burstable performance offers CPU elasticity ◮ Clients can "burst" to a higher level when required

◮ Allow changing resource consumption on the fly ◮ Exploiting resources that are momentarily unused by others

◮ More clients can be allocated to the same physical servers

Liran Funaro, Orna Agmon Ben-Yehuda, and Assaf Schuster. “Stochastic Resource Allocation”. In: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’19). USENIX Association. Providence, RI, USA: ACM, 2019. ISBN: 978-1-4503-6020-3/19/04

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 3 / 26

slide-7
SLIDE 7

Memory is the New Bottleneck

◮ Memory is the new bottleneck

◮ It is an expensive resource that limits machine occupancy

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 4 / 26

slide-8
SLIDE 8

Memory is the New Bottleneck

◮ Memory is the new bottleneck

◮ It is an expensive resource that limits machine occupancy

◮ Memory elasticity schemes should be a natural extension to CPU elasticity

◮ Allowing clients to use more memory in the same VM/container than their initial memory allocation

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 4 / 26

slide-9
SLIDE 9

Memory Elastic Applications

◮ Applications that can burst

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 5 / 26

slide-10
SLIDE 10

Memory Elastic Applications

◮ Applications that can burst

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 5 / 26

slide-11
SLIDE 11

Memory Elastic Applications

◮ Applications that can burst ◮ Whose performance is proportional to their memory usage

1024 1536 2048 2560 3072 3584 Memory (MB) 0K 20K 40K 60K Items/second

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 5 / 26

slide-12
SLIDE 12

Memory Elastic Applications Exists?

400 500 600 700 800 Memory (MB) 0K 1K 2K 3K Hits per Second

◮ Memory-elastic applications are scarce

◮ Maximal memory footprint is dictated by the current application workload

◮ The OS’s swapping allows seamless application operation

◮ Even a minor memory loss may degrade the performance significantly

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 6 / 26

slide-13
SLIDE 13

Where are the Memory Elastic Applications?

?

◮ Why most applications can scale with CPU? But not for memory?

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 7 / 26

slide-14
SLIDE 14

Where are the Memory Elastic Applications?

?

◮ Why most applications can scale with CPU? But not for memory? ◮ Multi-core architectures and CPU schedulers were the incentive

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 7 / 26

slide-15
SLIDE 15

Circular Dependency

Making applications memory elastic Developing memory elasticity systems E v a l u a t i

  • n

r e q u i r e s b e n c h m a r k s P u t t i n g e f f

  • r

t r e q u i r e s i n c e n t i v e

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 8 / 26

slide-16
SLIDE 16

Applications with Resource Trade-off

◮ Mechanisms that were designed to allow trade-off between memory and other resources can be used to provide memory elasticity

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 9 / 26

slide-17
SLIDE 17

Memory as Cache

Applications that use the RAM to cache computation results, network traffic, and so on (e.g., using Memcached) ◮ Improve cache hit-rate when more memory is available to the operating system

1024 1536 2048 2560 3072 3584 Memory (MB) 0K 20K 40K 60K Items/second

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 10 / 26

slide-18
SLIDE 18

Intermediate Buffers

Applications that use intermediate buffers (e.g., Hadoop, Spark) ◮ Can use larger memory buffers to reduce disk access and speed up temporarily data-heavy operations

◮ E.g., sorting and large matrix multiplication

1000 2000 3000 4000 5000 6000 Memory (MB) 1000 TPS

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 11 / 26

slide-19
SLIDE 19

Garbage Collected Memory

Applications with automatic memory management (e.g., Java applications) ◮ May need fewer garbage-collection cycles with a larger heap, and improve their performance

100 200 300 400 500 600 Maximal Heap Size (MB) 10 20 30 Executions per Minute

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 12 / 26

slide-20
SLIDE 20

Multiple Short-Lived Jobs

Applications that have multiple short-lived jobs, each with different memory requirements (e.g., Nginx) ◮ Web servers might require a certain memory to handle each session ◮ They may be able to handle more concurrent sessions when more memory is available

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 13 / 26

slide-21
SLIDE 21

Memory-Aware Applications

◮ Memory-aware applications adjust their memory consumption according to the available memory observed during their initiation period

◮ But cannot adjust it during runtime

◮ Most of the commonly used memory trade-offs we mentioned are predefined and implemented as memory-aware applications

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 14 / 26

slide-22
SLIDE 22

Memory-Aware Applications

◮ Memory-aware applications adjust their memory consumption according to the available memory observed during their initiation period

◮ But cannot adjust it during runtime

◮ Most of the commonly used memory trade-offs we mentioned are predefined and implemented as memory-aware applications ◮ Can be made memory-elastic by restarting them when the memory changes

◮ Not suitable when the application needs to be continuously available

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 14 / 26

slide-23
SLIDE 23

Memory-Aware Applications

◮ Memory-aware applications adjust their memory consumption according to the available memory observed during their initiation period

◮ But cannot adjust it during runtime

◮ Most of the commonly used memory trade-offs we mentioned are predefined and implemented as memory-aware applications ◮ Can be made memory-elastic by restarting them when the memory changes

◮ Not suitable when the application needs to be continuously available

◮ With a small effort, these applications can be tweaked to become memory-elastic

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 14 / 26

slide-24
SLIDE 24

Tweaked Memcached

Elastic memcached supports changing its memory footprint upon receiving a command via a socket

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 15 / 26

slide-25
SLIDE 25

Naive Metric

0.0 0.2 0.4 0.6 0.8 1.0 Time 5 10 15 20 Memory (GB) Allocation (GB)

◮ Compare the performance of two applications under the same dynamic memory conditions and consider the one with the better results as more memory-elastic

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 16 / 26

slide-26
SLIDE 26

Naive Metric

0.0 0.2 0.4 0.6 0.8 1.0 Time 5 10 15 20 Memory (GB) Allocation (GB)

◮ Compare the performance of two applications under the same dynamic memory conditions and consider the one with the better results as more memory-elastic ◮ The results may be sensitive to the order or frequency of memory allocations

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 16 / 26

slide-27
SLIDE 27

Naive Metric

0.0 0.2 0.4 0.6 0.8 1.0 Time 5 10 15 20 Memory (GB) Allocation (GB)

◮ Compare the performance of two applications under the same dynamic memory conditions and consider the one with the better results as more memory-elastic ◮ The results may be sensitive to the order or frequency of memory allocations

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 16 / 26

slide-28
SLIDE 28

Naive Metric

0.0 0.2 0.4 0.6 0.8 1.0 Time 5 10 15 20 Memory (GB) Allocation (GB)

◮ Compare the performance of two applications under the same dynamic memory conditions and consider the one with the better results as more memory-elastic ◮ The results may be sensitive to the order or frequency of memory allocations

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 16 / 26

slide-29
SLIDE 29

Naive Metric

0.0 0.2 0.4 0.6 0.8 1.0 Time 5 10 15 20 Memory (GB) Allocation (GB)

◮ Compare the performance of two applications under the same dynamic memory conditions and consider the one with the better results as more memory-elastic ◮ The results may be sensitive to the order or frequency of memory allocations ◮ This is because we try to infer memory elasticity from observations of metrics that

  • nly hint about elasticity, but do not measure it directly
  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 16 / 26

slide-30
SLIDE 30

Our Goal

◮ Our goal is

◮ To quantify an application’s behavior in a dynamic memory scenario ◮ To compare it to other applications ◮ Using metrics that directly relate to memory elasticity

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 17 / 26

slide-31
SLIDE 31

Static Metrics

◮ Static memory→performance function (Pmem) that describes the performance of the application given a static memory allocation ◮ Elasticity domain: [memL, memH] ◮ Elasticity range: memH − memL

1 2 3 4 5 6 RAM (GB) 500 1000 TPS Insufficient RAM Elasticity Domain No perf. improvement memL memH

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 18 / 26

slide-32
SLIDE 32

Memcached Static Metrics

◮ Elasticity domain: from 1 GB to 3.5 GB ◮ Elasticity range: 2.5 GB

1024 1536 2048 2560 3072 3584 Memory (MB) 0K 20K 40K 60K Hits per second memL memH

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 19 / 26

slide-33
SLIDE 33

Dynamic Metrics

t0 t1 t2 t3 t4 t5 Time Pmem(α) Pmem(β) Performance Increase memory Decrease memory Phase A mem = α Phase B mem = β Phase C mem = α Tmem(α, β) Tmem(β, α)

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 20 / 26

slide-34
SLIDE 34

Performance Loss During the Transient Period (1)

Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 21 / 26

slide-35
SLIDE 35

Performance Loss During the Transient Period (1)

Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 21 / 26

slide-36
SLIDE 36

Performance Loss During the Transient Period (2)

Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss A:Emem B:Emem Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 22 / 26

slide-37
SLIDE 37

Performance Loss During the Transient Period (2)

Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss A:Emem B:Emem Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 22 / 26

slide-38
SLIDE 38

Performance Loss During the Transient Period (2)

Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss A:Emem B:Emem Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 22 / 26

slide-39
SLIDE 39

Performance Loss During the Transient Period (3)

A:Emem B:Emem Tmem Time Pmem(l) Pmem(h) Performance A Perf. B Perf. A Perf. Loss B Perf. Loss

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 23 / 26

slide-40
SLIDE 40

Elastic Memcached Average Measured Emem

896 1024 1536 2048 2560 3072 3584 From Memory (MB) 3584 3072 2560 2048 1536 1024 896 To Memory (MB)

112 96 87 71 59 48 90 89 69 63 47 8 83 73 64 31 9 6 54 60 19 5 6 6 35 31 8 5 5 6 7 5 5 5 5 5 5 5 5 5 5 5

20 40 60 80 100 Emem (Seconds)

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 24 / 26

slide-41
SLIDE 41

Elastic vs. Off-the-shelf Memcached

Elastic Off-the-shelf

896 1024 1536 2048 2560 3072 3584 From Memory (MB) 3584 3072 2560 2048 1536 1024 896 To Memory (MB)

112 96 87 71 59 48 90 89 69 63 47 8 83 73 64 31 9 6 54 60 19 5 6 6 35 31 8 5 5 6 7 5 5 5 5 5 5 5 5 5 5 5

20 40 60 80 100 Emem (Seconds) 896 1024 1536 2048 2560 3072 3584 From Memory (MB) 3584 3072 2560 2048 1536 1024 896 To Memory (MB)

167 185 249 327 598 1321 152 177 225 367 748 1070 160 147 233 569 527 374 116 153 286 344 163 133 97 153 130 77 52 61 164 58 25 17 21 18 98 21 14 17 12 14

250 500 750 1000 1250 Emem (Seconds)

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 25 / 26

slide-42
SLIDE 42

Conclusions

◮ We showed a few major building blocks that can be made memory elastic

◮ Cache, intermediate buffers, garbage-collection and schedulers

◮ We defined metrics that are comparable across applications

◮ Elasticity range and Emem

◮ We defined characteristics that can be used by clients to configure their virtual machine and their application in a memory elastic cloud environment

◮ Pmem and Tmem

◮ Our framework is available from github.com/liran-funaro/elastic-benchmarks Liran Funaro: funaro@cs.technion.ac.il

  • L. Funaro, O. Agmon Ben-Yehuda, A. Schuster (Technion)

Memory Elasticity Benchmark 26 / 26