The Case for Benchmarking Control Operations in Cloud Native Storage - - PowerPoint PPT Presentation

▶
the case for benchmarking control operations in cloud
SMART_READER_LITE
LIVE PREVIEW

The Case for Benchmarking Control Operations in Cloud Native Storage - - PowerPoint PPT Presentation

Feb 06, 2023 228 likes •575 views

The Case for Benchmarking Control Operations in Cloud Native Storage 12 th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 20) Alex Merenstein 1 , Vasily Tarasov 2 , Ali Anwar 2 , Deepavali Bhagwat 2 , Lukas Rupprecht 2 ,

slide-1
SLIDE 1

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020

Alex Merenstein1, Vasily Tarasov2, Ali Anwar2, Deepavali Bhagwat2, Lukas Rupprecht2, Dimitris Skourtis2, and Erez Zadok1

The Case for Benchmarking Control Operations in Cloud Native Storage

12th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage ’20)

1Stony Brook University; 2IBM Research - Almaden

slide-2
SLIDE 2

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 2

  • Introduction
  • Storage Control Operations
  • Impact of Storage Control Operations
  • Benchmark Design
  • Conclusion

Outline

slide-3
SLIDE 3

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 3

  • Cloud native software

◆ Often container based ◆ Microservice architectures ◆ Frequent scaling and updates

  • Cloud native storage

◆ Used by applications, not systems ◆ Automated management ◆ Container Storage Interface (CSI)

provides standard interface

New Trends in Clouds

https://landscape.cncf.io

slide-4
SLIDE 4

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 4

Benchmarking’s Blind Spot

  • Storage challenges

◆ Choosing a storage provider ◆ Evaluating different storage configurations

  • Current benchmarks (e.g., fio1, pgbench2, NoSQLBench3)

◆ I/O operations ◆ Metadata operations ◆ Storage control operations

1. https://fio.readthedocs.io/en/latest/index.html 2. https://www.postgresql.org/docs/current/pgbench.html 3. https://www.datastax.com/blog/2020/03/nosqlbench

slide-5
SLIDE 5

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 5

  • Introduction
  • Storage Control Operations
  • Impact of Storage Control Operations
  • Benchmark Design
  • Conclusion

Outline

slide-6
SLIDE 6

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 6

Storage Control Operations

  • Storage control operations

◆ Creating volumes, attaching volumes, snapshotting, resizing, etc. ◆ Volumes: single unit of storage provisioned by a storage provider

  • More frequent in cloud native environments
  • Existing benchmarks do not generate

storage control operations

slide-7
SLIDE 7

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 7

Increasing Number of Storage Control Operations

  • Some companies have increased deployments from

2–3 ⨉ week to 150 ⨉ day1

  • On one platform, 54% of containers ran for ≤5 minutes

and hosts ran a median of 30 containers2

◆ On a 20 nodes cluster, that results in a rate of one container creation per second

1. https://www.weave.works/technologies/going-cloud-native-6-essential-things-you-need-to-know 2. https://sysdig.com/blog/sysdig-2019-container-usage-report/

slide-8
SLIDE 8

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 8

User Creates Container Requiring Storage

Container

Storage Provider Node (VM)

slide-9
SLIDE 9

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 9

Create Volume (Storage Control Operation #1)

Container

Storage Provider Node (VM)

Volume

Storage control

  • peration #1
slide-10
SLIDE 10

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 10

Container Scheduled on Node

Storage Provider Node (VM)

Container Container

Storage control

  • peration #1

Volume

slide-11
SLIDE 11

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 11

Volume Mounted on Node (Storage Control Operation #2)

Storage Provider Node (VM)

Container Volume Mount

Storage control

  • peration #2

Storage control

  • peration #1

Volume

slide-12
SLIDE 12

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 12

Volume Attached to Container (Storage Control Operation #3)

Storage Provider Node (VM)

Container Volume Mount

Storage control

  • peration #2

Storage control

  • peration #3

Storage control

  • peration #1

Volume

slide-13
SLIDE 13

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 13

  • Introduction
  • Storage Control Operations
  • Impact of Storage Control Operations
  • Benchmark Design
  • Conclusion

Outline

slide-14
SLIDE 14

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 14

Impact of Storage Control Operations

  • Experiment 1: creating and attaching volumes

◆ Do storage providers have different performance characteristics when

executing these operations?

  • Experiment 2: snapshots with concurrent workload

◆ Can storage control operations impact other workloads? ◆ Is the level of impact different across different storage providers?

slide-15
SLIDE 15

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 15

Experimental Setup

Kubernetes Master Node Kubernetes Master Node Kubernetes Master Node Kubernetes Worker Node Kubernetes Worker Node

Gluster

Kubernetes with three masters in high availability configuration and two workers nodes Three different-by-design storage providers

slide-16
SLIDE 16

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 16

Experiment 1: Volume Creation and Attachment

slide-17
SLIDE 17

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 17

Median > 2⨉ higher Median ~2⨉ lower

Experiment 1: Volume Creation and Attachment

slide-18
SLIDE 18

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 18

Experiment 1: Volume Creation and Attachment

Performance does differ between storage providers

slide-19
SLIDE 19

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 19

Experiment 2: Snapshotting

slide-20
SLIDE 20

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 20

Experiment 2: Snapshotting

slide-21
SLIDE 21

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 21

Experiment 2: Snapshotting

slide-22
SLIDE 22

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 22

Experiment 2: Snapshotting

P99.9 latency 3.3⨉ higher with 20 snapshots P99.9 latency 24⨉ higher with 20 snapshots

slide-23
SLIDE 23

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 23

Experiment 2: Snapshotting

Storage control

  • perations can

impact other workloads Impact varies depending on storage provider

slide-24
SLIDE 24

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 24

  • Introduction
  • Storage Control Operations
  • Impact of Storage Control Operations
  • Benchmark Design
  • Conclusion

Outline

slide-25
SLIDE 25

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 25

Benchmark Design Requirements

Workload

1. Create I/O and storage control workloads 2. Specify complex & realistic storage control workloads 3. Use existing tools for I/O workloads 4. Include QoS targets

Useability

1. Enable reproducibility 2. Be easy to use

Result Measurement & Visualization

1. Measurement should be decoupled from I/O generation 2. Results should be aggregated in clear, actionable manner 3. Metrics collection should have low

  • verhead
slide-26
SLIDE 26

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 26

Proposed Design

1. Benchmark Controller: creates I/O workload containers and executes control operations

slide-27
SLIDE 27

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 27

Proposed Design

2. User creates Benchmark object 3. Benchmark objects: custom

  • bject type, created by users to

define a benchmark

slide-28
SLIDE 28

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 28

4. I/O containers: created by Benchmark Controller to run I/O workload 5. Container image repository: I/O workloads can be created using existing I/O benchmarking tools such as fio or filebench

Proposed Design

slide-29
SLIDE 29

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 29

6. The Benchmark Controller executes Control Operation workloads by acting directly on PVs and PVCs

Proposed Design

slide-30
SLIDE 30

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 30

7. The volumes used by the benchmark are provisioned by the storage provider specified in the Benchmark object

Proposed Design

slide-31
SLIDE 31

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 31

Proposed Design

8. Results and metrics are collected and can be analyzed and visualized using tools such as ELK or Grafana

slide-32
SLIDE 32

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 32

  • Introduction
  • Storage Control Operations
  • Impact of Storage Control Operations
  • Benchmark Design
  • Conclusion

Outline

slide-33
SLIDE 33

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020 33

  • New benchmark is needed to support cloud native

workflows

  • Proposed nine requirements and an initial design for

such a benchmark

  • Looking for community input, especially for storage

control operation rates

Conclusion

slide-34
SLIDE 34

The Case for Benchmarking Control Operations in Cloud Native Storage (HotStorage ’20) July 14, 2020

Alex Merenstein1, Vasily Tarasov2, Ali Anwar2, Deepavali Bhagwat2, Lukas Rupprecht2, Dimitris Skourtis2, and Erez Zadok1

The Case for Benchmarking Control Operations in Cloud Native Storage

1Stony Brook University; 2IBM Research - Almaden

Thank You Q&A

Contact: mmerenstein@cs.stonybrook.edu