Automated Workload Characterization for I/O Performance Analysis in - - PowerPoint PPT Presentation

▶
automated workload characterization for i o performance
SMART_READER_LITE
LIVE PREVIEW

Automated Workload Characterization for I/O Performance Analysis in - - PowerPoint PPT Presentation

Sep 12, 2022 323 likes •565 views

Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Axel Busch, Qais Noorshams, Samuel Kounev, February 4 th , 2015 Anne Koziolek, Ralf Reussner, Erich Amrehn ICPE 2015 , Austin,

slide-1
SLIDE 1

KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association ARCHITECTURE-DRIVEN REQUIREMENTS ENGINEERING GROUP INSTITUTE FOR PROGRAM STRUCTURES AND DATA ORGANIZATION, FACULTY OF INFORMATICS

www.kit.edu

Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

Axel Busch, Qais Noorshams, Samuel Kounev, Anne Koziolek, Ralf Reussner, Erich Amrehn busch@kit.edu February 4th, 2015 ICPE 2015, Austin, TX

slide-2
SLIDE 2

2 15-02-04

Motivation

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study App

S i m u l t a n e

  • u

s R e q u e s t s 20 40 60 80 100 Request Size (KB) 20 40 60 Response Time (ms) 5 10 15

  • Many measurements to perform

Invasive instrumentation needed Time consuming model development

Kunden Speicher-System

slide-3
SLIDE 3

3 15-02-04

Motivation

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments App

1.

Fully automated workload characterization Lightweight approach

Non-invasive instrumentation No need to install the full software stack No need to develop any complex models by hand

Applicable in virtualized environments Fast estimation of performance behaviour in typical scenarios

Motivation Approach Conclusion Case Study

2. 3.

slide-4
SLIDE 4

4 15-02-04

Methodology

Workload Characterization Metrics Set Workload Monitoring & Extraction I/O-intensive workload

e.g., file server, mail server

Workload Characteristics Model

Model Process Input Output

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

  • 1. Workload Characterization

Motivation Approach Conclusion Case Study

slide-5
SLIDE 5

5 15-02-04

Methodology

Workload Emulation Workload Generator Monitoring Parameters

e.g., Response Time

Results of emulated Workload Workload Characteristics Model

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

  • 2. Workload Emulation

Motivation Approach Conclusion Case Study

slide-6
SLIDE 6

6 15-02-04

Metrics set

Workload Mix Request Mix Files Workload Intensity Average Request Size Request Access Pattern File Set Size File Sizes Read Proportion Write Proportion Sequential Random

  • ptional

mandatory alternative

(Experimental Evaluation of the Performance-Influencing Factors of Virtualized Storage Systems.

  • Q. Noorshams, S. Kounev, and R. Reussner. In EPEW ’12, volume 7587 of LNCS. Springer, 2012.)

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

(simplified)

Motivation Approach Conclusion Case Study

slide-7
SLIDE 7

7 15-02-04

File size & File set size

File 1 File n ... File 2 File size File size File size File set size

fileSetSizeavg = Z T Pn(t)

ι=1 φι(t)

T dt P

re φι(t) X he size of the ι-th file at time t z

:

nd n(t) : he number of files at time t,

e observation period. Let [0, T], T > 0 b :

fileSizeavg = T n(t)

ι=1 φι(t)

T · n(t) dt

  • Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

T Size

Motivation Approach Conclusion Case Study

slide-8
SLIDE 8

8 15-02-04

Workload Intensity

Application

workloadIntensityavg = Z T χ(t) T dt nd) at time t. , re χ(t) he workload intensity

:

e observation period. Let [0, T], T > 0 b

:

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

System

Motivation Approach Conclusion Case Study

slide-9
SLIDE 9

9 15-02-04

Request Mix

R R W R W R R R W W

reqMix = #readRequests #readRequests + #writeRequests

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

System

Motivation Approach Conclusion Case Study

slide-10
SLIDE 10

10 15-02-04

Request Size

R R W R W Read Req Write Req Request Size Read Request Size Write

requestSize = P|Γ|−1

j=0

Γj |Γ| , Γ contains all observed request sizes

(

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments

System

Motivation Approach Conclusion Case Study

slide-11
SLIDE 11

11 15-02-04

Access Pattern

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

slide-12
SLIDE 12

12 15-02-04

Access Pattern

1. Searching for consecutive block access 2. Counts the number of consecutive blocks 3. Results in number of consecutive block accesses in percent

while i < req do // Iterate through requests for j such that i < j < req do block end = Ri2 // End block of request Ri block start = Rj1 // Start block of request Rj if block end = block start then req seq req seq + 2 // Count both Ri, Rj R R \ {Ri, Rj} continue while; end if end for i i + 1 end while return req seq

req

Algorithm 1 Access Pattern Recognition Algorithm

  • ps ← Number of operations

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

slide-13
SLIDE 13

13 15-02-04

Application

Workload Emulation Workload Generator Monitoring Parameters

e.g., Response Time

Results of emulated Workload Workload Characteristics Model

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

slide-14
SLIDE 14

14 15-02-04

Workload characterization performed on an IBM System z and DS8700 storage system Both systems represent high-end virtualized environments for critical business applications

System Setup

IBM System z Processors, Memory PR/SM (Hypervisor) LPAR1 LPAR2 z/Linux z/Linux App. App. IBM DS8700 Harddisks (RAID) Storage Controller Volatile Cache Non-Volatile Cache

Switched Fibre Channel Fibre Channel Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

slide-15
SLIDE 15

15 15-02-04

System z configuration:

Debian z/Linux VM (=LPAR) 2 IFLs (cores) ~2760 MIPS 4 GB RAM

DS8700 configuration:

50 GB volatile cache (VC) 2 GB non-volatile cache (NVC), i.e., battery-backed cache RAID5 array with 7 HDDs (15k r/min) with 1 hot spare disk

System Setup

IBM System z Processors, Memory PR/SM (Hypervisor) LPAR1 LPAR2 z/Linux z/Linux App. App. IBM DS8700 Harddisks (RAID) Storage Controller Volatile Cache Non-Volatile Cache

Switched Fibre Channel Fibre Channel Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

slide-16
SLIDE 16

16 15-02-04

Tools

Filebench as storage system benchmark Used for generating workloads to be characterized Storage Performance Analyzer as measurement coordinator

https://github.com/Filebench-Revise http://research.spec.org/tools/overview/spa.html https://github.com/FFSB-Prime/ffsb

FFSB

Flexible File System Benchmark as application layer I/O benchmark Used for emulating workloads

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

slide-17
SLIDE 17

17 15-02-04

Systematic experiments

SPA extension allowing automatized Workload execution Monitoring mechanisms Workload characteristics extraction

Workload results File server Mail server File size 17 KiB 130 KiB File set size 684 MiB 1163 MiB Workload intensity 16 50 Request mix 56 % 42 % Request size (r) 14 KiB 103 KiB Request size (w) 15 KiB 79 KiB Access pattern (r) 29 % 97 % Access pattern (w) 57 % 99 %

Workload characterization results [avg]: Measurements performed using 1 min warm up + 20x 5 min benchmark time Low standard deviations synchronized

Benchmark Controller Benchmark Driver Monitor Driver Benchmark Harness Controller Machine SQLite Database SUT Benchmark Monitor SSH

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

slide-18
SLIDE 18

18 15-02-04

Evaluation Scenarios

Workload characterization approach evaluated by two case studies I) Workload Characterization

How accurate is the estimation of the workload characterization approach?

II) Scenarios

a) Migration scenario

How accurate is the estimation in migration scenarios?

b) Consolidation scenario How accurate is the estimation in consolidation scenarios?

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

slide-19
SLIDE 19

19 15-02-04

Evaluation: Estimation Accuracy

Read error Write error Mail server 20.82 % 35.72 % File server 3.93 % 36.96 % Response Times Comparison

5 10 15 20 Mail server (r) Mail server (w) File server (r) File server (w) Original Emulation

IBM System z + DS8700

[ms]

How accurate is the estimation of the workload characterization approach?

Original Workload Emulated Workload

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

Response Times

WCM

slide-20
SLIDE 20

20 15-02-04

Evaluation: Migration Scenario

How accurate is the estimation in migration scenarios?

Sun Fire X4440 Sun Fire X4440

10 20 30 40 50 60 70 File server (r) File server (w) Original Emulation

Read error Write error File server 21.59 % 20.98 %

[ms]

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

Response Times Comparison Original Workload Emulated Workload

WCM FS

Response Times

slide-21
SLIDE 21

21 15-02-04

Evaluation: Consolidation Scenario

How accurate is the estimation in consolidation scenarios?

15 30 45 60 75 90 Mixed (r) Mixed (w) Original Emulation

Read error Write error Mixed 12.95 % 24.51 %

[ms]

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

Sun Fire X4440 Sun Fire X4440 Response Times Comparison Original Workload Emulated Workload

WCM MS

Response Times

WCM FS WCM MS

slide-22
SLIDE 22

22 15-02-04

Evaluation: Summary

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study

þ Estimation Accuracy þ Migration Scenario þ Migration + Consolidation Scenario Accurate results for a fast initial performance estimation in typical scenarios

slide-23
SLIDE 23

23 15-02-04

Conclusion

  • Fully automated approach to derive a workload characteristics model
  • Capturing I/O performance-relevant workload parameters using a

formalized metrics set

  • Approach applied in real-world scenarios using state-of-the art

virtualization hardware Summary

  • Promising accuracy for fast initial performance estimation
  • Migration and consolidation scenarios show low error rates < 25 %

Evaluation results

  • Using workload characterization approach as a basis for other

performance models.

  • Applying scenarios when interpolate and extrapolate workload

parameters Future Work

Axel Busch – Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments Motivation Approach Conclusion Case Study