On The Scalability of Storage Sub-System Back-end Network Yan Li, - - PowerPoint PPT Presentation

▶

Jun 24, 2023 263 likes •389 views

V I N E U R S E I H T Y T O H F G R E U D B I N On The Scalability of Storage Sub-System Back-end Network Yan Li, Roland Ibbett, Nigel Topham and Tim Courtney School of Informatics University of Edinburgh Xyratex,

SLIDE 1

T H E U N I V E R S I T Y O F E D I N B U R G H

On The Scalability of Storage Sub-System Back-end Network

Yan Li, Roland Ibbett, Nigel Topham and Tim Courtney‡ School of Informatics University of Edinburgh

‡Xyratex, UK Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 2

Project Motivation 1

T H E U N I V E R S I T Y O F E D I N B U R G H

Project Motivation

Theoretically the more disks used in a disk array, the higher the degree of

parallesim, so leading to larger the performance potential performance benefits.

However, in a real system there is a limitation on the scale of RAID systems

due to the limitation of interconnection network.

The more disks are added to the system, the higher the contention for the

shared media.

When the number of disks and cache size in a RAID system reaches a certain

threshold, there will be no further gain in performance by adding more disk or cache due to the saturation of the back-end network.

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 3

Project Goal 2

T H E U N I V E R S I T Y O F E D I N B U R G H

Project Goal

Investigate the capacity of interconnection networks in terms of the numbers
f disks that can be included in one chain. In particular, Fibre Channel (FC)

SBOD is chosen as the interconnection network. – Give a certain number of disks and cache size, how much bandwidth is necessary to support them to get the maximum performance? – Likewise, how many disks (and cache) can be connected to a 2GFC (4GFC) port?

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 4

Analytical Models 3

T H E U N I V E R S I T Y O F E D I N B U R G H

Assumptions:

– The request size of random access workload is equal to the size of stripe unit. – The access address of each request is aligned to the stripe unit boundary. – The capacity of the RAID system keeps fixed for all the study. – The queue length of disk is 1, ie., no disk command waits in the disk for service. – The FC SBOD is chosen as the research subject.

Disk command transmission time Tx = S

B + overhead. .

Disk command execution time Td =

S Bport + S Bmedia + Tseek .

S size of stripe unit; B network bandwidth;

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 5

Analytical Models 4

T H E U N I V E R S I T Y O F E D I N B U R G H

Large Sequential Access

D Tx Td Tv Tx Td Tv Time D + T x =

For sequential access workload, a large volume command is divided into D disk

commands, so that the response time for that command Tv = DTx + Td.

Throughput is the major performance metric:

Throughput = K Tv = K DTx + Td.

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 6

Analytical Models 5

T H E U N I V E R S I T Y O F E D I N B U R G H

Small Random Access

Tx Td Tv Tx Td Tv = 5) (D Td > Tv = 3) (D Td Td Tv

Time Time = (D−1)T <

x x x

(D−1)T (D−1)T (D−1)T + = DT T x

Tv =

T x + T d

(D − 1) ∗ T x < T d D ∗ T x (D − 1) ∗ T x >= T d

For random access workload, IOPS is the major performance metric,

IOPS =

8 > < > :

D T x + T d (D − 1) ∗ T x < T d T d T x ∗ 1 D ∗ T x (D − 1) ∗ T x >= T d

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 7

Analytical Models 6

T H E U N I V E R S I T Y O F E D I N B U R G H

Analytical Results (no cache)

20 40 60 80 2 4 6 8 10 12 14 16 x 10

number of disks throughput (Bytes/s) Seq Read, K=128KBytes 20 40 60 80 100 2000 4000 6000 8000 10000 12000 number of disks IOPS RANOM, S=16KBytes Sequential Access Random Access

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 8

Analytical Models 7

T H E U N I V E R S I T Y O F E D I N B U R G H

General Model

B = F(D, C, S, L, P) = Numd ∗ S + overhead

B, the bandwidth required to achieve the maximum performance with D disks and C cache in system. D: number of disks C: cache size S: size of stripe unit L: workload characteristic P: cache destage threshold, P=0 for our study Numd: number of disk commands send to disk per second

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 9

Simulation Results 8

T H E U N I V E R S I T Y O F E D I N B U R G H

Systems without Cache (S = 16KB)

20 40 60 80 100 20 40 60 80 number of disks Max SPC−1 BSU RAID5 RAID6 20 40 60 80 100 0.2 0.4 0.6 0.8 1 number of disks utilization Disk Uti RAID5 Disk Uti RAID6 Net Uti RAID5 Net Uti RAID6 20 40 60 80 100 120 140 50 100 150 number of disks Max SPC−1 BSU RAID5 RAID6 20 40 60 80 100 120 140 0.2 0.4 0.6 0.8 1 number of disks utilization Disk Uti RAID5 Disk Uti RAID6 Net Uti RAID5 Net Uti RAID6 Port Bandwidth = 2.125 Gbps Port Bandwidth = 4.25 Gbps

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 10

Simulation Results 9

T H E U N I V E R S I T Y O F E D I N B U R G H

Systems with Cache (S = 32KB)

20 40 60 50 100 150 200 number of disks Max SPC−1 BSU 20 40 60 1 2 3 4 5 6 7 number of disks bandwidth (Gbps) 20 40 60 0.8 0.85 0.9 0.95 1 number of disks disk utilization 20 40 60 0.65 0.7 0.75 number of disks read miss rate RAID5 RAID6 RAID5 RAID6 RAID5 RAID6 RAID5 RAID6 (a) (b) (c) (d)

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 11

Summary 10

T H E U N I V E R S I T Y O F E D I N B U R G H

Summary

When this is no cache, a 2G FC port is able to support up to 46 disks for

RAID5 and 53 disks for RAID6 (size of stripe unit = 16kBytes).

With enough cache, a 2G FC port is able to support up to 18 disks under

OLTP like workload. (size of stripe unit = 32KBytes).

When there is enough cache, the bandwidth required to support a certain

number of disks is fixed. It is irrelevant with protection level and cache size.

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk

SLIDE 12

Future Work 11

T H E U N I V E R S I T Y O F E D I N B U R G H

Future Work

Study the scalability of back-end network when the size of stripe unit is 16k

and the system performance.

Study the network bandwidth requirement when there is cache coherency

between two controllers.

Yan Li On The Scalability of Storage Sub-System Back-end Network Y.Li-24@sms.ed.ac.uk