PICav: Precise, Iterative & Complement-based Cloud Storage - - PowerPoint PPT Presentation

▶

Aug 11, 2023 870 likes •1.02k views

Department of Computer Science | Institute of Systems Architecture | Chair of Computer Networks PICav: Precise, Iterative & Complement-based Cloud Storage Availability Calculation Scheme Josef Spillner, Johannes Mller

SLIDE 1

Department of Computer Science | Institute of Systems Architecture | Chair of Computer Networks

PICav: Precise, Iterative & Complement-based Cloud Storage Availability Calculation Scheme

7th IEEE/ACM International Conference on Utility and Cloud Computing (UCC), December 8-11, 2014, London, UK

Josef Spillner, Johannes Müller mailto:josef.spillner@tu-dresden.de

xmpp:josef.spillner@jabber.org

SLIDE 2

# 2 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Multj-Cloud Storage Systems

my precious file

k=3 significant fragments n=4 stored fragments m=1 redundant fragments

»»» user's decision: redundancy = 33⅓%

SLIDE 3

# 3 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

The Old Way of Storage Confjguratjon

SLIDE 4

# 4 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

The New Way of Storage Confjguratjon

SLIDE 5

# 5 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Transitjon between Old Way and New Way

ptimum

SLIDE 6

# 6 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Related Work: Optjmal Redundancy

Heterogeneity: + availability

capacity

Savings: up to 70% 1 Fragment := x Elements Proportional element distribution β-distribution of incidents

SLIDE 7

# 7 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Optjmal Redundancy Example

*) O(2^n) 4 nodes max: 342,19 = (1 - 0.54) * 365; 2^4 = 16 combinations 8 nodes max: 363,57 = (1 - 0.58) * 365; 2^8 = 256 combinations (replication factor n/k)

SLIDE 8

# 8 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Iteratjve Fragment Distributjon Algorithm

SLIDE 9

# 9 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Algorithm Propertjes

Precise Capacity consideration β-distribution → normal distribution of incidents >30 storage nodes: approximation by assuming homogeneous availability (like original approach) Iterative Fragments: composed of many elements Proportional distribution → clustered distribution of elements Iterative refinement: Classes of intervals of availability Complement-based When k < n/2, m > n/2 → many redundant nodes can be excluded → work with complement availabilities, less exclusions

SLIDE 10

# 10 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Evaluatjon Setup

Method

availabilities: 25%, 50%, 75%
find availability gain

& reduction of elements = 6 setups [5..10] redundant elements 4 variances = 144 data points Hardware HPI Future SOC Lab Potsdam HP DL980 G7-1, 8x Xeon CPU 128 cores @ 2.27 GHz Software NubiSave Standalone Python script

SLIDE 11

# 11 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Evaluatjon: Artjfjcial Clouds

SLIDE 12

# 12 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Evaluatjon: Real Clouds (with traces)

SLIDE 13

# 13 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage Availability Calculation Scheme - J. Spillner & J. Müller

Summary

PICav: Precise, Iterative, Complement-based Compared with original (homogeneous-capacity, proportional) approach

average complexity: ~50%
worst-case complexity: ~50%

Less than 40 nodes: real-time calculation possible Implementation NubiSave storage controller Standalone Python script (for validation) Bonus: Constraints & Optimisation Provider dependencies (e.g. Dropbox using Amazon S3) Recovery hinting: prefer „cheap“ providers (e.g. Dropbox over S3) GUI support for eliminating „redundant redundancy“ Many elements: merged into larger fragment files (bulking)