PICav: Precise, Iterative & Complement-based Cloud Storage - PowerPoint PPT Presentation

Department of Computer Science | Institute of Systems Architecture | Chair of Computer Networks PICav: Precise, Iterative & Complement-based Cloud Storage Availability Calculation Scheme Josef Spillner, Johannes Müller mailto:josef.spillner@tu-dresden.de xmpp:josef.spillner@jabber.org 7th IEEE/ACM International Conference on Utility and Cloud Computing (UCC), December 8-11, 2014, London, UK

Multj-Cloud Storage Systems n=4 stored fragments k=3 significant fragments my precious file m=1 redundant fragments »»» user's decision: redundancy = 33⅓% 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 2 Availability Calculation Scheme - J. Spillner & J. Müller

The Old Way of Storage Confjguratjon 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 3 Availability Calculation Scheme - J. Spillner & J. Müller

The New Way of Storage Confjguratjon 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 4 Availability Calculation Scheme - J. Spillner & J. Müller

Transitjon between Old Way and New Way optimum 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 5 Availability Calculation Scheme - J. Spillner & J. Müller

Related Work: Optjmal Redundancy 1 Fragment := x Elements Proportional element distribution β-distribution of incidents Savings: Heterogeneity: up to 70% + availability - capacity 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 6 Availability Calculation Scheme - J. Spillner & J. Müller

Optjmal Redundancy Example (replication factor n/k) *) O(2^n) 4 nodes max: 342,19 = (1 - 0.5 4 ) * 365; 2^4 = 16 combinations 8 nodes max: 363,57 = (1 - 0.5 8 ) * 365; 2^8 = 256 combinations 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 7 Availability Calculation Scheme - J. Spillner & J. Müller

Iteratjve Fragment Distributjon Algorithm 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 8 Availability Calculation Scheme - J. Spillner & J. Müller

Algorithm Propertjes Precise Capacity consideration β-distribution → normal distribution of incidents >30 storage nodes: approximation by assuming homogeneous availability (like original approach) Iterative Fragments: composed of many elements Proportional distribution → clustered distribution of elements Iterative refinement: Classes of intervals of availability Complement-based When k < n/2, m > n/2 → many redundant nodes can be excluded → work with complement availabilities, less exclusions 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 9 Availability Calculation Scheme - J. Spillner & J. Müller

Evaluatjon Setup Method - availabilities: 25%, 50%, 75% - find availability gain & reduction of elements = 6 setups [5..10] redundant elements 4 variances = 144 data points Hardware HPI Future SOC Lab Potsdam HP DL980 G7-1, 8x Xeon CPU 128 cores @ 2.27 GHz Software NubiSave Standalone Python script 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 10 Availability Calculation Scheme - J. Spillner & J. Müller

Evaluatjon: Artjfjcial Clouds 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 11 Availability Calculation Scheme - J. Spillner & J. Müller

Evaluatjon: Real Clouds (with traces) 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 12 Availability Calculation Scheme - J. Spillner & J. Müller

Summary PICav: Precise, Iterative, Complement-based Compared with original (homogeneous-capacity, proportional) approach ● average complexity: ~50% ● worst-case complexity: ~50% Less than 40 nodes: real-time calculation possible Implementation NubiSave storage controller Standalone Python script (for validation) Bonus: Constraints & Optimisation Provider dependencies (e.g. Dropbox using Amazon S3) Recovery hinting: prefer „cheap“ providers (e.g. Dropbox over S3) GUI support for eliminating „redundant redundancy“ Many elements: merged into larger fragment files (bulking) 7th UCC, 11.12.2014 PICav: Precise, Iterative and Complement-based Cloud Storage # 13 Availability Calculation Scheme - J. Spillner & J. Müller