Contents Introduction Basic Model High Availability, Scalable - - PowerPoint PPT Presentation

1

High Availability, Scalable Storage, Dynamic Peer Networks: Pick Two

• Nov. 24, 2003

Byung-Gon Chun

2

Contents

• Introduction
• Basic Model
• Availability and Redundancy
• Discussion
• High Availability, Scalable Storage,

Dynamic Peer Networks: Pick Three

3

Introduction

• Peer-to-peer lookup: robust, scalable with

dynamic membership ⇒ Robust and scalable storage with dynamic membership ?

• Pick two

– Lookup is not bottleneck. – (upstream) bandwidth limitation – Disk space grows faster than access bandwidth

4

Basic Model

• Assumptions

– Simple redundancy maintenance mechanism (enter and exit) – Static data placement strategy (f: RB-> N) – Identical per-node space and bandwidth contributions – Constant rate of entering and exiting. – Independence of exit events – Constant steady-state number of nodes and total data size – Maintenance bandwidth

• Average case analysis

5

Basic Model

• N: number of hosts
• D: data
• S: data + redundancy (S = kD)
• α: entering rate
• λ: exiting rate (α = λ)
• T: lifetime (T=N/λ)
• B: bandwidth

6

Understanding the Scaling

• Short membership : enormous nodes to scale
• How fast storage of systems can grow?

(k = 20)

7

Availability & Redundancy

• Membership timeout: distinguish true departures

from temporary downtime, delay its response to failures

• Counting offline hosts as members

– Lifetime is longer – Hosts serve as a fraction of time (a: availability) – More redundancy is needed – Effective bandwidth is reduced

• Redundancy: replication vs. erasure coding

8

Model

9

Availability & Redundancy

• 33000 hosts Gnutella network, 1TB data, six nine

data availability

• 30-fold savings by membership timeout
• Additional 8-fold savings by erasure coding

– 75Kbps maintenance bandwidth per node – 500MB of disk per host contributed

• 5000 of 33000 hosts usually available

– Aggregate bandwidth 500Mbps – 5 dedicated, reliable PCs with 250GB drives and 50Mbps connection up 99% of the time

10

Membership Timeout

11

Replication vs. Coding

12

Admission Control, Load- Shifting

• Do not admit highly volatile nodes, Shift

responsibility to non-volatile hosts

• 5% most available hosts - 40% of service years.

– 30Kbps per node per unique-TB using coding – 1000-fold savings using delayed response, coding, and admission control

• Still bounded by bandwidth

– 100Kbps maintenance bandwidth, 3GB disk space – 10 universities with 1/3 OC3

• Two million cable modem users at 40%

availability ~ 2000 universities with ½ OC3

13

Hardware Trends

• Participation should be more stable to

contribute meaningful fraction of disks

14

Incentive Issues

• Stable membership is necessary.
• How to incent?

– Added value of service guarantees – Allow client bandwidth usage to be only proportional to contributed bandwidth

• - Prioritizing traffic

15

Discussion

• High availability, scale, dynamic

membership: high service bandwidth ⇒ Current DHT research trajectory ???

• Static membership – small lookup-state
• ptimization do more harm than good

(another approach - one-hop lookup) (another approach – distributed directory)

• Dynamic membership – why leverage many

flaky nodes to serve data a few reliable ones

16

Discussion

• Why worry about lookup guarantees if

storage guarantees are inappropriate?

• When anonymity or related security

properties are the high, why not plan to include the defense from the beginning?

17

Availability

[Bhagwan, Savage, and Voelker 2003]

18

Pick Three

• Distributed directory (DD)

– Uses a level of indirection – Controls the data placement – Exploits heterogeneity (availability, lifetime, and bandwidth)

Pick Three!!!

19

Discussion?