Consistency in Replication Consistency in Replication ! Replication comes with consistency cost: Continuous Consistency Continuous Consistency ! Reasons for replication: Better performance and availability and Availability and Availability ! Replication transforms client-server client communication to server-server Haifeng Yu communication: client • Decrease performance server server CPS 212 Fall 2002 Decrease availability • server client 2 Problems with Binary Choice Strong Consistency and Optimistic Consistency Strong Consistency and Optimistic Consistency Problems with Binary Choice ! Traditionally, two choices for consistency level: ! Strong consistency incurs prohibitive overheads for many WAN apps • Replication may even decrease performance, availability and scalability • Strong consistency : Strictly “in sync” relative to a single server! • Optimistic consistency : No guarantee at all • Associated tradeoffs with each model ! Optimistic consistency provides no consistency guarantee at all Availability / • Resulting in upset users: Unbounded reservation conflicts Performance / • Potentially render the app unusable: If traffic data is more than 1 hour stale, Scalability probably of little use Optimistic Consistency ! Applications cannot tune consistency level based on its environment • Need to adapt to client, service and network characteristics Strong Consistency Consistency 3 4 Continuous Consistency Continuous Consistency Quantifying Consistency Quantifying Consistency ! Consistency is continuous rather than binary for many ! Many ways: WAN apps • Staleness (TTL in web caching): Invalidate • These apps can benefit from exploiting the consistency spectrum between strong and optimistic consistency. • Limit number of locally buffered writes Availability / Availability / Performance / Performance / Scalability Scalability To Other buffered Optimistic Consistency updates Replicas Continuous Consistency Strong Consistency Consistency Consistency 5 6 1
Applications ? Trading Consistency for Performance Applications ? Trading Consistency for Performance ! Applications: ! Airline reservation: running at Berkeley, Utah, Duke • Web caching • Airline reservation • Distributed games 50 Optimistic • Shared editor Consistency 40 (updates/sec) Throughput ! Non-Applications: 30 • Some scientific computing problems 20 • Banking system [Yu’02, TOCS] 10 • Any application that has binary output 0 Strong 0% 50% 100% ! Application’s nature determines whether continuous consistency Consistency Inconsistency is applicable 7 8 The Cost of Increased Performance Model vs. Protocol The Cost of Increased Performance Model vs. Protocol ! Increased performance comes with a cost ! Continuous consistency model is a spec. • Adaptively trade consistency for performance based on client , network , and service conditions ! Protocol is anything that can enforce the spec. 25% • Corollary: Strong consistency protocol is a protocol for any model Resv. Conflict Rate 20% ! Many protocols for a specific model, some are good, others are 15% not 10% 5% 0% 0% 20% 40% 60% 80% 100% Inconsistency 9 10 Designing a Continuous Consistency Model Designing a Continuous Consistency Model Distributed Consensus and Leader Election Distributed Consensus and Leader Election ! Model is a spec, thus quantifying consistency (in a bad way) is ! What does “continuous consistency” mean ? trivial • Allow at most k decision values • Allow at most k leaders ! Only applications know its definition of consistency • Airline reservation vs. distributed games ! Helps overcome some impossibilities • Unique decision value requires ½ majority • K decision values allow any partition with 1/(k + 1) nodes to decide ! What is a “good” continuous consistency model? • Can be used by diverse apps • Practical 11 12 2
Group Membership Service Continuous Consistency Summary Group Membership Service Continuous Consistency Summary ! Def: Keep track of which nodes belong to which group ! Traditionally, group membership only maintain a single group ! WAN replication needs dynamically tunable consistency • Primary-partition membership services • Corresponds to strong consistency ! Tradeoff between consistency and performance ! Recently, partitionable membership services ! How to design a continuous consistency model • Still active area of research • Corresponds to optimistic consistency ! Continuous consistency in other context ! Continuous consistency : ! Next: Availability • Allow at most k groups • Again, helps overcome the ½ majority limitation 13 14 What is Availability ? Perform- -ability ability What is Availability ? Perform ! No well-accepted availability metric for Internet services ! User satisfaction is not binary ! “Uptime” metric can be misleading for Internet services • What if a partial result is returned before time-out ? • What if the result is sent back after an hour, or a day ? • Server may be inaccessible because of network partition • Availability is related to performance ! Available: “present or ready for immediate use” ! Performability = reward function (quality and timeliness of • From Webster’s Collegiate Dictionary result) • What does “immediate” mean? • Time-out ! Determining reward function is hard ! ! Availability = (accepted accesses) / (submitted accesses) • Implicit time-out in the definition 15 16 Effects of Replication Effects of Replication Availability of an Internet Service Availability of an Internet Service communication × < 2% ! We use user-observed availability in our study: Replica Replica to maintain × reject Availability = (accepted accesses) / (submitted accesses) consistency client failed × 2% [ Chandra et.al., × reject Server USITS’01 ] > 0.1% client reject due to server × failure 0.1% [ MS press ! Consistency may force a replica to reject an otherwise acceptable request release,Jan’01 ] • Network Failure Rate Replica Rejection Rate 17 18 3
Limitations of Strong Consistency Effects of Continuous Consistency Limitations of Strong Consistency Effects of Continuous Consistency allow : Replicas replica to buffer : Clients 5 writes Option 1: accept reads accept reads reject writes reject writes Option 1: accept reads accept reads reject writes reject writes New Option 1: accept reads accept reads Option 2: accept reads reject reads accept first 10 writes accept first 5 writes accept writes reject writes 19 20 Effects of Continuous Consistency Consistency Impact is Inherent Inherent Effects of Continuous Consistency Consistency Impact is allow ! Hard bound always exist replica to ! We always know the to end points, but may not know the exact shape of the curve buffer Availability 5 writes Hard Bound Option 2: accept reads reject reads accept writes reject writes 0% Consistency 100% Availability New Option 2: accept reads accept first few reads 100% Consistency accept writes accept first 5 writes Inconsistency 21 22 Effects of Consistency Protocol Effects of Consistency Protocol Availability Optimizations Availability Optimizations ! Achieved availability also depends on protocol ! Technique should not be tied to model • Design better protocols • Job of system designers ! Focus on two techniques: • Retiring replicas Availability • Aggressive write propagation Upper Bound Protocol A Protocol B Inconsistency 23 24 4
Recommend
More recommend
