Prophecy : Using History for High Throughput Fault Tolerance - PowerPoint PPT Presentation

Delay ‐ once linearizability Delay once linearizability

Delay ‐ once linearizability Delay once linearizability 〈 W , R , W , W , R , R , W , R 〉

Delay ‐ once linearizability Delay once linearizability Read ‐ after ‐ write property 〈 W , R , W , W , R , R , W , R 〉

Delay ‐ once linearizability Delay once linearizability Read ‐ after ‐ write property 〈 W , R , W , W , R , R , W , R 〉

Example application Example application • Upload embarrassing photos Upload embarrassing photos 1. Remove colleagues from ACL 2 Upload photos 2. Upload photos 3. (Refresh) • Weak may reorder • Delay ‐ once preserves order

Byzantine fault tolerance (BFT) Byzantine fault tolerance (BFT) • Low throughput Low throughput D Prophecy D ‐ Prophecy • Modifies clients difi li Prophecy • Long ‐ lived sessions

Implementation Implementation • Modified PBFT Modified PBFT – PBFT is stable, complete – Competitive with Zyzzyva et. al. Competitive with Zyzzyva et al • C++, Tamer async I/O – Sketcher: ∼ 2000 LOC – PBFT library: ∼ 1140 LOC – PBFT client: ∼ 1000 LOC

Evaluation Evaluation • Prophecy vs proxied ‐ PBFT Prophecy vs. proxied PBFT – Proxied systems • D ‐ Prophecy vs. PBFT – Non ‐ proxied systems

Evaluation Evaluation • Prophecy vs proxied ‐ PBFT Prophecy vs. proxied PBFT – Proxied systems • We will study: – Performance on “null” workloads – Performance on null workloads – Performance with real replicated service – Where system bottlenecks, how to scale Where system bottlenecks how to scale

Basic setup Basic setup Sk t h Sketcher (concurrent) Clients Clients (100) Replica Group (PBFT)

Fraction of failed Fraction of failed fast reads

Alexa top sites: Alexa top sites: Fraction of failed Fraction of failed < 15% fast reads

Small benefit on null reads Small benefit on null reads

Small benefit on null reads Small benefit on null reads

Apache webserver setup Apache webserver setup Sk t h Sketcher Clients Clients Replica Group

Large benefit on real workload Large benefit on real workload

Large benefit on real workload Large benefit on real workload 3.7x

Large benefit on real workload Large benefit on real workload 3.7x 2.0x

Large benefit on real workload Large benefit on real workload 3.7x 2.0x

Benefit grows with work Benefit grows with work

Benefit grows with work Benefit grows with work

Benefit grows with work Benefit grows with work

Benefit grows with work Benefit grows with work 94 μ s (Apache)

Benefit grows with work Benefit grows with work 94 μ s (Apache) Null workloads are misleading! are misleading!

Benefit grows with work Benefit grows with work

Single sketcher bottlenecks Single sketcher bottlenecks

Single sketcher bottlenecks Single sketcher bottlenecks

Scaling out Scaling out

Scales linearly with replicas Scales linearly with replicas

Summary Summary • Prophecy good for Internet services p y g – Fast, load ‐ balanced reads • D Prophecy good for traditional services • D ‐ Prophecy good for traditional services • Prophecy scales linearly while PBFT stays flat Prophecy scales linearly while PBFT stays flat • Limitations: – Read ‐ mostly workloads (meas. study corroborates) – Delay ‐ once linearizability (useful for many apps)

Thank You

Additional slides