Demystifying Distributed Transactions with the - PowerPoint PPT Presentation

Get rid of distributed transactions? “In retrospect I think that [not supporting distributed transactions] was a mistake . … a lot of people did want distributed transactions, and so they hand-rolled their own protocols, sometimes incorrectly , and it would have been better to build it into the infrastructure.“ - Jeff Dean

Get rid of distributed transactions? “In retrospect I think that [not supporting distributed transactions] was a mistake . … a lot of people did want distributed transactions, and so they hand-rolled their own protocols, sometimes incorrectly , and it would have been better to build it into the infrastructure.“ - Jeff Dean

Get rid of distributed transactions? “In retrospect I think that [not supporting distributed transactions] was a mistake . … a lot of people did want distributed … and many more transactions, and so they hand-rolled their own protocols, sometimes incorrectly , and it would have been better to build it into the infrastructure.“ - Jeff Dean

Why are distributed txns expensive?

Why are distributed txns expensive? Suppose T 1 observes T 0 ’s writes T 0 : Read R Read B . Write R . . Write R Write B T 1 : Read B . . . Write B

Why are distributed txns expensive? Suppose T 1 observes T 0 ’s writes T 0 : WAIT! Read R Read B Distributed Transactions entail . Write R . unavoidable coordination . Write R Write B T 1 : Read B . . . Write B

Why are distributed txns expensive? Suppose T 1 observes T 0 ’s writes T 0 : Read R Read B . Write R . . Write R Write B T 1 : Read B . T1 must wait for commit . . protocol to finish Write B

Why are distributed txns expensive? • Mechanisms for atomicity and isolation overlap in time • Atomicity: Distributed coordination • Isolation: Wait during distributed coordination

Fairness-Isolation-Throughput tradeoff • Any system implementing distributed transactions can get at most two of these three properties • Three classes of systems • Fairness-Isolation • Isolation-Throughput • Throughput-Fairness

FIT Tradeoff • Poor performance of distributed transactions attributable to two fundamental issues • Expensive commit protocol (required due to atomicity) • Waiting (required for isolation) • Commit protocol and waiting overlap in time • Space characterized by how to separate commit protocol from waiting

Intuition • Badness results from overlapping commit with isolation • To avoid impact of coordination, separate the two

Option 1: Weaken isolation • Allow conflicting txns to execute without observing each other’s writes • Implementable without making txns wait for each other • Susceptible to concurrency bugs • Transactions execute against potentially stale state • E.g., RAMP transactions

Option 2: Re-order coordination • Move coordination outside of transaction boundaries • Amortize coordination across several transactions • Compromises fairness because we penalize certain txns to benefit overall throughput • E.g., Calvin, G-Store

FIT Tradeoff • Fairness-Isolation • Give up throughput • Fairness-Throughput • Give up isolation • Isolation-Throughput • Give up fairness

FIT Tradeoff • Fairness-Isolation Atomicity and isolation • Give up throughput mechanisms overlap • Fairness-Throughput • Give up isolation • Isolation-Throughput • Give up fairness

FIT Tradeoff • Fairness-Isolation • Give up throughput • Fairness-Throughput • Give up isolation • Isolation-Throughput Atomicity and isolation • Give up fairness mechanisms are decoupled

Weak Isolation Example • Read Atomic Multi-Partition (RAMP) transactions • Decouples concurrent transactions • Research system • Appeared in SIGMOD 2014 • Peter Bailis et al. from UC Berkeley

RAMP transactions R B G

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G Run a commit protocol

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B Commit protocol represents B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) coordination required for Write R 1 G 1 B 1 atomicity G Run a commit protocol

RAMP transactions R, R T1 T 1 : R 0 , G 0 , B 0 = Read R G B B, B T1 R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G, G T1

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B T 1 and T 2 read the same B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) snapshot Write R 1 G 1 B 1 G T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R T 1 : R 0 , G 0 , B 0 = Read R G B B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R, R T1 T 1 : R 0 , G 0 , B 0 = Read R G B B, B T1 R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 Commit T 1 G, G T1 T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R, R T1 T 1 : R 0 , G 0 , B 0 = Read R G B B, B T1 R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 G, G T1 T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R, R T1 , R T2 T 1 : R 0 , G 0 , B 0 = Read R G B B, B T1, B T2 R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 Commit T 2 G, G T1 , G T2 T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R, R T1 , R T2 T 1 : R 0 , G 0 , B 0 = Read R G B B, B T1, B T2 R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 T 1 and T 2 don’t see each other’s writes G, G T1 , G T2 T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions R, R T1 , R T2 T 1 : R 0 , G 0 , B 0 = Read R G B B, B T1, B T2 R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1 State must be merged in some app dependent manner G, G T1 , G T2 T 2 : R 0 , G 0 , B 0 = Read R G B R 1 , G 1 , B 1 = f(R 0 , G 0 , B 0 ) Write R 1 G 1 B 1

RAMP transactions • Decouples execution of concurrent txns • “Synchronization independence” • Great for scalability • More resources means more throughput • Weak isolation • Diverged state must be reconciled and merged • Cannot enforce important class of constraints

RAMP transactions • Decouples execution of concurrent txns • “Synchronization independence” • Great for scalability Txns never blocks others • More resources means more throughput • Weak isolation • Diverged state must be reconciled and merged • Cannot enforce important class of constraints

RAMP transactions • Decouples execution of concurrent txns Extra development effort • “Synchronization independence” • Great for scalability • More resources means more throughput • Weak isolation • Diverged state must be reconciled and merged • Cannot enforce important class of constraints

RAMP transactions • Decouples execution of concurrent txns • “Synchronization independence” Irrelevant for many • Great for scalability applications • More resources means more throughput • Weak isolation • Diverged state must be reconciled and merged • Cannot enforce important class of constraints

Fairness-Isolation-Throughput tradeoff • Any system implementing distributed transactions can get at most two of these three properties • Three classes of systems • Fairness-Isolation • Isolation-Throughput • Throughput-Fairness

Why are distributed txns expensive? Suppose T 1 observes T 0 ’s writes T 0 : Read R Read B . Write R . . Write R Write B T 1 : Read B . T1 must wait for commit . . protocol to finish Write B

Re-ordered coordination example • Move distributed coordination outside txn boundaries • Amortize its cost across several txns • Guarantee isolation • Conflicts still induce waiting • But txns don’t wait for distributed coordination • By re-ordering coordination, some txns are penalized • Unfairly delay txns to benefit overall throughput

G-Store • Built for workloads with temporal locality • E.g., multi-player games • Research system • Appeared in SoCC 2010 • Sudipto Das et al. from UC Santa Barbara • Built as a transaction layer on top of Hbase

G-Store R Supports txns on “KeyGroups” B Y G

G-Store R B Y G