[PPT] - Write Fast, Read in the Past: Causal Consistency for Client-side PowerPoint Presentation

SLIDE 1

Write Fast, Read in the Past: Causal Consistency for Client-side Apps with SwiftCloud

Presented by Marek Zawirski Inria / UPMC-LIP6, Paris

(now at Google, Zürich)

Marek Zawirski, Nuno Preguiça, Sérgio Duarte, Annette Bieniusa, Valter Balegas, Marc Shapiro

Challenge: Database Access for Client-side Apps

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 1

App App

Challenge: Database Access for Client-side Apps

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 2

App App

A P I A P I

Limited boundaries of server-side database guarantees

Challenge: Database Access for Client-side Apps

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 3

App App

ad-hoc on the client-side

A P I A P I

Limited boundaries of server-side database guarantees

SLIDE 2

Challenge: Database Access for Client-side Apps

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 4

A P I

App App

A P I

App App

A P I A P I

Extended boundaries with SwiftCloud

Challenge: Database Access for Client-side Apps

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 5

A P I

App App

A P I

App App

A P I A P I

Extended boundaries with SwiftCloud

Consistent, available and convergent data access
Scalability with #objects and #clients
Fault-tolerance

Stronger than Eventual: Causal Consistency

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 6

bob_posts.add(“don’t think of visiting Vancouver…”) bob_posts.add(“… just do it! YOLO”) replies.add(“Alice: totally J”) client order read

Default on client-side: eventual consistency anomalies

Stronger than Eventual: Causal Consistency

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 7

bob_posts.add(“don’t think of visiting Vancouver…”) bob_posts.add(“… just do it! YOLO”) replies.add(“Alice: totally J”) client order read

Default on client-side: eventual consistency anomalies

read

SLIDE 3

Stronger than Eventual: Causal Consistency

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 8

!?!

bob_posts.add(“don’t think of visiting Vancouver…”) bob_posts.add(“… just do it! YOLO”) replies.add(“Alice: totally J”) client order read read

Default on client-side: eventual consistency anomalies

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 9

ü

Causal consistency: reads from causally-closed snapshot

bob_posts.add(“don’t think of visiting Vancouver…”) replies.add(“Alice: totally J”) read read client order

Stronger than Eventual: Causal Consistency

bob_posts.add(“… just do it! YOLO”)

Default on client-side: eventual consistency anomalies

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 10

bob_posts.add(“don’t think of visiting Vancouver…”) replies.add(“Alice: totally J”) read replies.add( “Eve: Bob ”) replies: ??? client order

Convergent Causal Consistency: No Lost Updates

bob_posts.add(“… just do it! YOLO”)

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 11

replies: {“Alice: totally J”, “Eve: Bob ”}

High-level convergent objects[CRDTs] resolve concurrency

bob_posts.add(“don’t think of visiting Vancouver…”) replies.add(“Alice: totally J”) client order read replies.add( “Eve: Bob ”)

Convergent Causal Consistency: No Lost Updates

bob_posts.add(“… just do it! YOLO”)

SLIDE 4

Challenge: Causal Consistency with Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 12

bjB.op
bjB.op
bjA.op
bjC.op

[PRACTI, NSDI’06]

Challenge: Causal Consistency with Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 13

bjB.op
bjB.op
bjA.op
bjC.op
bjA
bjB

[PRACTI, NSDI’06]

current state

Challenge: Causal Consistency with Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 14

bjB.op
bjB.op
bjA.op
bjC.op
bjA
bjB
bjB
bjC

[PRACTI, NSDI’06]

Challenge: Causal Consistency with Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 15

bjB.op
bjB.op
bjA.op
bjC.op
bjA
bjC
bjA
bjB
bjB
bjC

[PRACTI, NSDI’06]

SLIDE 5

Challenge: Causal Consistency with Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 16

bjB.op
bjB.op
bjA.op
bjC.op
bjA
bjB
bjB
bjC

[PRACTI, NSDI’06]

bjA
bjC

Challenge: Causal Consistency with Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 17

bjB.op
bjB.op
bjA.op
bjC.op
bjA
bjB
bjB
bjC

dependencies

[PRACTI, NSDI’06]

bjB.op
bjA.op
bjC.op
bjB.op
bjA
bjC

dependencies

Challenge: Causal Consistency with Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 18

bjB.op
bjB.op
bjA.op
bjC.op
bjA
bjB
bjB
bjC

[PRACTI, NSDI’06]

bjB.op
bjB.op

1 2 Inherent trade-offs in the general case: scalability vs. availability vs. fault-tolerance

bjA
bjC

Approach: Cloud-backed Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 19

DC

geo-replication

Data Center full replicas: üProvide consistent view Assign small metadata ü

4 9

DC

SLIDE 6

Approach: Cloud-backed Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 20

DC

geo-replication

4 9

DC

log

cache

App

Data Center full replicas: üProvide consistent view Assign small metadata ü

Approach: Cloud-backed Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 21

DC

geo-replication

4 9

DC

log

cache

App

update Data Center full replicas: üProvide consistent view Assign small metadata ü

Approach: Cloud-backed Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 22

DC

geo-replication

4 9

DC

log

cache

App

update

4 9

Data Center full replicas: üProvide consistent view Assign small metadata ü

5

Approach: Cloud-backed Partial Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 23

DC

geo-replication

4 9

DC

log

cache

App

Client reads: cached fragment of cloud version own log ü High availability Consistency w/read-your-writes ü Data Center full replicas: üProvide consistent view Assign small metadata ü read

SLIDE 7

Potential of Cloud-backed Client Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 24

Setup: DCs in 3 AWS EC2 regions, YCSB workload, cache=256 objects

Response time of operation [ms] CDF for 1000 clients

Potential of Cloud-backed Client Replicas

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 25

Response time of operation [ms] CDF for 1000 clients Objects in the cache immediate, consistent response

Setup: DCs in 3 AWS EC2 regions, YCSB workload, cache=256 objects

Challenge for the Cloud Approach: Safe DC Failover

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 26

cache

log

Challenge for the Cloud Approach: Safe DC Failover

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 27

cache

log

risky read

SLIDE 8

Challenge for the Cloud Approach: Safe DC Failover

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 28

cache

log

risky read

perations with

risky dependencies

Challenge for the Cloud Approach: Safe DC Failover

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 29

cache

log

risky read

perations with

risky dependencies new DC in incompatible state

Supporting Failover by Conservative Reads

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 30

cache K=2

log

Foreign updates: read version replicated in K > 1 DCs Own writes: read from the log, recover to a new DC

Supporting Failover by Conservative Reads

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 31

cache K=2

log

Foreign updates: read version replicated in K > 1 DCs Own writes: read from the log, recover to a new DC conservative read

SLIDE 9

Supporting Failover by Conservative Reads

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 32

cache K=2

log

ü

Foreign updates: read version replicated in K > 1 DCs Own writes: read from the log, recover to a new DC new DC in compatible state

Experiment: Injection of Short DC Disconnection

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 33

response time [ms]

Experiment: Injection of Short DC Disconnection

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 34

remote reads fast conservative reads

response time [ms]

Experiment: Injection of Short DC Disconnection

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 35

remote reads fast conservative reads remote ops: smooth failover … unaffected …

response time [ms]

SLIDE 10

Experiment: Injection of Short DC Disconnection

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 36

Trade-off controlled by K: staleness vs. availability

Staleness negligible in most K=2 setups, < 1% reads
In cherry-picked unfavorable setup, 1.0–2.5% reads

remote reads fast conservative reads remote ops: smooth failover … unaffected …

response time [ms]

Challenge for the Cloud Approach: Protocol Retries

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 37

inc() 4 4

x=0 x=0 x=0 x=1 time x.inc() (x=1)

Challenge for the Cloud Approach: Protocol Retries

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 38

inc() 4 4 inc() 3

x=0 x=0 x=0 x=1 x=1 x.inc() (x=1)

Challenge for the Cloud Approach: Protocol Retries

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 39

inc() 4 4 inc() 3 3

x=0 x=0 x=0 x=1 x=1 x.inc() (x=1)

SLIDE 11

Challenge for the Cloud Approach: Protocol Retries

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 40

inc() 4 4 inc() 3 3 inc() 3 inc()4

geo-replication

x.inc() (x=1) x=0 x=0 x=0 x=1 x=1 x=2

Challenge for the Cloud Approach: Protocol Retries

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 41

duplicate delivery!

inc() 4 4 inc() 3 3 inc() 3 inc()4

geo-replication

x.inc() (x=1) x=0 x=0 x=0 x=1 x=1 x=2

Safe Retries with Decoupled Metadata

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 42

geo-replication

inc() 4 4 inc() 3 3 3 inc()4 1 1 4 1

~

1 ~

Solution: client-assigned timestamps for safety + 1..N DC timestamps for efficient summary

ü

safe: ≤ 1 delivery

3

x=0 x=0 x=0 x.inc() (x=1) x=1 x=1 x=1

1

Safe Retries with Decoupled Metadata

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 43

geo-replication

inc() 4 4 inc() 3 3 3 inc()4 1 1 4 1

~

1 ~

Solution: client-assigned timestamps for safety + 1..N DC timestamps for efficient summary

ü

safe: ≤ 1 delivery

3

x=0 x=0 x=0 x.inc() (x=1) x=1 x=1 x=1

1

SLIDE 12

Safe Retries with Decoupled Metadata

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 44

geo-replication

inc() 4 4 inc() 3 3 3 inc()4 1 1 4 1

~

1 ~

Solution: client-assigned timestamps for safety + 1..N DC timestamps for efficient summary Extension: log pruning independent of client availability

ü

safe: ≤ 1 delivery

3

x=0 x=0 x=0 x.inc() (x=1) x=1 x=1 x=1

1

# client replicas

Experiment: Size of Metadata on Client-DC Link

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 45

1 10 100 1K metadata / update [B] SwiftCloud’s decoupled metadata: const size

Setup: 3DCs, YCSB B uniform workload

Client-assigned vectors[PRACTI, NSDI’06]: unbounded overhead

Summary

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 46

SwiftCloud provides client-side apps:

Consistent, available and convergent object database
Scalability: full replicas at DC back partial at client

small causality metadata (< 15B/update)

Fast failover thanks to conservative reads (< 1% stale)
Safe retry of interrupted transfer and safe log pruning

thanks to decoupled metadata Research prototype at github.com/SyncFree/SwiftCloud

SwiftCloud compared to “Lazy Replication”

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 47

Assume client-side application logic
Describe causal consistency support
Support communication with multiple servers
Use decoupled metadata
DB = RDT objects + global

transactions

Supports partial client

replicas => fast reads and read-your-writes

K-stability-driven trade-off
GC independent of clients
Monolithic DB
No client-side replicas
Stability discussion
Physical-clock-driven GC
More consistency choices

SLIDE 13

Challenge for the Cloud Approach: Protocol Retries

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 48

cache

log inc 3 inc 4

duplicate delivery!

inc 4

Safe Retries with Decoupled Metadata

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 49

cache

log inc 4 inc 3 1 1 4

ü

safe: ≤ 1 delivery Solution: client-assigned timestamps for safety + 1..N DC timestamps for efficient summary

Safe Retries with Decoupled Metadata

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 50

cache

log inc 4 inc 3 1 1 4

ü

safe: ≤ 1 delivery Solution: client-assigned timestamps for safety + 1..N DC timestamps for efficient summary

Safe Retries with Decoupled Metadata

Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 51

cache

log inc 4 inc 3 1 1 4

ü

safe: ≤ 1 delivery Solution: client-assigned timestamps for safety + 1..N DC timestamps for efficient summary Extension: log pruning independent of client availability