[PPT] - Geo-distributed DB Aynchronous updates P Available under Partition PowerPoint Presentation

SLIDE 1

AntidoteDB

The developer-friendly open- source cloud database

Just-Right Consistency

Marc Shapiro, UPMC-LIP6 & Inria Annette Bieniusa, U. Kaiserslautern Nuno Preguiça, Valter Balegas, U. Nova Lisboa Christopher Meiklejohn, U. Catholique de Louvain

[AntidoteDB & Just-Right Consistency] 4

trois bases

[AntidoteDB & Just-Right Consistency]

Geo-distributed DB

5

+ fault-tolerance + low latency read Updates?

[AntidoteDB & Just-Right Consistency]

Aynchronous updates

6

P

Available under Partition Concurrency anomalies Eventual Consistency: hell for developers!

SLIDE 2

[AntidoteDB & Just-Right Consistency]

Synchronous updates

7

P

Consistent under Partition Conservative: one at a time Simple programming model Expensive infrastructure Not available under partition

Google Spanner

[AntidoteDB & Just-Right Consistency]

Byrum

FMK Fælles Medicinkort

8

R X

Dr Alice Aalborg Hospital Patient: Mr Bob Pharmacy: Byrum Causatin: 2 boxes ⟶ 1 Transactol: 1 box

Dr Alice Patient Bob Byrum pharma

create (…) add-med (…) get-med (…) process (…) Causatin: 2 boxes ⟶ 1

[AntidoteDB & Just-Right Consistency]

Byrum

FMK invariants

9

R X

Dr Alice Aalborg Hospital Patient: Mr Bob Pharmacy: Byrum Causatin: 2 boxes ⟶ 1 Transactol: 1 box

Dr Alice Patient Bob Byrum pharma

create (…) add-med (…) get-med (…) process (…) Causatin: 2 boxes ⟶ 1 precondition relative

rder

2 1 all or nothing

[AntidoteDB & Just-Right Consistency]

Byrum

Geo-distrib: invariants?

10

R X

Dr Alice Aalborg Hospital Patient: Mr Bob Pharmacy: Byrum Causatin: 2 boxes ⟶ 1 Transactol: 1 box

Dr Alice Patient Bob Byrum pharma

create (…) add-med (…) get-med (…) process (…) Causatin: 2 boxes ⟶ 1 precondition relative

rder

2 1 all or nothing EC does not maintain! CP is overkill!

SLIDE 3

[AntidoteDB & Just-Right Consistency]

The developer-friendly

pen-source alternative

Preserve sequential data invariant patterns Transactional Causal Consistency (TCC)

Maintains AP-compatible invariants
Strongest form of consistency that is AP

CAP-sensitive: Synchronise only when strictly necessary for application ⟹ tools No specialised infrastructure, no lock-in

11

.org

[AntidoteDB & Just-Right Consistency]

Data model: registers?

Concurrent, asynchronous updates

Standard register model: assignments ⟹ CP
AP ⟹ concurrent updates merged

CRDT: register, counter, set, map, sequence

Extends sequential type
Encapsulates convergent merge

12

cnt ≔ 1 cnt ≔ 1 cnt = 0 cnt ≔ 2 cnt ≔ 2 cnt = 0 add-med(1) add-med(2) cnt = 2 cnt = 1

[AntidoteDB & Just-Right Consistency]

AP data model: CRDTs

Concurrent, asynchronous updates

Standard register model: assignments ⟹ CP
AP ⟹ concurrent updates merged

CRDT: register, counter, set, map, sequence

Extends sequential type
Encapsulates convergent merge

13

cnt += 1 cnt += 1 cnt = 0 cnt += 2 cnt += 2 cnt = 0 add-med(1) add-med(2) cnt = 3 cnt = 3

[AntidoteDB & Just-Right Consistency]

CRDT API

CRDT: register, counter, set, map, sequence

Extends sequential type
Encapsulates convergent merge

antidote:update_objects ([{

{meds, antidote_crdt_set, bucket}, add, {"Causatin", client1} }], TxId1).

14

.org

SLIDE 4

[AntidoteDB & Just-Right Consistency]

Relative order: AP-Compatible

create-p before add-pp Relative-order invariant pattern:

“Patient record points to valid

prescription”

x valid ∧ x points to y ⟹ y valid
admin-login-enabled ⟹ non-

default-password

Make RHS true; then LHS true

Transmit in the right order! AP-compatible: Causal Consistency

15

1=RHS! 2=LHS! P >1

[AntidoteDB & Just-Right Consistency]

patient? pharma?

Joint update: AP-Compatible

create-p updates doctor, patient & pharmacy record Transmit joint updates together

write-atomic

+ Read from common set of txns

snapshot property

= All-or-Nothing (A of ACID) AP-compatible

16

P p h a r m a !

patient! pharma! patient?

p h a r m a ? p a t i e n t !

Atomic update Snapshot

[AntidoteDB & Just-Right Consistency]

Txnl Causal Consistency

Transactional Causal Consistency (TCC) = strongest AP model Guarantees AP-compatible invariant patterns Antidote: first industrial-strength TCC database

alpha

Guarantees Relative-Order and Joint-Update invariant patterns

17

.org

[AntidoteDB & Just-Right Consistency]

Transaction API

{ok, T1} = antidote:start_transaction (ignore, []). … antidote:update_objects ([{ {meds, antidote_crdt_set, bucket}, add, {"Causatin", client1} }], T1). … antidote:commit_transaction (T1).

18

SLIDE 5

[AntidoteDB & Just-Right Consistency]

process-p (…, nb) { if cnt ≥ nb // precondition at source cnt –= nb // at every replica } // ≥ 0

CAP-sensitive invariants

19

cnt –= 1 cnt –= 1 pp(…, 1) cnt ≥ 1

cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 2 2 2 1 1

[AntidoteDB & Just-Right Consistency]

process-p (…, nb) { if cnt ≥ nb // precondition at source cnt –= nb // at every replica } // cnt ≥ 0 Precondition stable w.r.t. concurrent add-med Concurrency OK

CAP-sensitive invariants

20

–= 1 cnt –= 1 pp(…, 1) cnt ≥ 1

cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 2 2 2 1 1

cnt += 3 cnt += 3 add-med (…, 3)

4 4

[AntidoteDB & Just-Right Consistency]

process-p (…, nb) { if cnt ≥ nb // precondition at source cnt –= nb // at every replica } // cnt ≥ 0

CAP-sensitive invariants

21

cnt –= 1 cnt –= 1 pp(…, 1) cnt ≥ 1

cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 2 2 2 1 1

[AntidoteDB & Just-Right Consistency]

process-p (…, nb) { if cnt ≥ nb // precondition at source cnt –= nb // at every replica } // cnt ≥ 0 Precondition not stable w.r.t. concurrent process-p

Forbid concurrency? Synchro, CP.
Or remove invariant? AP, degraded semantics

CAP-sensitive invariants

22

cnt –= 1 cnt –= 1 pp(…, 1) cnt ≥ 1

cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 2 2 2 1 1

cnt –= 2 cnt –= 2 pp(…, 2) cnt ≥ 1

–1 –1 CISE Static Analysis

SLIDE 6

[AntidoteDB & Just-Right Consistency]

Bounded Counter

Specific but common case Shared counter:

x ≥ 0
increment (n)
decrement (n) // precondition x ≥ n

Escrow:

Local share, decrement share –= n
Synchronise only when share < n
Donate share

Encapsulated, proven correct (CISE) Mostly AP

23

AP CP AP

[AntidoteDB & Just-Right Consistency]

CISE tools

Static analysis of any application:

Operations, invariants
Does each individual op maintain invariant?
Do concurrent updates converge?
Is precondition of u stable w.r.t. concurrent v?

If not:

Change specification (invariant)
or Synchronise
Designer decision, per pair (u, v)

Ex: medication count= inc||inc, inc||dec, dec||dec

24 [AntidoteDB & Just-Right Consistency]

Just-Right Consistency

Methodology for provably ensuring As Available as Possible, Consistent Enough TCC ⟹ AP-compatible invariants CAP-sensitive invariants: Bounded Ctr, CISE

25

AntidoteDB:

CRDTs
Causal Consistency
Transactions
Bounded Counter

⟹ AP- compatible

CISE verification & co-design (+ related tools)

CP when necessary mostly AP

[AntidoteDB & Just-Right Consistency]

AntidoteDB

Rich interface with concurrent abstract data types (CRDTs) Geo-replication Low latency, Availiable under Partition Erlang / Riak Core New, growing community Active development

Secondary Indexes, configurable back-ends

and protocols, partial replication, clients in many languages, SQL interface, etc.

26

.org

SLIDE 7

[AntidoteDB & Just-Right Consistency]

Get started with

27

.org

info@antidotedb.com

[AntidoteDB & Just-Right Consistency]

Creative Commons Attribution-ShareAlike 4.0

Intl. License

You are free to:

Share — copy and redistribute the material in any medium or

format

Adapt — remix, transform, and build upon the material

for any purpose, even commercially, under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.

28