Just-Right As available as possible, consistent when necessary - PowerPoint PPT Presentation

Just-Right Consistency: Just-Right As available as possible, consistent when necessary Consistency The CAP theorem forces a choice In contrast, checking a data between strong consistency (CP) precondition on partitioned state is and availability and responsiveness CAP-sensitive. However, if two (AP) when the network can partition. updates do not negate each other's As available as possible precondition, they may legally The Just-Right Consistency (JRC) execute concurrently . Updates must As consistent as necessary approach defines, for a particular synchronise only if one negates the application, a consistency model precondition of the other. Correct by design that is sufficient to maintain the application invariants , otherwise The JRC approach is supported: by remaining as available as possible . the CRDT data model that ensures Marc Shapiro, UPMC-LIP6 & Inria JRC leverages knowledge of the that concurrent updates converge; application. Two invariant- by Antidote, a cloud-scale CRDT Annette Bieniusa, U. Kaiserslautern maintaining patterns are AP- data store that guarantees Nuno Preguiça, U. Nova Lisboa compatible: ordered updates and transactional causal consistency; Christopher Meiklejohn, U. Catholique de Louvain atomic grouping. They are and by developer tools (static compatible with concurrent and analysers and domain-specific Valter Balegas, U. Nova Lisboa asynchronous updates, orthogonally languages) that help guarantee to CAP. invariants. [AntidoteDB & Just-Right Consistency] 1 Bridging the CAP gap trois bases P CP ∩ AP = ∅ No single consistency model is best for all applications correct! Insight: Maintain invariants • Preserve sequential patterns • Synchronise only when strictly best possible necessary for application ⟹ tools availability and performance [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 3 4

FMK Fælles Medicinkort FMK invariants Byrum Byrum Byrum Byrum Dr Patient Dr Patient pharma pharma Alice Alice Bob Bob relative 2 order joint update R R Dr Alice Dr Alice X X Aalborg Hospital Aalborg Hospital create (…) 1 create (…) Patient: Mr Bob Patient: Mr Bob add-med (…) add-med (…) Pharmacy: Byrum Pharmacy: Byrum get-med (…) get-med (…) Causatin: 2 boxes ⟶ 1 Causatin: 2 boxes ⟶ 1 Causatin: 2 boxes ⟶ 1 Causatin: 2 boxes ⟶ 1 pre- process (…) process (…) Transactol: 1 box Transactol: 1 box condition [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 5 6 AP-compatible Geo-distrib: invariants? programming constructs Byrum Dr Byrum Patient Alice pharma Bob relative 2 order joint update Available under partition R ⟹ no synchronisation Dr Alice X ⟹ asynchronous updates Aalborg Hospital 1 create (…) ⟹ fast response Patient: Mr Bob add-med (…) Pharmacy: Byrum AP-compatible: EC does not maintain! • CRDT data model get-med (…) Causatin: 2 boxes ⟶ 1 Causatin: 2 boxes ⟶ 1 CP is overkill! • Relative-order pattern pre- process (…) Transactol: 1 box • Joint-update pattern condition [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 7 8

Relative order is without CC animation AP data model: CRDTs AP-Compatible 1=RHS! cnt += 1 add-med(1) cnt += 2 cnt = 0 cnt = 3 P cnt = 0 cnt = 3 cnt += 1 cnt += 2 create-p before add-pp • Referential integrity add-med(2) ‣ x valid ∧ x points to y ⟹ y valid • admin-login-enabled ⟹ non- Concurrent, asynchronous updates >1 default-password • Standard register model: assignments ⟹ CP RHS ≔ true; LHS ≔ true • AP ⟹ concurrent updates merged 2=LHS! Transmit in the right order! CRDT: register, counter, set, map, sequence • Extends sequential type AP-compatible: Causal Consistency • Encapsulates convergent merge [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 9 10 Joint update is without animations CAP-sensitive invariants AP-Compatible pp(…, 1) cnt –= 1 cnt ≥ 1 1 2 p cnt ≥ 0 cnt ≥ 0 h a Atomic r m a ! 1 P 2 update cnt ≥ 0 cnt ≥ 0 patient! cnt –= 1 2 cnt ≥ 0 create-p updates doctor, patient & patient! pharmacy record pharma! process-p (…, nb) { Transmit joint updates together • write-atomic if cnt ≥ nb // precondition at source pharma? cnt –= nb // at every replica + Read from common set of txns pharma? • snapshot property } // ≥ 0 patient? patient? = All-or-Nothing (A of ACID) AP-compatible Snapshot [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 11 12

CAP-sensitive invariants CAP-sensitive invariants pp(…, 1) pp(…, 1) cnt –= 1 cnt –= 1 cnt ≥ 1 cnt ≥ 1 4 2 1 2 1 cnt += 3 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 4 2 1 2 1 cnt += 3 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 cnt ≥ 0 –= 1 cnt –= 1 2 2 cnt ≥ 0 cnt ≥ 0 add-med (…, 3) process-p (…, nb) { process-p (…, nb) { if cnt ≥ nb // precondition at source if cnt ≥ nb // precondition at source cnt –= nb // at every replica cnt –= nb // at every replica } // cnt ≥ 0 } // cnt ≥ 0 Precondition stable w.r.t. concurrent add-med Concurrency OK [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 13 14 CAP-sensitive invariants CISE tools pp(…, 1) cnt –= 1 cnt ≥ 1 –1 1 2 cnt –= 2 cnt ≥ 0 cnt ≥ 0 –1 1 2 cnt –= 2 Static analysis of any application: cnt ≥ 0 cnt ≥ 0 cnt –= 1 • Operations, invariants 2 cnt ≥ 0 • Does each individual op maintain invariant? pp(…, 2) CISE cnt ≥ 1 • Do concurrent updates converge? Static process-p (…, nb) { • Is precondition of u stable w.r.t. concurrent v ? Analysis if cnt ≥ nb // precondition at source If not: cnt –= nb // at every replica ‣ Change specification (invariant) ‣ or Synchronise } // cnt ≥ 0 ‣ Designer decision, per pair (u, v) Precondition not stable w.r.t. concurrent process-p Ex: medication count= inc || inc , inc || dec , dec || dec • Forbid concurrency? Synchro, CP. • Or remove invariant? AP, degraded semantics [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 15 16

Just-Right Consistency AntidoteDB .eu Methodology for provably ensuring As Available as Possible, Consistent Enough CRDT data model TCC ⟹ AP-compatible invariants • Register, counter, set, map, sequence CAP-sensitive invariants: Bounded Ctr, CISE • Extend sequential semantics • AP compatible ⟹ AP- CP when compatible necessary Transactional Causal Consistency (TCC) AntidoteDB: • Strongest AP-compatible model • CRDTs • Joint Updates / Transactional CISE verification • Causal Consistency • Partial Order / Causal Consistency & co-design • Transactions Open source, well engineered (+ related tools) • Bounded Counter mostly Community of users AP [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 17 18 Creative Commons Attribution-ShareAlike 4.0 Intl. License You are free to: Share — copy and redistribute the material in any medium or • format Adapt — remix, transform, and build upon the material • for any purpose, even commercially, under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 19 20

accrue 5% +5 € –100 I ? I ? 100 € ≥ 0 u ? u ! v ! u ? u ! v ! σ : I σ : I 5 € ≥ 0 I ? I ? σ : I σ : I 5 € ≥ 0 v ! v ! u ! u ! 100 € ≥ 0 –100 +5 € debit(100) CISE Rules 1: Individually Correct Asynchronous, replicated updates • Individual operations maintain the invariant • State σ 2: Convergence • Invariant I • Concurrent effectors commute • Prepare: read one, generate effector 3: Precondition Stability • Update all, deferred: deliver effector • Every precondition is stable under every concurrent operation Converge? Invariant OK? If satisfied: invariant is guaranteed [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 21 22 Simple example: bank I u PRE u ! σ : I account • u ? • I u PRE σ : I u ! Operations: credit(amount), debit(amount) CISE Rules Invariant: balance ≥ 0 1: Sequential correctness • Start with weak specification • Individual operations maintain the invariant • Rule 1 ⟶ strengthen precondition for debit 2: Convergence • Rule 2: OK • Concurrent effectors commute • Rule 3 ⟶ debit || debit unsafe, fixed with 3: Precondition Stability concurrency control • Every precondition is stable under every concurrent operation If satisfied: invariant is guaranteed [AntidoteDB & Just-Right Consistency] [AntidoteDB & Just-Right Consistency] 23 24