Relational Reasoning for Mergeable Replicated Data Types KC - - PowerPoint PPT Presentation

Relational Reasoning for Mergeable Replicated Data Types

• 1

KC Sivaramakrishnan

joint work with

Gowtham Kaki, Swarn Priya, Suresh Jagannathan

INTERNET

INTERNET

INTERNET

≠

INTERNET

≠

• Serializability
• Linearizability
• Weak Consistency & Isolation

When system-level concerns like replication & availability affect application-level design decisions, programming becomes complicated.

4

5

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d end

Sequential Counter

• Written in idiomatic style
• Composable

5

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d end type counter_list = Counter.t list

Sequential Counter

INTERNET

Replicated Counter

INTERNET

Replicated Counter

INTERNET

Replicated Counter

INTERNET

Replicated Counter

INTERNET

Replicated Counter

INTERNET

Replicated Counter

+2 2

INTERNET

Replicated Counter

+2 2

INTERNET

Replicated Counter

+2 2 +3 3

INTERNET

Replicated Counter

+2 +3 5 5 5 5

8

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

8

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7

8

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8

+1

8

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

8

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

24

*3

8

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

24 22

*3 +1

8

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

24 22

*3 +1 Diverges

8

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

24 22

*3 +1 Diverges

Addition and multiplication do not commute

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

• Capture the effect of multiplication through the commutative

addition operation

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7

• Capture the effect of multiplication through the commutative

addition operation

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8

+1

• Capture the effect of multiplication through the commutative

addition operation

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

• Capture the effect of multiplication through the commutative

addition operation

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

22

+14

• Capture the effect of multiplication through the commutative

addition operation

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

22 22

+14 +1

• Capture the effect of multiplication through the commutative

addition operation

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

22 22

+14 +1 Converges

• Capture the effect of multiplication through the commutative

addition operation

9

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n end

7 8 21

+1 *3

22 22

+14 +1 Converges

• Capture the effect of multiplication through the commutative

addition operation

• CRDTs

Conflict-free Replicated Data Types (CRDT)

10

Conflict-free Replicated Data Types (CRDT)

• CRDT is guaranteed to ensure strong eventual consistency (SEC)

★ G-counters, PN-counters, OR-Sets, Graphs, Ropes, docs, sheets ★ Simple interface for the clients of CRDTs 10

Conflict-free Replicated Data Types (CRDT)

• CRDT is guaranteed to ensure strong eventual consistency (SEC)

★ G-counters, PN-counters, OR-Sets, Graphs, Ropes, docs, sheets ★ Simple interface for the clients of CRDTs

• Need to reengineer every datatype to ensure SEC

(commutativity)

★ Do not mirror sequential counter parts => implementation & proof

burden

★ Do not compose!

✦

counter set is not a composition of counter and set CRDTs

10

Can we program & reason about replicated data types as an extension of their sequential counterparts?

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

7

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

7 8

+1

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

7 8 21

+1 *3

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

7 8 21

+1 *3

22

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

7 8 21

+1 *3

22 22

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

7 8 21

+1 *3

22 22

22 = 7 + (8-1) + (21 -7)

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

7 8 21

+1 *3

22 22

22 = 7 + (8-1) + (21 -7)

• 3-way merge function makes the counter suitable for distribution

12

module Counter : sig type t val read : t -> int val add : t -> int -> t val sub : t -> int -> t val mult : t -> int -> t val merge : lca:t -> v1:t -> v2:t -> t end = struct type t = int let read x = x let add x d = x + d let sub x d = x - d let mult x n = x * n let merge ~lca ~v1 ~v2 = lca + (v1 - lca) + (v2 - lca) end

7 8 21

+1 *3

22 22

22 = 7 + (8-1) + (21 -7)

• 3-way merge function makes the counter suitable for distribution
• Does not appeal to individual operations => independently

extend data-type

13

Systems ➞ PL

13

Systems ➞ PL

• CRDTs need to take care of

systems level concerns such as exactly once delivery

13

Systems ➞ PL

• CRDTs need to take care of

systems level concerns such as exactly once delivery

• 3-way merge handles it

automatically

13

7 8 21

+1 *3

22 22

Systems ➞ PL

• CRDTs need to take care of

systems level concerns such as exactly once delivery

• 3-way merge handles it

automatically

??

13

7 8 21

+1 *3

22 22

Systems ➞ PL

• CRDTs need to take care of

systems level concerns such as exactly once delivery

• 3-way merge handles it

automatically

??

13

7 8 21

+1 *3

22 22 22

22 = 21 + (21-21) + (22 -21)

Systems ➞ PL

• CRDTs need to take care of

systems level concerns such as exactly once delivery

• 3-way merge handles it

automatically

Does the 3-way merge idea generalise?

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2]

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2]

pop() 1

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2]

pop() 1 pop() 1

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2] [ ]

pop() 1 pop() 1 pop()

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2] [ ] [ ]

pop() 1 pop() 1 pop() pop()

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2] [ ] [ ]

pop() 1 pop() 1 pop() pop()

• Convergence is not sufficient; Intent is not preserved
• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2] [ ] [ ]

pop() 1 pop() 1 pop() pop()

• Convergence is not sufficient; Intent is not preserved

[1]

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2] [ ] [ ]

pop() 1 pop() 1 pop() pop()

• Convergence is not sufficient; Intent is not preserved

[1] [1,2]

push(2)

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2] [ ] [ ]

pop() 1 pop() 1 pop() pop()

• Convergence is not sufficient; Intent is not preserved

[1] [1,2] [1,3]

push(2) push(3)

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2] [ ] [ ]

pop() 1 pop() 1 pop() pop()

• Convergence is not sufficient; Intent is not preserved

[1] [1,2] [1,3] [1,2,3]

push(2) push(3) push(3)

• Try replicating queues by asynchronously transmitting operations

15

module type Queue = sig type 'a t val push : 'a t -> 'a -> 'a t val pop : 'a t -> ('a * 'a t) option (* at-least once semantics *) end

[1,2] [2] [2] [ ] [ ]

pop() 1 pop() 1 pop() pop()

• Convergence is not sufficient; Intent is not preserved

[1] [1,2] [1,3] [1,2,3] [1,3,2]

push(2) push(3) push(3) push(2)

• Try replicating queues by asynchronously transmitting operations

Concretising Intent

16

Concretising Intent

• Intent is a woolly term

★ How can we formalise the intent of operations on a data

structure?

16

Concretising Intent

• Intent is a woolly term

★ How can we formalise the intent of operations on a data

structure?

16

l v1 v2 v

Concretising Intent

• Intent is a woolly term

★ How can we formalise the intent of operations on a data

structure?

• For a replicated queue,

16

l v1 v2 v

Concretising Intent

• Intent is a woolly term

★ How can we formalise the intent of operations on a data

structure?

• For a replicated queue,
• 1. Any element popped in either v1 or v2 does not remain in v

16

l v1 v2 v

Concretising Intent

• Intent is a woolly term

★ How can we formalise the intent of operations on a data

structure?

• For a replicated queue,
• 1. Any element popped in either v1 or v2 does not remain in v
• 2. Any element pushed into either v1 or v2 appears in v

16

l v1 v2 v

Concretising Intent

• Intent is a woolly term

★ How can we formalise the intent of operations on a data

structure?

• For a replicated queue,
• 1. Any element popped in either v1 or v2 does not remain in v
• 2. Any element pushed into either v1 or v2 appears in v
• 3. An element that remains untouched in l, v1, v2 remains in v

16

l v1 v2 v

Concretising Intent

• Intent is a woolly term

★ How can we formalise the intent of operations on a data

structure?

• For a replicated queue,
• 1. Any element popped in either v1 or v2 does not remain in v
• 2. Any element pushed into either v1 or v2 appears in v
• 3. An element that remains untouched in l, v1, v2 remains in v
• 4. Order of pairs of elements in l, v1, v2 must be preserved in

m, if those elements are present in v.

16

l v1 v2 v

Relational Specification

17

Relational Specification

• Let’s define relations Rmem and Rob to capture membership and
• rdering

★ Rmem [1,2,3] = {1,2,3} ★ Rob [1,2,3] = { (1,2), (1,3), (2,3) }

17

Relational Specification

• Let’s define relations Rmem and Rob to capture membership and
• rdering

★ Rmem [1,2,3] = {1,2,3} ★ Rob [1,2,3] = { (1,2), (1,3), (2,3) }

17

l v1 v2 v

Relational Specification

• Let’s define relations Rmem and Rob to capture membership and
• rdering

★ Rmem [1,2,3] = {1,2,3} ★ Rob [1,2,3] = { (1,2), (1,3), (2,3) }

17

l v1 v2 v

Relational Specification

• Let’s define relations Rmem and Rob to capture membership and
• rdering

★ Rmem [1,2,3] = {1,2,3} ★ Rob [1,2,3] = { (1,2), (1,3), (2,3) }

1.Any element popped in either v1 or v2 does not remain in v

17

l v1 v2 v

Relational Specification

• Let’s define relations Rmem and Rob to capture membership and
• rdering

★ Rmem [1,2,3] = {1,2,3} ★ Rob [1,2,3] = { (1,2), (1,3), (2,3) }

1.Any element popped in either v1 or v2 does not remain in v

• 2. Any element pushed into either v1 or v2 appears in v

17

l v1 v2 v

Relational Specification

• Let’s define relations Rmem and Rob to capture membership and
• rdering

★ Rmem [1,2,3] = {1,2,3} ★ Rob [1,2,3] = { (1,2), (1,3), (2,3) }

1.Any element popped in either v1 or v2 does not remain in v

• 2. Any element pushed into either v1 or v2 appears in v
• 3. An element that remains untouched in l, v1, v2 remains in v

17

l v1 v2 v

Relational Specification

18

l v1 v2 v

Relational Specification

18

l v1 v2 v

• RHS has to be confined to Rmem(v) x Rmem(v) since certain
• rders might be missing

★ Consider l = [0], v1= [0,1], v2 = [ ], v = [1]

Relational Specification

18

l v1 v2 v

• RHS has to be confined to Rmem(v) x Rmem(v) since certain
• rders might be missing

★ Consider l = [0], v1= [0,1], v2 = [ ], v = [1]

• RHS is an underspecification since orders between concurrent

insertions will only be present in Rob(v)

★ Consider l = [ ], v1= [0], v2 = [1], v = [0,1]

19

19

[1,2]

Rmem = {1,2} Rob = { (1,2) }

19

[1,2] [2]

pop() 1

Rmem = {1,2} Rob = { (1,2) } Rmem = {2} Rob = { }

19

[1,2] [2] [2]

pop() 1 pop() 1

Rmem = {1,2} Rob = { (1,2) } Rmem = {2} Rob = { } Rmem = {2} Rob = { }

19

[1,2] [2] [2] [2] [2]

pop() 1 pop() 1

Rmem = {1,2} Rob = { (1,2) } Rmem = {2} Rob = { } Rmem = {2} Rob = { } Rmem = {2} Rob = { } Rmem = {2} Rob = { }

20

20

[1]

Rmem = {1} Rob = { }

20

[1] [1,2]

push(2)

Rmem = {1} Rob = { } Rmem = {1,2} Rob = { (1,2) }

20

[1] [1,2] [1,3]

push(2) push(3)

Rmem = {1} Rob = { } Rmem = {1,2} Rob = { (1,2) } Rmem = {1,3} Rob = { (1,3) }

20

[1] [1,2] [1,3]

push(2) push(3)

Rmem = {1} Rob = { } Rmem = {1,2} Rob = { (1,2) } Rmem = {1,3} Rob = { (1,3) }

Use < as an arbitration function between concurrent insertions

20

[1] [1,2] [1,3] [1,2,3] [1,3,2]

push(2) push(3)

Rmem = {1} Rob = { } Rmem = {1,2} Rob = { (1,2) } Rmem = {1,3} Rob = { (1,3) } Rmem = {1,2,3} Rob = { (1,2), (1,3), (2,3) } Rmem = {1,2,3} Rob = { (1,2), (1,3). (2,3) }

Use < as an arbitration function between concurrent insertions

Characteristic Relations

21

A sequence of relations RT is called a characteristic relation

• f a data type T, if for every x : T and y : T, RT (x) = RT (y) iff

x and y are extensionally equal as interpreted under T.

• Rmem and Rob are the characteristic relations of queue

Characteristic Relations

21

A sequence of relations RT is called a characteristic relation

• f a data type T, if for every x : T and y : T, RT (x) = RT (y) iff

x and y are extensionally equal as interpreted under T.

• Rmem and Rob are the characteristic relations of queue
• Appeals only to the sequential properties of the data type

★ Ignore distribution when defining characteristic relations.

Characteristic Relations

21

A sequence of relations RT is called a characteristic relation

• f a data type T, if for every x : T and y : T, RT (x) = RT (y) iff

x and y are extensionally equal as interpreted under T.

Synthesizing Merge

22

Synthesizing Merge

• Semantics of merge in relational domain is quite standard

across data types

22

Synthesizing Merge

• Semantics of merge in relational domain is quite standard

across data types

★ Can we synthesise merge functions for arbitrary data type? 22

Synthesizing Merge

• Semantics of merge in relational domain is quite standard

across data types

★ Can we synthesise merge functions for arbitrary data type? 22

Synthesizing Merge

• Semantics of merge in relational domain is quite standard

across data types

★ Can we synthesise merge functions for arbitrary data type? 22

Abstraction function

Synthesizing Merge

• Semantics of merge in relational domain is quite standard

across data types

★ Can we synthesise merge functions for arbitrary data type? 22

Abstraction function Concretisation function

Synthesizing Merge

• Semantics of merge in relational domain is quite standard

across data types

★ Can we synthesise merge functions for arbitrary data type? 22

Abstraction function Concretisation function Synthesize (goal)

Synthesizing Merge

• Semantics of merge in relational domain is quite standard

across data types

★ Can we synthesise merge functions for arbitrary data type? 22

Abstraction function Concretisation function

Directed synthesis using the type of the characteristic relations

Synthesize (goal)

Abstraction Function: Queue

23

Abstraction Function: Queue

23

Abstraction Function: Queue

23

Abstraction Function: Queue

23

Abstraction Function: Queue

23

Abstraction Function: Binary Tree

24

Abstraction Function: Binary Tree

24

Abstraction Function: Binary Tree

24

Abstraction Function: Binary Heap

25

Abstraction Function: Binary Heap

25

26

Compositionality: Pair

27

Compositionality: Pair

• The merge of a pair is the merge of the corresponding

constituents

27

Compositionality: Pair

• The merge of a pair is the merge of the corresponding

constituents

• A pair data type is defined by the relations:

27

Compositionality: Pair

• The merge of a pair is the merge of the corresponding

constituents

• A pair data type is defined by the relations:
• Assume that the pair is composed of 2 counters. The counter

merge spec is:

27

Compositionality: Pair

• The merge of a pair is the merge of the corresponding

constituents

• A pair data type is defined by the relations:
• Assume that the pair is composed of 2 counters. The counter

merge spec is:

• Then, pair merge spec is:

27

Generalising Pairs to Ordinates

28

Generalising Pairs to Ordinates

• An alternative characteristic relation for a pair is:

28

Generalising Pairs to Ordinates

• An alternative characteristic relation for a pair is:

28

Generalising Pairs to Ordinates

• An alternative characteristic relation for a pair is:
• Corresponding merge specification is:

28

Generalising Pairs to Ordinates

• An alternative characteristic relation for a pair is:
• Corresponding merge specification is:

28

Generalising Pairs to Ordinates

• An alternative characteristic relation for a pair is:
• Corresponding merge specification is:
• Given appropriate characteristic relation for a n-tuple (Rn-tuple),

the same merge specification can be used.

28

Generalising Pairs to Ordinates

• Similar encoding can be given to maps with non-mergeable

types as keys and mergeable types as values

29

Types of Characteristic Relations

30

Types of Characteristic Relations

• Practical characteristic relations fall into 3 types:

30

Types of Characteristic Relations

• Practical characteristic relations fall into 3 types:

★ Membership (Rmem) 30

R : {v : T} → P

• T
• , where T is a non-mergeable type

Types of Characteristic Relations

• Practical characteristic relations fall into 3 types:

★ Membership (Rmem) ★ Ordering (Rob, Rans, Rto)

✦

where 𝜍 is a sequence of non-mergeable types and other relations, which flattens to a sequence of non-mergeable types

30

R : {v : T} → P

• T
• , where T is a non-mergeable type

R : {v : T} → P (ρ)

Types of Characteristic Relations

• Practical characteristic relations fall into 3 types:

★ Membership (Rmem) ★ Ordering (Rob, Rans, Rto)

✦

where 𝜍 is a sequence of non-mergeable types and other relations, which flattens to a sequence of non-mergeable types

★ Ordinates (Rpair, Rkv) 30

R : {v : T} → P

• T
• , where T is a non-mergeable type

R : {v : T} → P (ρ)

R : {v : T} → P

• T × τ
• , where τ is a mergeable type

Deriving merge spec

31

Deriving merge spec

• Not complete, but practical
• Can derive merge spec for

★ Data structures: Set, Heap, Graph, Queue, TreeDoc ★ Larger apps: TPC-C, TPC-E, Twissandra, Rubis

31

Distributed Implementation

32

Distributed Implementation

• For making this programming model practical, we need to:

★ Quickly compute LCA ★ Optimise storage through sharing ★ Optimise network transmissions (state based merge)

32

Distributed Implementation

• For making this programming model practical, we need to:

★ Quickly compute LCA ★ Optimise storage through sharing ★ Optimise network transmissions (state based merge)

• Irmin

★ A reimplementation of Git in pure OCaml ★ Arbitrary OCaml objects, not just files + User-defined 3-way merges ★ Only transmit diffs over the network ★ Multiple storage backends including in-memory, filesystems, log-structured-

merge database, distributed databases

32

Performance

33

Performance

• What is the size of diff compared to the size of data structure?

33

Performance

• What is the size of diff compared to the size of data structure?
• Setup

★ 2 Replicas, fixed number of rounds, each round has N operations ★ 75% inserts, 25% deletions ★ Synchronise after each round 33

Performance

• What is the size of diff compared to the size of data structure?
• Setup

★ 2 Replicas, fixed number of rounds, each round has N operations ★ 75% inserts, 25% deletions ★ Synchronise after each round 33

Binary Heap Growable Array

Thanks for listening!

34