[PPT] - Beyond Replicated Storage: Eventually-Consistent Distributed Data PowerPoint Presentation

SLIDE 1

Beyond Replicated Storage: Eventually-Consistent Distributed Data Structures

Konrad Iwanicki University of Warsaw

PaPEC 2014, Amsterdam, the Netherlands, April 13th, 2014

SLIDE 2

What we do

Extreme distributed systems

SLIDE 3

What we do

Extreme distributed systems
Based on wireless sensor networks

SLIDE 4

What we do

Extreme distributed systems
Based on wireless sensor networks:

– Hundreds or even thousands of nodes in a network – A single node is severely constrained in resources

SLIDE 5

Extreme distributed systems

More complex than sense-and-send:
Sensing
Analyzing and deciding
Actuating

SLIDE 6

Extreme distributed systems

More complex than sense-and-send:
Sensing
Analyzing and deciding
Actuating

collaborative

SLIDE 7

Extreme distributed systems

More complex than sense-and-send:
Sensing
Analyzing and deciding
Actuating
Subject to various challenges:
Resource constraints
Unreliable communication
Interactions with the surrounding environment

collaborative

SLIDE 8

Extreme distributed systems

More complex than sense-and-send:
Sensing
Analyzing and deciding
Actuating
Subject to various challenges:
Resource constraints
Unreliable communication
Interactions with the surrounding environment
Distributed algorithms are increasingly complex

➔ e.g., employ specific organizations.

collaborative

SLIDE 9

Cluster hierarchy

Wireless connectivity between nodes.

SLIDE 10

Cluster hierarchy

P L B Q G K C R E D J H F O M N A

Each node forms a level-0 cluster

f which it becomes the head.

SLIDE 11

Cluster hierarchy

P.L L.L I.H B.G Q.G G.G K.G C.G R.G E.G D.D J.D H.H F.H O.H M.H N.H A.L

Proximate level-0 clusters are grouped into level-1 clusters.

SLIDE 12

Cluster hierarchy

P.L.G L.L.G I.H.G B.G.G Q.G.G G.G.G K.G.G C.G.G R.G.G E.G.G D.D.G J.D.G H.H.G F.H.G O.H.G M.H.G N.H.G A.L.G

And so on at higher levels, typically until a single cluster remains.

SLIDE 13

Cluster hierarchy

P.L.G L.L.G I.H.G B.G.G Q.G.G G.G.G K.G.G C.G.G R.G.G E.G.G D.D.G J.D.G H.H.G F.H.G O.H.G M.H.G N.H.G A.L.G

The membership of a node in the hierarchy is reflected in the node’s label.

SLIDE 14

Cluster hierarchy

P.L.G L.L.G I.H.G B.G.G Q.G.G G.G.G K.G.G C.G.G R.G.G E.G.G D.D.G J.D.G H.H.G F.H.G O.H.G M.H.G N.H.G A.L.G

Each node also maintains information for each sibling cluster in the hierarchy (e.g., a routing entry).

SLIDE 15

The problem

Using the hierarchy is relatively easy:
Routing
Aggregation
In-network storage
Maintaining it is a different story.
Connectivity changes
Node failures & arrivals
Nodes should be autonomous

SLIDE 16

General scheme – gossiping

Each node maintains its state which it occasionally

updates.

For communication:
Each node operates in rounds.
In each round, it broadcasts its state to its neighbors.
It also receives the neighbors' states, which it merges

with its own one.

time a round Tx Rx

SLIDE 17

Observation

Gossiping can be efficient

SLIDE 18

Observation

Gossiping can be efficient, but...
… it makes it difficult to control when a bit of

information reaches a particular node.

SLIDE 19

Observation

Gossiping can be efficient, but...
… it makes it difficult to control when a bit of

information reaches a particular node.

We have:
Updates done by nodes
Lazy update propagation

SLIDE 20

Observation

Gossiping can be efficient, but...
… it makes it difficult to control when a bit of

information reaches a particular node.

We have:
Updates done by nodes
Lazy update propagation

Resemblance to eventually-consistent replicated storage systems.

SLIDE 21

Observation

Gossiping can be efficient, but...
… it makes it difficult to control when a bit of

information reaches a particular node.

We have:
Updates done by nodes
Lazy update propagation

Resemblance to eventually-consistent replicated storage systems.

Let's thus take a look at the problem of cluster hierarchy maintenance from the eventual consistency perspective.

SLIDE 22

EC perspective

We treat the cluster hierarchy as a distributed

structure.

The sate of each node
label
routing table

is a part of this structure.

Each node can autonomously update its local state:
Locally altering the structure.
The updates propagate through gossiping:
Eventually the structure becomes consistent globally.

SLIDE 23

EC perspective

P.L.G L.L.G I.H.G B.G.G Q.G.G G.G.G K.G.G C.G.G R.G.G E.G.G D.D.G J.D.G H.H.G F.H.G O.H.G M.H.G N.H.G A.L.G

Consider node labels.

SLIDE 24

EC perspective

A L P B C E G K Q R D J F H I M N O L G D H G

They can be viewed as a distributed tree.

SLIDE 25

EC perspective

What is different from the “traditional” model?
The state of each node is not a replica.
On the contrary:

– Some of its parts are unique. – Some are replicated at other nodes (to a varying degree).

On the global scale the states of all nodes should

form a coherent structure.

SLIDE 26

EC perspective

A L P B C E G K Q R D J F H I M N O L G D H G