E ff ortless Eventual Consistency Gossip, CRDTs, and Weave Mesh - - PowerPoint PPT Presentation

▶
e ff ortless eventual consistency
SMART_READER_LITE
LIVE PREVIEW

E ff ortless Eventual Consistency Gossip, CRDTs, and Weave Mesh - - PowerPoint PPT Presentation

Mar 25, 2024 183 likes •2.05k views

E ff ortless Eventual Consistency Gossip, CRDTs, and Weave Mesh weave works - Outline Theory Practice Extension weave works - Outline Theory Gossip CRDT Practice Weave Mesh Weave Net Extension

slide-1
SLIDE 1

weaveworks-

Effortless Eventual Consistency

Gossip, CRDTs, and Weave Mesh

slide-2
SLIDE 2

weaveworks-

Outline

  • Theory
  • Practice
  • Extension

slide-3
SLIDE 3

weaveworks-

Outline

  • Theory


Gossip
 CRDT

  • Practice


Weave Mesh
 Weave Net

  • Extension


User data types
 Strong consistency

slide-4
SLIDE 4

weaveworks-

Motivating Example

Increment-only counter

slide-5
SLIDE 5

weaveworks-

Motivating Example

Increment-only counter

Increment

slide-6
SLIDE 6

weaveworks-

Motivating Example

Increment-only counter

1

Increment

slide-7
SLIDE 7

weaveworks-

Motivating Example

Increment-only counter

1

slide-8
SLIDE 8

weaveworks-

Motivating Example

Increment-only counter

1

Increment

slide-9
SLIDE 9

weaveworks-

Motivating Example

Increment-only counter

2

Increment

slide-10
SLIDE 10

weaveworks-

Motivating Example

Increment-only counter

2

slide-11
SLIDE 11

weaveworks-

Motivating Example

Increment-only counter

2

Increment Increment Increment Increment

slide-12
SLIDE 12

weaveworks-

Motivating Example

Increment-only counter

☹

Increment Increment Increment Increment

slide-13
SLIDE 13

weaveworks-

Motivating Example

Increment-only counter

Increment Increment Increment Increment

😰

slide-14
SLIDE 14

weaveworks-

Motivating Example

Increment-only counter

Increment Increment Increment Increment

2

slide-15
SLIDE 15

weaveworks-

Motivating Example

Increment-only counter

Increment Increment Increment Increment

2

slide-16
SLIDE 16

weaveworks-

Motivating Example

Increment-only counter

Increment Increment Increment Increment

3

slide-17
SLIDE 17

weaveworks-

Motivating Example

Increment-only counter

Increment Increment Increment

3

slide-18
SLIDE 18

weaveworks-

Motivating Example

Increment-only counter

Increment Increment Increment

3

slide-19
SLIDE 19

weaveworks-

Motivating Example

Increment-only counter

Increment Increment Increment

4

slide-20
SLIDE 20

weaveworks-

Motivating Example

Increment-only counter

Increment Increment

4

slide-21
SLIDE 21

weaveworks-

Motivating Example

Increment-only counter

Increment Increment

4

slide-22
SLIDE 22

weaveworks-

Motivating Example

Increment-only counter

Increment Increment

5

slide-23
SLIDE 23

weaveworks-

Motivating Example

Increment-only counter

Increment

5

slide-24
SLIDE 24

weaveworks-

Motivating Example

Increment-only counter

Increment

5

slide-25
SLIDE 25

weaveworks-

Motivating Example

Increment-only counter

Increment

6

slide-26
SLIDE 26

weaveworks-

Motivating Example

Increment-only counter

6

slide-27
SLIDE 27

weaveworks-

Motivating Example

Increment-only counter

6

slide-28
SLIDE 28

weaveworks-

slide-29
SLIDE 29

weaveworks-

slide-30
SLIDE 30

weaveworks-

1x 1x

slide-31
SLIDE 31

weaveworks-

slide-32
SLIDE 32

weaveworks-

1x

slide-33
SLIDE 33

weaveworks-

1x

slide-34
SLIDE 34

weaveworks-

☹ ☹ ☹ ☹ ☹ ☹ ☹

1x

slide-35
SLIDE 35

weaveworks-

Theory:

Gossip

slide-36
SLIDE 36

weaveworks-

Gossip solves…

  • Information dissemination in irregular networks
  • Unreliable links
  • Dynamic, changing topologies
slide-37
SLIDE 37

weaveworks-

Gossip requires…

  • Periodic and pairwise communication
  • Frequency of interaction ≪ individual message latency
  • Redundancy in delivered information
slide-38
SLIDE 38

weaveworks-

Gossip requires…

  • Periodic and pairwise communication
  • Frequency of interaction ≪ individual message latency
  • Redundancy in delivered information
slide-39
SLIDE 39

weaveworks-

Gossip modes

  • Push — if we know X, every round, tell a peer about X
  • Pull — every round, ask a peer about all possible X
  • Push/pull — push for some cycles, then pull for some cycles
slide-40
SLIDE 40

weaveworks-

Rumor

slide-41
SLIDE 41

weaveworks-

Rumor

slide-42
SLIDE 42

weaveworks-

Rumor

slide-43
SLIDE 43

weaveworks-

Rumor

slide-44
SLIDE 44

weaveworks-

Rumor

slide-45
SLIDE 45

weaveworks-

Rumor

slide-46
SLIDE 46

weaveworks-

Anti-entropy

slide-47
SLIDE 47

weaveworks-

Anti-entropy

slide-48
SLIDE 48

weaveworks-

Anti-entropy

slide-49
SLIDE 49

weaveworks-

Anti-entropy

slide-50
SLIDE 50

weaveworks-

Anti-entropy gossip

slide-51
SLIDE 51

weaveworks-

Aggregate

slide-52
SLIDE 52

weaveworks-

Motivating Example

Increment-only counter

slide-53
SLIDE 53

weaveworks-

Motivating Example

Increment-only counter

slide-54
SLIDE 54

weaveworks-

Motivating Example

Increment-only counter

Increment

slide-55
SLIDE 55

weaveworks-

Motivating Example

Increment-only counter

1

Increment

slide-56
SLIDE 56

weaveworks-

Motivating Example

Increment-only counter

1

slide-57
SLIDE 57

weaveworks-

Motivating Example

Increment-only counter

1

t=1

slide-58
SLIDE 58

weaveworks-

Motivating Example

Increment-only counter

1 1

t=2

slide-59
SLIDE 59

weaveworks-

Motivating Example

Increment-only counter

1 1 1 1

t=3

slide-60
SLIDE 60

weaveworks-

Motivating Example

Increment-only counter

1 1 1 1 1 1

t=4

slide-61
SLIDE 61

weaveworks-

Motivating Example

Increment-only counter

1 1 1 1 1 1 1

t=5

slide-62
SLIDE 62

weaveworks-

Motivating Example

Increment-only counter

1 1 1 1 1 1 1 1

t=6

slide-63
SLIDE 63

weaveworks-

Motivating Example

Increment-only counter

1 1 1 1 1 1 1 1 1

t=7

slide-64
SLIDE 64

weaveworks-

Motivating Example

Increment-only counter

1 1 1 1 1 1 1 1 1

t=7

slide-65
SLIDE 65

weaveworks-

Motivating Example

Increment-only counter

slide-66
SLIDE 66

weaveworks-

Motivating Example

Increment-only counter

slide-67
SLIDE 67

weaveworks-

Motivating Example

Increment-only counter

1

slide-68
SLIDE 68

weaveworks-

Motivating Example

Increment-only counter

2

slide-69
SLIDE 69

weaveworks-

Motivating Example

Increment-only counter

2

slide-70
SLIDE 70

weaveworks-

Motivating Example

Increment-only counter

2 1 1 1 1 1 1 1 1

🤕

slide-71
SLIDE 71

weaveworks-

Motivating Example

Increment-only counter

👒

slide-72
SLIDE 72

weaveworks-

Motivating Example

Increment-only counter

👒

Increment

slide-73
SLIDE 73

weaveworks-

Motivating Example

Increment-only counter

👒

Increment

slide-74
SLIDE 74

weaveworks-

Motivating Example

Increment-only counter

1

👒

Increment

slide-75
SLIDE 75

weaveworks-

Motivating Example

Increment-only counter

1

👒

Read

slide-76
SLIDE 76

weaveworks-

Motivating Example

Increment-only counter

😧 😧 😧 😧 😧 😧 😧 😧

x

👒

slide-77
SLIDE 77

weaveworks-

Theory:

CRDT

slide-78
SLIDE 78

weaveworks-

CRDTs solve…

  • Shared state in a distributed system
  • Provide availability (A)
  • Provide partition tolerance (P)
slide-79
SLIDE 79

weaveworks-

slide-80
SLIDE 80

weaveworks-

slide-81
SLIDE 81

weaveworks-

CRDTs require...

  • Operations that are
  • Associative — A • (B • C) = (A • B) • C
  • Commutative — A • B = B • A
  • Idempotent* — A • A = A
slide-82
SLIDE 82

weaveworks-

CRDTs require...

  • Operations that are
  • Associative — A • (B • C) = (A • B) • C
  • Commutative — A • B = B • A
  • Idempotent* — A • A = A

😍

slide-83
SLIDE 83

weaveworks-

CRDT

  • Conflict-free
  • Replicated
  • Data
  • Type
slide-84
SLIDE 84

weaveworks-

CmRDT

  • Commutative Replicated Data Type
  • Operation-based: transmit the op itself — "edge trigger"
  • Assumes reliable, exactly-once delivery (ha!)
slide-85
SLIDE 85

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

slide-86
SLIDE 86

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

slide-87
SLIDE 87

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

Increment

slide-88
SLIDE 88

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1

Increment

slide-89
SLIDE 89

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1

slide-90
SLIDE 90

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1

A++

slide-91
SLIDE 91

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1

A++

slide-92
SLIDE 92

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1

slide-93
SLIDE 93

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1

A++

slide-94
SLIDE 94

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1

A++

slide-95
SLIDE 95

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1

slide-96
SLIDE 96

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1

Read

slide-97
SLIDE 97

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1

Read

Σ

= 1

slide-98
SLIDE 98

weaveworks-

CvRDT

  • Convergent Replicated Data Type
  • State-based: transmit the complete state — "level trigger"
  • Achieve idempotency in fragile networks
  • All CmRDT can be expressed as CvRDT (Shapiro et. al.)
slide-99
SLIDE 99

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1

slide-100
SLIDE 100

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1

Increment

slide-101
SLIDE 101

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1

Increment

slide-102
SLIDE 102

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1

slide-103
SLIDE 103

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1

Σ Σ Σ

= 2 = 1 = 1

slide-104
SLIDE 104

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1

slide-105
SLIDE 105

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1

{A=1 B=1 C=0}

slide-106
SLIDE 106

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1 1

{A=1 B=1 C=0}

slide-107
SLIDE 107

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1 1

slide-108
SLIDE 108

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1 1

{A=1 B=1 C=0}

slide-109
SLIDE 109

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1 1 1

{A=1 B=1 C=0}

slide-110
SLIDE 110

weaveworks-

Motivating Example

Increment-only counter 
 CmRDT

1 1 1 1 1 1

slide-111
SLIDE 111

weaveworks-

δ-CRDT

  • Op-based CRDT: small/fast messages, needs impossible network
  • State-based CRDT: idempotent, but large message size
  • Delta CRDT: small messages + idempotency?
slide-112
SLIDE 112

weaveworks-

δ-CRDT

  • Delta-mutator 𝑛ᵟ is an update as a function — also a CRDT
  • Given prior state X, 𝑛ᵟ(X) encodes next state
  • Multiple 𝑛ᵟ can be merged to a set Mᵟ and transmit
  • Causal consistency only possible with constraints & anti-entropy!
slide-113
SLIDE 113

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT 1 1 1 1 1 1

slide-114
SLIDE 114

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT 1 1 1 1 1 1

Increment

slide-115
SLIDE 115

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT 1 1 1 2 1 1

Increment

slide-116
SLIDE 116

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT 1 1 1 2 1 1

slide-117
SLIDE 117

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT

A++

1 1 1 2 1 1

slide-118
SLIDE 118

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT

{A=2 B=1 C=0}

1 1 1 2 1 1

slide-119
SLIDE 119

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT

{A=2}

1 1 1 2 1 1

slide-120
SLIDE 120

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT

{A=2}

1 1 1 2 2 1

slide-121
SLIDE 121

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT 1 1 1 2 2 1

slide-122
SLIDE 122

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT

{A=2}

1 1 1 2 2 1

slide-123
SLIDE 123

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT

{A=2}

1 1 1 2 2 2

slide-124
SLIDE 124

weaveworks-

Motivating Example

Increment-only counter 


δ-CRDT 1 1 1 2 2 2

slide-125
SLIDE 125

weaveworks-

Theory → Practice

slide-126
SLIDE 126

weaveworks-

Communication Theory + Information Theory = System Theory Gossip
 + 
 CRDT
 =
 weavemesh

→

slide-127
SLIDE 127

weaveworks-

Practice:

weavemesh

slide-128
SLIDE 128

weaveworks-

Communication abstraction

type GossipSender interface { GossipUnicast(dst Peer, m Msg) (err error) GossipBroadcast(m Msg) } type GossipReceiver interface { OnGossipUnicast(src Peer, m Msg) (err error) OnGossipBroadcast(src Peer, m Msg) (received Msgs, err error) OnGossip(m Msg) (delta Msgs, err error) DumpState() (complete Msgs) }

slide-129
SLIDE 129

weaveworks-

Data abstraction

type GossipData interface { Encode() Msgs Merge(other GossipData) (result GossipData) }

slide-130
SLIDE 130

weaveworks-

Communication component

Router B Router C Router A Router D

slide-131
SLIDE 131

weaveworks-

Communication component

Router B Router C Router A Router D Channel 1 Channel 1 Channel 1 Channel 1

slide-132
SLIDE 132

weaveworks-

Communication component

Router B Router C Router A Router D Channel 1 Channel 1 Channel 1 Channel 1 Gossip Receiver Gossip Receiver Gossip Receiver Gossip Receiver

slide-133
SLIDE 133

weaveworks-

Communication component

Router B Router C Router A Router D Channel 1 Channel 1 Channel 1 Channel 1 Gossip Receiver Gossip Receiver Gossip Receiver Gossip Receiver

slide-134
SLIDE 134

weaveworks-

Communication component

Channel GossipSender User GossipReceiver Router

slide-135
SLIDE 135

weaveworks-

Router

Communication component

Channel GossipSender User GossipReceiver User GossipData

slide-136
SLIDE 136

weaveworks-

Router

Communication component

Channel GossipSender Node User GossipData OnGossipUnicast
 OnGossipBroadcast
 OnGossip
 DumpState

slide-137
SLIDE 137

weaveworks-

Router

Communication component

Channel GossipSender Node State OnGossipUnicast
 OnGossipBroadcast
 OnGossip
 DumpState Merge

slide-138
SLIDE 138

weaveworks-

Router

Communication component

Channel GossipSender Node State OnGossipUnicast
 OnGossipBroadcast
 OnGossip
 DumpState Merge Encode API

slide-139
SLIDE 139

weaveworks-

Router

Communication component

Channel GossipSender Node State OnGossipUnicast
 OnGossipBroadcast
 OnGossip
 DumpState Merge Encode API

slide-140
SLIDE 140

weaveworks-

Practice:

weavenet

slide-141
SLIDE 141

weaveworks-

DC1 DC2 DC3 weavenet

slide-142
SLIDE 142

weaveworks-

weavenet

  • Weave Net depends on Weave Mesh for…
  • Peer discovery and routing
  • IP address management
  • Service discovery and load balancing
  • All of which is scalable, available, and partition-tolerant
slide-143
SLIDE 143

weaveworks-

weavedns

  • Starting a named container anywhere

hostA$ docker run --name=foo ...

  • Makes that name resolvable everywhere

containerX@hostB$ ping foo
 containerY@hostC$ ping foo

slide-144
SLIDE 144

weaveworks-

Demo

slide-145
SLIDE 145

weaveworks-

weavedns

  • Scalable
  • Available
  • Partition tolerant
  • Eventually consistent
  • Fast
  • Thanks to Weave Mesh
slide-146
SLIDE 146

weaveworks-

Practice → Extension

slide-147
SLIDE 147

weaveworks-

Extension:

User data types

slide-148
SLIDE 148

weaveworks-

type GossipData interface { Encode() Msgs Merge(other GossipData) (result GossipData) }

slide-149
SLIDE 149

weaveworks-

type GossipData interface { Encode() Msgs Merge(other GossipData) (result GossipData) } type myData struct { m map[Node]int }

slide-150
SLIDE 150

weaveworks-

type GossipData interface { Encode() Msgs Merge(other GossipData) (result GossipData) } type myData struct { m map[Node]int } func (d myData) Encode() Msgs { return Msgs{json.Encode(d.m)} // single msg, complete state }

slide-151
SLIDE 151

weaveworks-

func (d myData) Merge(other GossipData) (result GossipData) { theirs = other.(myData) for node, val := range theirs.m { if val > d.m[node] { d.m[node] = val } } return d }

slide-152
SLIDE 152

weaveworks-

type GossipReceiver interface { OnGossipUnicast(src Peer, m Msg) (err error) OnGossipBroadcast(src Peer, m Msg) (received Msgs, err error) OnGossip(m Msg) (delta Msgs, err error) DumpState() (complete Msgs) }

slide-153
SLIDE 153

weaveworks-

type GossipReceiver interface { OnGossipUnicast(src Peer, m Msg) (err error) OnGossipBroadcast(src Peer, m Msg) (received Msgs, err error) OnGossip(m Msg) (delta Msgs, err error) DumpState() (complete Msgs) } func OnGossipUnicast(src Peer, m Msg) (err error) { update := json.UnmarshalFrom(m) state.Merge(update) return nil }

slide-154
SLIDE 154

weaveworks-

func OnGossipBroadcast(src Peer, m Msg) (received Msgs, err error) { update := json.UnmarshalFrom(m) received := state.MergeReceived(update) return received, nil }

slide-155
SLIDE 155

weaveworks-

func OnGossipBroadcast(src Peer, m Msg) (received Msgs, err error) { update := json.UnmarshalFrom(m) received := state.MergeReceived(update) return received, nil } func OnGossip(m Msg) (delta Msgs, err error) { update := json.UnmarshalFrom(m) delta := state.MergeDelta(update) return delta, err }

slide-156
SLIDE 156

weaveworks-

func OnGossipBroadcast(src Peer, m Msg) (received Msgs, err error) { update := json.UnmarshalFrom(m) received := state.MergeReceived(update) return received, nil } func OnGossip(m Msg) (delta Msgs, err error) { update := json.UnmarshalFrom(m) delta := state.MergeDelta(update) return delta, err } func DumpState() (complete Msgs) { return json.Marshal(state) }

slide-157
SLIDE 157

weaveworks-

type GossipSender interface { GossipUnicast(dst Peer, m Msg) (err error) GossipBroadcast(m Msg) }

slide-158
SLIDE 158

weaveworks-

type GossipSender interface { GossipUnicast(dst Peer, m Msg) (err error) GossipBroadcast(m Msg) } func (d myData) Increment() { d.m[self]++ // make local state change update := d.Encode() // create state-based CRDT update GossipBroadcast(update) // eventual consistency }

slide-159
SLIDE 159

weaveworks-

type GossipSender interface { GossipUnicast(dst Peer, m Msg) (err error) GossipBroadcast(m Msg) } func (d myData) Increment() { d.m[self]++ // make local state change update := d.Encode() // create state-based CRDT update GossipBroadcast(update) // eventual consistency } func (d myData) Read() (result int) { for _, val := range d.m { result += val } return result }

slide-160
SLIDE 160

weaveworks-

Router

Component diagram

Channel GossipSender User GossipReceiver User GossipData

slide-161
SLIDE 161

weaveworks-

Router

Component diagram

Channel GossipSender User GossipReceiver myData Merge Encode Read
 Incr

slide-162
SLIDE 162

weaveworks-

Node OnGossipUnicast
 OnGossipBroadcast
 OnGossip
 DumpState Router

Component diagram

Channel GossipSender myData Merge Encode Read
 Incr

slide-163
SLIDE 163

weaveworks-

Extension:

Strong consensus

slide-164
SLIDE 164

weaveworks-

Intermediary:

UDP — net.PacketConn

slide-165
SLIDE 165

weaveworks-

type PacketConn interface { ReadFrom(b []byte) (n int, addr Addr, err error) WriteTo(b []byte, addr Addr) (n int, err error) Close() error LocalAddr() Addr SetDeadline(t time.Time) error SetReadDeadline(t time.Time) error SetWriteDeadline(t time.Time) error }

slide-166
SLIDE 166

weaveworks-

type PacketConn interface { ReadFrom(b []byte) (n int, addr Addr, err error) WriteTo(b []byte, addr Addr) (n int, err error) Close() error LocalAddr() Addr SetDeadline(t time.Time) error SetReadDeadline(t time.Time) error SetWriteDeadline(t time.Time) error }

slide-167
SLIDE 167

weaveworks-

Component diagram

net.PacketConn GossipReceiver

Raft Mesh

slide-168
SLIDE 168

weaveworks-

Component diagram

ReadFrom OnGossipUnicast OnGossipBroadcast OnGossip Channel GossipSender WriteTo

slide-169
SLIDE 169

weaveworks-

// Called from network driver to recv a packet. func (p *Peer) ReadFrom(p []byte) (..., remote Addr, ...) { pkt := <-p.recv // from mesh network copy(p, pkt) // copy data into p return ..., pkt.Src, ... // pkt.Src addr is remote }

slide-170
SLIDE 170

weaveworks-

// Called from network driver to recv a packet. func (p *Peer) ReadFrom(p []byte) (..., remote Addr, ...) { pkt := <-p.recv // from mesh network copy(p, pkt) // copy data into p return ..., pkt.Src, ... // pkt.Src addr is remote } // Called from network driver to send a packet. func (p *Peer) WriteTo(b []byte, dst Addr) ... { pkt := Pkt{Src: p.Self, Dst: dst, Buf: b} p.sender.GossipUnicast(dst, pkt.Encode()) }

slide-171
SLIDE 171

weaveworks-

func (p *Peer) OnGossipBroadcast(src Peer, m Msg) ... { p.recv <- makePkt(m) } func (p *Peer) OnGossipUnicast(src Peer, m Msg) ... { p.recv <- makePkt(m) } func (p *Peer) OnGossip(m Msg) ... { p.recv <- makePkt(m) } func (p *Peer) DumpState() ... { return nil // stateless }

slide-172
SLIDE 172

weaveworks-

Weak Consistency + Adapter = Strong Consistency Mesh
 + 
 MeshConn
 =
 Raft

→

slide-173
SLIDE 173

weaveworks-

etcd-flavored Raft

Node

Step ProposeEntry ProposeConfChange

slide-174
SLIDE 174

weaveworks-

etcd-flavored Raft

Node

Step ProposeEntry ProposeConfChange

Packet
 Oriented
 Transport

slide-175
SLIDE 175

weaveworks-

etcd-flavored Raft

Node

Step ProposeEntry ProposeConfChange

Packet Oriented Transport

slide-176
SLIDE 176

weaveworks-

etcd-flavored Raft

Node

Step ProposeEntry ProposeConfChange

Packet Oriented Transport State

slide-177
SLIDE 177

weaveworks-

Controller

etcd-flavored Raft

Node

Step ProposeEntry ProposeConfChange

Packet Oriented Transport State

slide-178
SLIDE 178

weaveworks-

etcd-flavored Raft

  • Weakly-consistent substrate — Mesh, gossip
  • Adapter — MeshConn, UDP
  • Strongly-consistent semantics — etcd, Raft
slide-179
SLIDE 179

weaveworks-

Extension → Desire?

slide-180
SLIDE 180

weaveworks-

Theory

slide-181
SLIDE 181

weaveworks-

Theory Practice

▶ ︎

slide-182
SLIDE 182

weaveworks-

Theory Practice Extension

▶ ︎ ▶︎

slide-183
SLIDE 183

weaveworks-

Theory Practice Extension Desire

▶ ︎ ▶︎ ▶ ︎ ▶︎

slide-184
SLIDE 184

weaveworks-

Theory Practice Extension Desire

▶ ︎ ▶︎ ▶ ︎ ▶︎

😍

slide-185
SLIDE 185

weaveworks-

Thank you!

Matthias Radestock, CTO Peter Bourgon, Typist

https://weave.works scope.weave.works

weavescope