In fi nite Parallel Universes: State at the Edge Peter Bourgon - - PowerPoint PPT Presentation

▶
in fi nite parallel universes
SMART_READER_LITE
LIVE PREVIEW

In fi nite Parallel Universes: State at the Edge Peter Bourgon - - PowerPoint PPT Presentation

Jun 01, 2023 97 likes •696 views

In fi nite Parallel Universes: State at the Edge Peter Bourgon Fastly In fi nite Parallel Universes: State at the Edge Peter Bourgon Fastly @ peterbourgon Fastly In fi nite Parallel Universes Context Architecture Protocol

slide-1
SLIDE 1

Infinite Parallel Universes:

State at the Edge

Peter Bourgon · Fastly

slide-2
SLIDE 2

Infinite Parallel Universes:

State at the Edge

Peter Bourgon · Fastly

slide-3
SLIDE 3

Infinite Parallel Universes

Context Architecture Protocol Complications Conclusions

@peterbourgon · Fastly

slide-4
SLIDE 4

@peterbourgon

slide-5
SLIDE 5

Infinite Parallel Universes · Context

@peterbourgon · Fastly

https://www.fastly.com/press/press-releases/fastly-expands-serverless-capabilities-launch-compute-edge

slide-6
SLIDE 6

Infinite Parallel Universes · Context

If Fastly provides compute at the edge, what about state?

@peterbourgon · Fastly

slide-7
SLIDE 7

Infinite Parallel Universes · Context

@peterbourgon · Fastly

as of 31 Dec 2019 · https://www.fastly.com/network-map

slide-8
SLIDE 8

Infinite Parallel Universes · Context

133ms

@peterbourgon · Fastly

as of 31 Dec 2019 · https://www.fastly.com/network-map

slide-9
SLIDE 9

Infinite Parallel Universes · Context

What are we doing here

  • Not a general purpose database
  • Operate in the request lifecycle
  • Local (POP) reads and writes
  • Eventually consistent
  • State conflicts are normal

⇒ Sort of a writeable cache

@peterbourgon · Fastly

slide-10
SLIDE 10

Infinite Parallel Universes · Context

Data model

  • Lean in to the physical constraints
  • A single, global truth? — No, a fiction!
  • Multiple, simultaneous truths — Reality!
  • Converge toward a stable global state

@peterbourgon · Fastly

slide-11
SLIDE 11

Infinite Parallel Universes · Context

State primitive

  • CRDT
  • Type-specific methods + Merge
  • Associative, commutative, idempotent
  • Tolerates out-of-order, duplicate merges

⇒ Reduces higher-order complexity

@peterbourgon · Fastly

slide-12
SLIDE 12

Infinite Parallel Universes · Context

Integer addition

  • Associative? (a + b) + c = a + (b + c) ✔
  • Commutative? a + b = b + a ✔
  • Idempotent? a + a = a ✘

⇒ Not a CRDT

@peterbourgon · Fastly

slide-13
SLIDE 13

Infinite Parallel Universes · Context

Set union

  • Associative? ({a} ∪ {b}) ∪ {c} = {a} ∪ ({b} ∪ {c}) — Yes
  • Commutative? {a} ∪ {b} = {b} ∪ {a} — Yes
  • Idempotent? {a} ∪ {a} = {a} — Yes

⇒ A CRDT

@peterbourgon · Fastly

slide-14
SLIDE 14

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1}

slide-15
SLIDE 15

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1}

slide-16
SLIDE 16

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6

slide-17
SLIDE 17

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6}

slide-18
SLIDE 18

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4}

slide-19
SLIDE 19

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4 5 6}

slide-20
SLIDE 20

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6} {1 3 4 5 6}

slide-21
SLIDE 21

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 3 4 5 6} {1 2 3 4 5 6}

slide-22
SLIDE 22

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 3 4 5 6} {1 2 3 4 5 6} {1 2 3 4 5 6}

slide-23
SLIDE 23

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 2 3 4 5 6}

slide-24
SLIDE 24

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 2 3 4 5 6}

slide-25
SLIDE 25

Infinite Parallel Universes · Context

@peterbourgon · Fastly

{1} {1} {1} {1} 2 3 3 4 4 5 6 {1 2 3} {1 3 4} {1 4 5 6} {1 2 3 4} {1 3 4 5 6} {1 2 3 4 5 6} {1 2 3 4 5 6}

Join semilattice

slide-26
SLIDE 26

Infinite Parallel Universes

Context Architecture Protocol Complications Conclusions

@peterbourgon · Fastly

slide-27
SLIDE 27

Infinite Parallel Universes · Architecture

@peterbourgon · Fastly

AKL AMS YVR JHB HND EZE

as of 31 Dec 2019 · https://www.fastly.com/network-map

slide-28
SLIDE 28

Write primary AKL AMS YVR

···

Read replica Read replica Read replica

Write

  • Familiar. Consistent. Slow. Doesn't satisfy local read/write criteria.
  • 1. Single write primary

Infinite Parallel Universes · Architecture

@peterbourgon · Fastly

slide-29
SLIDE 29

AKL AMS YVR

···

Replica Replica Replica

Write

  • 2. Gossip network

Lots of connections. Where do objects live? How do objects propagate? etc.

Infinite Parallel Universes · Architecture

@peterbourgon · Fastly

slide-30
SLIDE 30

AKL AMS YVR

···

Replica Replica Replica Upstream

Write

  • 3. Hub-and-spoke

Fewer connections. Objects live in root. Replica like LRU cache.

Infinite Parallel Universes · Architecture

@peterbourgon · Fastly

slide-31
SLIDE 31

Infinite Parallel Universes

Context Architecture Protocol Complications Conclusions

@peterbourgon · Fastly

slide-32
SLIDE 32

A B C

Infinite Parallel Universes · Protocol

Protocol (i)

  • How do objects get from A to B?
  • Track objects that receive any request
  • Batch them into epochs
  • Regularly emit batches

@peterbourgon · Fastly

slide-33
SLIDE 33

A B C

Infinite Parallel Universes · Protocol

Protocol (ii)

  • How do objects get from B to C?
  • Bad answer: push from B to C
  • Key insight: sites know which objects have been requested
  • Better answer: pull relevant objects to C from B — sync

@peterbourgon · Fastly

slide-34
SLIDE 34

Site

  • 1. Write K1=V1
  • 2. Read K2: miss
  • 3. Write K3=V3

Site

Interest {K1 K2 K3} State {K1:V1 K3:V3}

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-35
SLIDE 35

Site

Interest {K1 K2 K3} State {K1:V1 K3:V3} SyncRequest {K1:V1 K2:-- K3:V3} Sync step 1 After some time, make a SyncRequest with all interesting keys and values. + =

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-36
SLIDE 36

Site

SyncRequest {K1:V1 K2:-- K3:V3}

Upstream

Sync step 2 Send the SyncRequest to the upstream.

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-37
SLIDE 37

SyncRequest {K1:V1 K2:-- K3:V3}

Upstream

State {K1:V5 K2:V6 K4:V7} Sync step 3 Upstream merges incoming data to its own data. New state {K1:V5⋃V1=V8 K2:V6 K3:V3 K4:V7} ⋃ =

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-38
SLIDE 38

Site

SyncRequest {K1:V1 K2:-- K3:V3}

Upstream

SyncResponse {K1:V8 K2:V6 K3:V3} Sync step 4 Upstream returns a SyncResponse to the site, with latest value for each requested key.

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-39
SLIDE 39

Sync step 5 Site merges the SyncResponse to its own state, and resets the interest set. New state {K1:V1⋃V8=V8 K2:V6 K3:V3⋃V3=V3} ⋃ = State {K1:V1 K3:V3} SyncResponse {K1:V8 K2:V6 K3:V3}

Site

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-40
SLIDE 40

Site Upstream

State {K1:V8 K2:V6 K3:V3 K4:V7} Properties of sync (i)

  • Synchronous
  • The only way data moves
  • Bandwidth minimized
  • Data sets minimized
  • Authoritative upstream

State {K1:V8 K2:V6 K3:V3}

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-41
SLIDE 41

Site Upstream

State {K1:V8 K2:V6 K3:V3 K4:V7} Properties of sync (ii)

  • Schedule is flexible
  • Missed syncs impact liveness,

not correctness

  • Eventually consistent
  • Quiet by default

State {K1:V8 K2:V6 K3:V3}

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-42
SLIDE 42

Upstream Site Site Site

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-43
SLIDE 43

Upstream Site Site

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-44
SLIDE 44

Incoming user requests

  • Requests are hashed and proxied to N replicas
  • Response is the union of all returned CRDTs
  • Compare union with individual responses
  • Discrepant replicas easily identified, fixed

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-45
SLIDE 45

Upstream Site Site

Infinite Parallel Universes · Protocol

?

@peterbourgon · Fastly

slide-46
SLIDE 46

Outgoing sync requests

  • Each instance has different set of objects
  • But CRDTs tolerate over-merging
  • Instances may sync totally independently
  • Upstream state converges to stability

Upstream

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-47
SLIDE 47

Infinite Parallel Universes · Protocol

? ? ? ?

@peterbourgon · Fastly

slide-48
SLIDE 48

Infinite Parallel Universes · Protocol

Incoming sync requests

  • User request = 1 key · Op
  • Sync = N keys · Merge
  • Otherwise identical
  • Choose any upstream replica

@peterbourgon · Fastly

slide-49
SLIDE 49

Fractal design

  • N-ary tree
  • Can be hierarchical
  • May insulate against

regional connectivity issues

  • Tradeoff between liveness

and capacity

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-50
SLIDE 50

Smart primitives, simple systems

  • One operation for moving data
  • No coördination between
  • Faults need no handling besides retry

⇒ All from CRDT properties

Infinite Parallel Universes · Protocol

@peterbourgon · Fastly

slide-51
SLIDE 51

Infinite Parallel Universes

Context Architecture Protocol Complications Conclusions

@peterbourgon · Fastly

slide-52
SLIDE 52

Infinite Parallel Universes · Complications

@peterbourgon · Fastly

Complication: Read miss

  • First read of an object will likely miss
  • Object may "appear" later
  • Approach: blocking sync request to upstream
  • Better UX, worse latency
slide-53
SLIDE 53

Infinite Parallel Universes · Complications

@peterbourgon · Fastly

Complication: API design

  • CRDTs can be nonintuitive to program with
  • Applications really like a single global truth
  • General-purpose CRDT-based state layer not obvious
  • Approach: narrowly-scoped APIs × time
slide-54
SLIDE 54

Infinite Parallel Universes · Complications

@peterbourgon · Fastly

Complication: Resources

  • Reliability via duplication of effort — cost to bandwidth
  • CRDTs larger than normal data types — cost to storage
  • Approach: careful attention to cost-per-byte
slide-55
SLIDE 55

Infinite Parallel Universes · Complications

@peterbourgon · Fastly

Complication: Quantum entanglement

  • Quantum entanglement renders all of this irrelevant
  • Pro: easy, instantaneous communication invariant to distance
  • Con: I'm probably out of a job
  • Approach: 🤸
slide-56
SLIDE 56

Infinite Parallel Universes

Context Architecture Protocol Complications Conclusions

@peterbourgon · Fastly

slide-57
SLIDE 57

Infinite Parallel Universes · Conclusions

@peterbourgon · Fastly

State at the edge — IMO

  • There are inescapable constraints at large physical scale
  • Old abstractions (single global truth) break down
  • New abstractions (multiple parallel truths) necessary
  • Reliable systems require more robust primitives (CRDTs)
slide-58
SLIDE 58

Infinite Parallel Universes:

State at the Edge

Peter Bourgon · Fastly