Bloom: Using disorderly programming to build eventually- consistent - - PowerPoint PPT Presentation

▶
bloom using disorderly programming to build eventually
SMART_READER_LITE
LIVE PREVIEW

Bloom: Using disorderly programming to build eventually- consistent - - PowerPoint PPT Presentation

Sep 02, 2023 253 likes •1.11k views

Bloom: Using disorderly programming to build eventually- consistent distributed systems Bill Marczak UC Berkeley Why is dist. programming hard? Why is dist. programming hard? Concurrency Why is dist. programming hard? Concurrency

slide-1
SLIDE 1

Bloom: Using disorderly programming to build eventually- consistent distributed systems

Bill Marczak UC Berkeley

slide-2
SLIDE 2

Why is dist. programming hard?

slide-3
SLIDE 3

Why is dist. programming hard?

  • Concurrency
slide-4
SLIDE 4

Why is dist. programming hard?

  • Concurrency
  • Asynchrony
  • Delay
  • Re-ordering
  • Batching
slide-5
SLIDE 5

Why is dist. programming hard?

  • Concurrency
  • Asynchrony
  • Delay
  • Re-ordering
  • Batching
  • Partial failure
slide-6
SLIDE 6

Why is dist. programming hard?

  • Concurrency
  • Asynchrony
  • Delay
  • Re-ordering
  • Batching
  • Partial failure

Non-determinism

slide-7
SLIDE 7

State of the art

  • “Strong consistency” (e.g., 2PC, Paxos)
  • Global sync
  • High price
  • Prohibit interleavings; guaranteed consistency
slide-8
SLIDE 8

State of the art

  • “Strong consistency” (e.g., 2PC, Paxos)
  • Global sync
  • High price
  • Prohibit interleavings; guaranteed consistency
  • “Loose” (eventual) consistency
  • Write program to avoid sync
  • Apply “ad-hoc genius”
slide-9
SLIDE 9

State of the art

  • “Strong consistency” (e.g., 2PC, Paxos)
  • Global sync
  • High price
  • Prohibit interleavings; guaranteed consistency
  • “Loose” (eventual) consistency
  • Write program to avoid sync
  • Apply “ad-hoc genius”
slide-10
SLIDE 10

Bloom

  • New language + analysis to check for

consistency (confluence)

  • Internal DSL in Ruby
slide-11
SLIDE 11

Bloom

  • New language + analysis to check for

consistency (confluence)

  • Internal DSL in Ruby
  • Why is this in the DB track?
  • Uses intuition from Datalog

– recursive SQL – FO[LFP]

  • “Disorderly evaluation”: rule-based language; set

abstraction

slide-12
SLIDE 12

Outline

  • Flavor of BLOOM: Shopping cart
  • Static analysis
  • A better shopping cart
slide-13
SLIDE 13

Bloom rules

table Default persist scratch Default delete channel Remote scratch collection

  • p

collection expr

slide-14
SLIDE 14

Bloom rules

table Default persist scratch Default delete channel Remote scratch <= Derive <+ Insert <- Delete <~ Send collection

  • p

collection expr

slide-15
SLIDE 15

Bloom rules

table Default persist scratch Default delete channel Remote scratch <= Derive <+ Insert <- Delete <~ Send map filter join (*) not include group ... collection

  • p

collection expr

slide-16
SLIDE 16

Operational Semantics

setup: scratches emptied, network messages placed in channels

slide-17
SLIDE 17

Operational Semantics

setup: scratches emptied, network messages placed in channels logic: Bloom <= rules evaluated until fixpoint

slide-18
SLIDE 18

Operational Semantics

setup: scratches emptied, network messages placed in channels logic: Bloom <= rules evaluated until fixpoint transition: items derived by <+ and <- inserted/deleted from collections, items derived by <~ sent

slide-19
SLIDE 19

Shopping cart

  • Insert & delete items
  • Check out
  • Receive summary

server client client

slide-20
SLIDE 20

Shopping cart

  • Abstract cart protocol

(messages exchanged between client & server)

  • Concrete implementation of cart server
  • Abstract key/value store
  • Concrete key/value store
slide-21
SLIDE 21

Abstract cart protocol

module CartProtocol state do end end

slide-22
SLIDE 22

Abstract cart protocol

module CartProtocol state do end end

slide-23
SLIDE 23

Abstract cart protocol

module CartProtocol state do end end

slide-24
SLIDE 24

Abstract cart protocol

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] end end

slide-25
SLIDE 25

Abstract cart protocol

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] end end

slide-26
SLIDE 26

Abstract cart protocol

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] end end

slide-27
SLIDE 27

Abstract cart protocol

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] end end server: the IP address of the cart server client: the IP address of the cart client reqid: a unique identifier for the request item: the shopping cart item to modify action: add or delete from cart

slide-28
SLIDE 28

Abstract cart protocol

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] end end server: the IP address of the cart server client: the IP address of the cart client reqid: a unique identifier for the request item: the shopping cart item to modify action: add or delete from cart right arrow indicates functional dependency at sign “@” indicates location of tuple

slide-29
SLIDE 29

Abstract cart protocol

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] channel :checkout_msg, [:@server, :client, :reqid] end end

slide-30
SLIDE 30

Abstract cart protocol

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] channel :checkout_msg, [:@server, :client, :reqid] channel :response_msg, [:@client, :server, :item] => [:cnt] end end

slide-31
SLIDE 31

Shopping cart

  • Abstract cart protocol

(messages exchanged between client & server)

  • Concrete implementation of cart server
  • Abstract key/value store
  • Concrete key/value store
slide-32
SLIDE 32

Abstract KVS

module KVSProtocol state do end end

slide-33
SLIDE 33

Abstract KVS

module KVSProtocol state do interface input, :kvput, [:key] => [:value] end end

slide-34
SLIDE 34

Abstract KVS

module KVSProtocol state do interface input, :kvput, [:key] => [:value] interface input, :kvget, [:reqid] => [:key] end end

slide-35
SLIDE 35

Abstract KVS

module KVSProtocol state do interface input, :kvput, [:key] => [:value] interface input, :kvget, [:reqid] => [:key] interface output, :kvget_response, [:reqid] => [:key, :value] end end

slide-36
SLIDE 36

Shopping cart

  • Abstract cart protocol

(messages exchanged between client & server)

  • Concrete implementation of cart server
  • Abstract key/value store
  • Concrete key/value store
slide-37
SLIDE 37

Concrete Server

module DestructiveCart include CartProtocol include KVSProtocol end

slide-38
SLIDE 38

Concrete Server

module DestructiveCart include CartProtocol include KVSProtocol bloom :queueing do end end

slide-39
SLIDE 39

Concrete Server

module DestructiveCart include CartProtocol include KVSProtocol bloom :queueing do kvget <= action_msg {|a| [a.reqid, a.client] } end end

slide-40
SLIDE 40

Concrete Server

module DestructiveCart include CartProtocol include KVSProtocol bloom :queueing do kvget <= action_msg {|a| [a.reqid, a.client] } kvput <= action_msg do |a| if a.action == "Add" and not kvget_response.map{|b| b.key}.include? a.client [a.client, Array.new.push(a.item)] end end end end

slide-41
SLIDE 41

Concrete Server

module DestructiveCart include CartProtocol include KVSProtocol bloom :queueing do kvget <= action_msg {|a| [a.reqid, a.client] } kvput <= action_msg do |a| if a.action == "Add" and not kvget_response.map{|b| b.key}.include? a.client [a.client, Array.new.push(a.item)] end end temp :old_state <= (kvget_response * action_msg).pairs(:key => :client) kvput <= old_state do |b, a| if a.action == "Add" [a.client, (b.value.clone.push(a.item))] elsif a.action == "Del" [a.client, delete_one(b.value, a.item)] end end end end

slide-42
SLIDE 42

Concrete Server

module DestructiveCart include CartProtocol include KVSProtocol bloom :queueing do kvget <= action_msg {|a| [a.reqid, a.client] } kvput <= action_msg do |a| if a.action == "Add" and not kvget_response.map{|b| b.key}.include? a.client [a.client, Array.new.push(a.item)] end end temp :old_state <= (kvget_response * action_msg).pairs(:key => :client) kvput <= old_state do |b, a| if a.action == "Add" [a.client, (b.value.clone.push(a.item))] elsif a.action == "Del" [a.client, delete_one(b.value, a.item)] end end end bloom :finish do kvget <= checkout_msg{|c| [c.reqid, c.client] } temp :lookup <= (kvget_response * checkout_msg).pairs(:key => :client) end end

slide-43
SLIDE 43

Shopping cart

  • Abstract cart protocol

(messages exchanged between client & server)

  • Concrete implementation of cart server
  • Abstract key/value store
  • Concrete key/value store
slide-44
SLIDE 44

Concrete KVS

module BasicKVS include KVSProtocol end

slide-45
SLIDE 45

Concrete KVS

module BasicKVS include KVSProtocol state do table :kvstate, [:key] => [:value] end end

slide-46
SLIDE 46

Concrete KVS

module BasicKVS include KVSProtocol state do table :kvstate, [:key] => [:value] end bloom :mutate do kvstate <+ kvput {|s| [s.key, s.value]} kvstate <- (kvstate * kvput).lefts(:key => :key) end end

slide-47
SLIDE 47

Concrete KVS

module BasicKVS include KVSProtocol state do table :kvstate, [:key] => [:value] end bloom :mutate do kvstate <+ kvput {|s| [s.key, s.value]} kvstate <- (kvstate * kvput).lefts(:key => :key) end bloom :get do temp :getj <= (kvget * kvstate).pairs(:key => :key) kvget_response <= getj { |g, t| [g.reqid, t.key, t.value] } end end

slide-48
SLIDE 48

Ad-hoc genius analysis

slide-49
SLIDE 49

action_msg client reqid item action “Alice” 1 “2TB HD” “add” “Alice” 2 “2TB HD” “del” “Alice” 3 “128GB SSD” “add” checkout_msg client reqid “Alice” 4

Ad-hoc genius analysis

slide-50
SLIDE 50

Execution 1

slide-51
SLIDE 51

action_msg client reqid item action “Alice” 1 “2TB HD” “add” Execution 1

slide-52
SLIDE 52

checkout_msg client reqid “Alice” 4 action_msg client reqid item action “Alice” 1 “2TB HD” “add” Execution 1

slide-53
SLIDE 53

checkout_msg client reqid “Alice” 4 action_msg client reqid item action “Alice” 1 “2TB HD” “add” Execution 1 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 3 “128GB SSD” “add”

slide-54
SLIDE 54

checkout_msg client reqid “Alice” 4 action_msg client reqid item action “Alice” 1 “2TB HD” “add” Execution 1 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 3 “128GB SSD” “add” response_msg client item cnt “Alice” “2TB HD” 1

slide-55
SLIDE 55

checkout_msg client reqid “Alice” 4 action_msg client reqid item action “Alice” 1 “2TB HD” “add” Execution 1 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 3 “128GB SSD” “add” Execution 2 response_msg client item cnt “Alice” “2TB HD” 1

slide-56
SLIDE 56

checkout_msg client reqid “Alice” 4 action_msg client reqid item action “Alice” 1 “2TB HD” “add” Execution 1 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 3 “128GB SSD” “add” Execution 2 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 1 “2TB HD” “add” “Alice” 3 “128GB SSD” “add” response_msg client item cnt “Alice” “2TB HD” 1

slide-57
SLIDE 57

checkout_msg client reqid “Alice” 4 action_msg client reqid item action “Alice” 1 “2TB HD” “add” Execution 1 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 3 “128GB SSD” “add” checkout_msg client reqid “Alice” 4 Execution 2 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 1 “2TB HD” “add” “Alice” 3 “128GB SSD” “add” response_msg client item cnt “Alice” “2TB HD” 1

slide-58
SLIDE 58

checkout_msg client reqid “Alice” 4 action_msg client reqid item action “Alice” 1 “2TB HD” “add” Execution 1 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 3 “128GB SSD” “add” checkout_msg client reqid “Alice” 4 Execution 2 action_msg client reqid item action “Alice” 2 “2TB HD” “del” “Alice” 1 “2TB HD” “add” “Alice” 3 “128GB SSD” “add” response_msg client item cnt “Alice” “2TB HD” 1 “Alice” “128GB SSD” 1 response_msg client item cnt “Alice” “2TB HD” 1

slide-59
SLIDE 59

Ad-hoc genius analysis

  • What went wrong?
  • Delete before add?
  • checkout_msg in the middle of action_msg?
slide-60
SLIDE 60

Static analysis

table Default persist scratch Default delete channel Remote scratch <= Derive <+ Insert <- Delete <~ Send map filter join (*) not include group ... collection

  • p

collection expr

  • Never confluent
  • Confluent if guarded by table
  • Always confluent
slide-61
SLIDE 61

Static analysis

table Default persist scratch Default delete channel Remote scratch <= Derive <+ Insert <- Delete <~ Send map filter join (*) not include group ... collection

  • p

collection expr

  • Tables are inflationary
  • map, filter, join are homomorphisms
  • <=, <+ are commutative, associative
slide-62
SLIDE 62

Abstract cart protocol, revisited

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] channel :checkout_msg, [:@server, :client, :reqid] channel :response_msg, [:@client, :server, :item] => [:cnt] end end

slide-63
SLIDE 63

Abstract KVS, revisited

module KVSProtocol state do interface input, :kvput, [:key] => [:value] interface input, :kvget, [:reqid] => [:key] interface output, :kvget_response, [:reqid] => [:key, :value] end end

slide-64
SLIDE 64

KVS, revisited

module BasicKVS include KVSProtocol state do table :kvstate, [:key] => [:value] end bloom :mutate do kvstate <+ kvput {|s| [s.key, s.value]} kvstate <- (kvstate * kvput).lefts(:key => :key) end bloom :get do temp :getj <= (kvget * kvstate).pairs(:key => :key) kvget_response <= getj do |g, t| [g.reqid, t.key, t.value] end end end

slide-65
SLIDE 65

Cart, revisited

[…] kvput <= action_msg do |a| if a.action=="Add" and not kvget_response{|b| b.key}.include? a.client [a.client, Array.new.push(a.item)] end end […]

slide-66
SLIDE 66

Static analysis

slide-67
SLIDE 67

Solutions?

  • Total ordering on request ID?
  • What if clients can modify cart from multiple locations?
  • Inconsistent w/ “disorderly” philosophy
slide-68
SLIDE 68

Solutions?

  • Total ordering on request ID?
  • What if clients can modify cart from multiple locations?
  • Inconsistent w/ “disorderly” philosophy
  • “Disorderly” cart
  • Don't use ordering
  • Shopping cart kind of like a set (except it's a multiset)
slide-69
SLIDE 69

Disorderly cart

module DisorderlyCart include CartProtocol end

slide-70
SLIDE 70

Disorderly cart

module DisorderlyCart include CartProtocol state do table :cart_action, [:client, :reqid] => [:item, :action] scratch :action_cnt, [:client, :item, :action] => [:cnt] end end

slide-71
SLIDE 71

Disorderly cart

module DisorderlyCart include CartProtocol state do table :cart_action, [:client, :reqid] => [:item, :action] scratch :action_cnt, [:client, :item, :action] => [:cnt] end bloom :saved do end end

slide-72
SLIDE 72

Disorderly cart

module DisorderlyCart include CartProtocol state do table :cart_action, [:client, :reqid] => [:item, :action] scratch :action_cnt, [:client, :item, :action] => [:cnt] end bloom :saved do cart_action <= action_msg {|c| [c.client, c.reqid, c.item, c.action]} end end

slide-73
SLIDE 73

Disorderly cart

module DisorderlyCart include CartProtocol state do table :cart_action, [:client, :reqid] => [:item, :action] scratch :action_cnt, [:client, :item, :action] => [:cnt] end bloom :saved do cart_action <= action_msg {|c| [c.client, c.reqid, c.item, c.action]} temp :checkout_acts <= (checkout_msg * cart_action).rights action_cnt <= checkout_acts.group([cart_action.client, cart_action.item, cart_action.action], count(cart_action.reqid)) end end

slide-74
SLIDE 74

Disorderly cart

module DisorderlyCart include CartProtocol state do table :cart_action, [:client, :reqid] => [:item, :action] scratch :action_cnt, [:client, :item, :action] => [:cnt] end bloom :saved do cart_action <= action_msg {|c| [c.client, c.reqid, c.item, c.action]} temp :checkout_acts <= (checkout_msg * cart_action).rights action_cnt <= checkout_acts.group([cart_action.client, cart_action.item, cart_action.action], count(cart_action.reqid)) response_msg <~ (checkout_msg * action_cnt).rights(:client => :client) end end

slide-75
SLIDE 75

Static Analysis

slide-76
SLIDE 76

Cart, revisited

module DisorderlyCart include CartProtocol state do table :cart_action, [:client, :reqid] => [:item, :action] scratch :action_cnt, [:client, :item, :action] => [:cnt] end bloom :saved do cart_action <= action_msg {|c| [c.client, c.reqid, c.item, c.action]} temp :checkout_acts <= (checkout_msg * cart_action).rights action_cnt <= checkout_acts.group([cart_action.client, cart_action.item, cart_action.action], count(cart_action.reqid)) response_msg <~ (checkout_msg * action_cnt).rights(:client => :client) end end

slide-77
SLIDE 77

Abstract cart protocol, revisited

module CartProtocol state do channel :action_msg, [:@server, :client, :reqid] => [:item, :action] channel :checkout_msg, [:@server, :client, :reqid] channel :response_msg, [:@client, :server, :item] => [:cnt] end end

slide-78
SLIDE 78

Static Analysis

slide-79
SLIDE 79

Static Analysis

slide-80
SLIDE 80

Current Work

  • Adding more commutative data structures and

functions

  • Minimizing execution traces for debugging
  • Fault tolerance support
  • Optimization of Bloom programs
slide-81
SLIDE 81

Other cool stuff

  • REBL: Bloom REPL
  • Deployment
  • Local thread/process
  • EC2
  • TokyoCabinet, Zookeeper collections
  • Visualizer for debugging
  • BUST: REST client & server interfaces
  • Lots of examples and documentation!
slide-82
SLIDE 82

Check it out!

Bud: “Bloom under development” gem install bud http://github.org/bloom-lang

slide-83
SLIDE 83

Bloom: Using disorderly programming to build eventually- consistent distributed systems

Bill Marczak UC Berkeley