` James R. Wilcox Zach Tatlock Ilya Sergey Distributed Systems - - PowerPoint PPT Presentation

▶
james r wilcox zach tatlock ilya sergey distributed
SMART_READER_LITE
LIVE PREVIEW

` James R. Wilcox Zach Tatlock Ilya Sergey Distributed Systems - - PowerPoint PPT Presentation

Mar 26, 2024 143 likes •797 views

Programming Language Abstractions for Modularly Verified Distributed Systems { P } c { Q } ` James R. Wilcox Zach Tatlock Ilya Sergey Distributed Systems Distributed Infrastructure Distributed Applications Verified Distributed Systems

slide-1
SLIDE 1

James R. Wilcox Zach Tatlock Ilya Sergey

Programming Language Abstractions for Modularly Verified Distributed Systems

{P} c {Q}

`

slide-2
SLIDE 2

Distributed Systems

slide-3
SLIDE 3

Distributed Infrastructure

slide-4
SLIDE 4

Distributed Applications

slide-5
SLIDE 5

Verified Distributed Systems

slide-6
SLIDE 6

Verified Distributed Systems

slide-7
SLIDE 7

Verified Distributed Infrastructure

slide-8
SLIDE 8

Verified Distributed Infrastructure

  • Cert

Veri- Iron Wow

slide-9
SLIDE 9

Verified Distributed Applications

slide-10
SLIDE 10

Verified Distributed Applications

  • Cert

Veri- Iron Wow

slide-11
SLIDE 11

Verified Distributed Applications

  • Cert

Veri- Iron Wow

Challenging to verify apps in terms of infra. verify clients by starting over! Indicates deeper problems with composition

  • ne node’s client is another’s server!
slide-12
SLIDE 12

Verified Distributed Applications

  • Cert

Veri- Iron Wow

Challenging to verify apps in terms of infra. verify clients by starting over! Indicates deeper problems with composition

  • ne node’s client is another’s server!

(Make it possible to) verify clients verify clients without starting over! Will also enable more general composition

slide-13
SLIDE 13

Composition: A way to make proofs harder

slide-14
SLIDE 14

Composition: A way to make proofs harder

When distracting language issues are removed and the underlying mathematics is revealed, compositional reasoning is seen to be of little use.

slide-15
SLIDE 15

Approach

`{P} c {Q}

Distributed Hoare Type Theory

slide-16
SLIDE 16

Distributed Interactions

Servers and Clients Optimizations Combining Services Horizons

gcc -O3

slide-17
SLIDE 17

Cloud Compute

S C

21

slide-18
SLIDE 18

Cloud Compute

slide-19
SLIDE 19

Cloud Compute

slide-20
SLIDE 20

Cloud Compute

while True: (from, n) <- recv send (n, factors(n)) to from

: Server

slide-21
SLIDE 21

Cloud Compute

while True: (from, n) <- recv send (n, factors(n)) to from

: Server

Traditional specification: messages from server have correct factors Proved by finding an invariant of the system

slide-22
SLIDE 22

Cloud Compute: Server

slide-23
SLIDE 23

Cloud Compute: Client

slide-24
SLIDE 24

Cloud Compute: Client

send 21 to server (_, ans) <- recv assert ans == {3, 7}

slide-25
SLIDE 25

Cloud Compute: Client

send 21 to server (_, ans) <- recv assert ans == {3, 7}

Expand system to include clients Need to reason about client-server interaction introduce protocol

slide-26
SLIDE 26

Protocols

slide-27
SLIDE 27

Protocols

slide-28
SLIDE 28

Protocols

Protocols make it possible to verify clients!

slide-29
SLIDE 29

Protocols

slide-30
SLIDE 30

Protocols

State: abstract state of each node

slide-31
SLIDE 31

Protocols

State: Transitions: abstract state of each node allowed sends and receives

slide-32
SLIDE 32

Cloud Compute Protocol

State: Transitions:

slide-33
SLIDE 33

Cloud Compute Protocol

State: Transitions:

permissions: Set<Msg>

slide-34
SLIDE 34

Cloud Compute Protocol

Send Req Recv Req Send Resp Recv Resp

State: Transitions:

permissions: Set<Msg>

slide-35
SLIDE 35

Cloud Compute: Protocol

Send Req Recv Req Send Resp Recv Resp

State: Transitions:

perm: Set<Msg>

Effect: add (from, n) to perm

Recv Request n

slide-36
SLIDE 36

Cloud Compute: Protocol

Send Req Recv Req Send Resp Recv Resp

State: Transitions:

perm: Set<Msg>

Effect: removes (n,to) from perm

Send Response (n,l)

Requires: l == factors(n) (n,to) in perm

slide-37
SLIDE 37

Cloud Compute: Protocol

Send Req Recv Req Send Resp Recv Resp

State: Transitions:

perm: Set<Msg>

Recv Response l

Ensures: l == factors(n) (n,to) in perm

slide-38
SLIDE 38

Cloud Compute: Protocol

Send Req Recv Req Send Resp Recv Resp

State: Transitions:

permissions: Set<Msg>

slide-39
SLIDE 39

Cloud Compute: Protocol

Send Req Recv Req Send Resp Recv Resp

State: Transitions:

permissions: Set<Msg>

Protocols make it possible to verify clients!

slide-40
SLIDE 40

From Protocols to Types

`{P} c {Q}

slide-41
SLIDE 41

From Protocols to Types

`{P}

{

}

t

sent ( )

m h

,

send m to h

slide-42
SLIDE 42

From Protocols to Types

`

send m to h

t

{P}

{

}

t

sent ( )

m h

,

slide-43
SLIDE 43

From Protocols to Types

`

send m to h

t t ∈

{P}

{

}

t

sent ( )

m h

,

slide-44
SLIDE 44

From Protocols to Types

`

send m to h

t t ∈

{P}

{

}

t

sent ( )

m h

,

P ⇒ Pre t

slide-45
SLIDE 45

From Protocols to Types

`

send m to h

t t ∈

{P}

{

}

t

sent ( )

m h

,

P ⇒ Pre t

slide-46
SLIDE 46

Cloud Compute: Client

send 21 to server (_, ans) <- recv assert ans == {3, 7}

slide-47
SLIDE 47

Cloud Compute: Client

send 21 to server (_, ans) <- recv assert ans == {3, 7}

recv ensures correct factors

slide-48
SLIDE 48

Cloud Compute: More Clients

send 21 to server1 send 35 to server2 (_, ans1) <- recv (_, ans2) <- recv assert ans1 ans2 == {3, 5, 7}

∪

slide-49
SLIDE 49

Cloud Compute: More Clients

send 21 to server1 send 35 to server2 (_, ans1) <- recv (_, ans2) <- recv assert ans1 ans2 == {3, 5, 7}

∪

Same protocol enables verification

slide-50
SLIDE 50

Cloud Compute: More Clients

send 21 to server1 send 35 to server2 (_, ans1) <- recv (_, ans2) <- recv assert ans1 ans2 == {3, 5, 7}

∪

Same protocol enables verification

slide-51
SLIDE 51

Cloud Compute

while True: (from, n) <- recv send (n, factors(n)) to from

: Server

slide-52
SLIDE 52

Cloud Compute

while True: (from, n) <- recv send (n, factors(n)) to from

: Server

Precondition on send requires correct factors

slide-53
SLIDE 53

Cloud Compute

cache = {}
 while True: (from, n) <- recv ans = if n cache then cache[n] else factors(n) cache[n] = ans send (n, ans) to from

: More Servers

∈

slide-54
SLIDE 54

Cloud Compute

cache = {}
 while True: (from, n) <- recv ans = if n cache then cache[n] else factors(n) cache[n] = ans send (n, ans) to from

: More Servers

∈

Still follows protocol!

slide-55
SLIDE 55

Cloud Compute

while True: (from, n) <- recv send n to backend (_, ans) <- recv send (n, ans) to from

: More Servers

slide-56
SLIDE 56

Cloud Compute

while True: (from, n) <- recv send n to backend (_, ans) <- recv send (n, ans) to from

: More Servers

Still follows protocol!

slide-57
SLIDE 57

Cloud Compute

while True: (from, n) <- recv send n to backend (_, ans) <- recv send (n, ans) to from

: More Servers

Still follows protocol! One node’s client is another’s server! Any combination of transitions follows protocol Well-typed programs don’t go wrong!

slide-58
SLIDE 58

Horizons

Adding other effects Sophisticated protocol composition

e.g. computation uses separate database e.g. mutable heap, threads, failure…

Fault tolerance

what do Verdi’s VSTs look like here?

slide-59
SLIDE 59

Verified Distributed Applications

slide-60
SLIDE 60

Verified Distributed Applications

  • Cert

Veri- Iron Wow

slide-61
SLIDE 61

Verified Distributed Applications

  • Cert

Veri- Iron Wow

Challenging to verify apps in terms of infra. verify clients by starting over! Indicates deeper problems with composition

  • ne node’s client is another’s server!
slide-62
SLIDE 62

Verified Distributed Applications

  • Cert

Veri- Iron Wow

Challenging to verify apps in terms of infra. verify clients by starting over! Indicates deeper problems with composition

  • ne node’s client is another’s server!

Protocols make it possible to verify clients reason about client-server interaction Also enable more general composition

Any combination of transitions follows protocol Well-typed programs don’t go wrong!

slide-63
SLIDE 63

Verified Distributed Applications

  • Cert

Veri- Iron Wow

Protocols make it possible to verify clients reason about client-server interaction Also enable more general composition

Any combination of transitions follows protocol Well-typed programs don’t go wrong!

slide-64
SLIDE 64

Verified Distributed Applications

  • Cert

Veri- Iron Wow

Composition is hard but important for infrastructure Achieve with types syntactic theory of composition

Protocols make it possible to verify clients reason about client-server interaction Also enable more general composition

Any combination of transitions follows protocol Well-typed programs don’t go wrong!