Proving Weakly Consistent Applications Correct Alexey Gotsman - - PowerPoint PPT Presentation

▶
proving weakly consistent applications correct
SMART_READER_LITE
LIVE PREVIEW

Proving Weakly Consistent Applications Correct Alexey Gotsman - - PowerPoint PPT Presentation

Oct 10, 2023 446 likes •972 views

Proving Weakly Consistent Applications Correct Alexey Gotsman IMDEA Software Institute, Madrid, Spain Joint work with Hongseok Yang (Oxford), Carla Ferreira (U Nova Lisboa), Mahsa Najafzadeh, Marc Shapiro (INRIA) Eventually consistent

slide-1
SLIDE 1

Proving Weakly Consistent Applications Correct

Joint work with Hongseok Yang (Oxford), Carla Ferreira (U Nova Lisboa), Mahsa Najafzadeh, Marc Shapiro (INRIA) Alexey Gotsman IMDEA Software Institute, Madrid, Spain

slide-2
SLIDE 2

Eventually consistent databases

  • No synchronisation: process an update locally,

propagate effects to other replicas later

  • Weakens consistency: deposit seen with a

delay

deposit(100)

slide-3
SLIDE 3

balance = 100 balance = 100

balance ≥ 0

slide-4
SLIDE 4

balance = 100 withdraw(100) : ✔ balance = 100 withdraw(100) : ✔ balance = 0

balance ≥ 0

balance = 0

slide-5
SLIDE 5

balance = 100 withdraw(100) : ✔ balance = 100 withdraw(100) : ✔ balance = 0 balance = -100

balance ≥ 0

balance = 0

slide-6
SLIDE 6

balance = 100 withdraw(100) : ✔ balance = 100 withdraw(100) : ✔ balance = 0 balance = -100

balance ≥ 0

balance = 0

deposit(100)

slide-7
SLIDE 7

balance = 100 withdraw(100) : ✔ balance = 100 withdraw(100) : ✔ balance = 0 balance = -100

balance ≥ 0

balance = 0

deposit(100)

  • Withdrawals strongly consistent
  • Deposits eventually consistent

Tune consistency:

slide-8
SLIDE 8

Consistency choices

  • Databases with multiple consistency levels:
  • Commercial: Amazon DynamoDB, Basho Riak,

Microsoft DocumentDB

  • Research: Li+ OSDI’12; Terry+ SOSP’13;

Balegas+ EuroSys’15...

  • Pay for stronger semantics with latency,

possible unavailability and money

  • Hard to figure out the minimum consistency

necessary to maintain correctness - proof rule and tool

slide-9
SLIDE 9

Consistency model

  • Generic model - not implemented, but can

encode many existing models that are: RedBlue consistency [Li+ 2012], reservation locks [Balegas+ 2015], parallel snapshot isolation [Sovran+ 2011], ...

  • Causal consistency as a baseline: observe an

update ➜ observe the updates it depends on

  • A construct for strengthening consistency on

demand

slide-10
SLIDE 10

σ ⟦op⟧val Replica states: σ ∈ State

  • p

Return value: ⟦op⟧val ∈ State ➞ Value

Operation semantics

slide-11
SLIDE 11

σ σʹ ⟦op⟧eff(σ)(σʹ) ⟦op⟧eff(σ) ⟦op⟧val Effector: ⟦op⟧eff ∈ State ➞ (State ➞ State) Replica states: σ ∈ State

  • p

Return value: ⟦op⟧val ∈ State ➞ Value

Operation semantics

slide-12
SLIDE 12

σ σʹ ⟦op⟧eff(σ)(σʹ) ⟦op⟧eff(σ) ⟦op⟧val Effector: ⟦op⟧eff ∈ State ➞ (State ➞ State) Replica states: σ ∈ State

  • p

Return value: ⟦op⟧val ∈ State ➞ Value

Operation semantics

slide-13
SLIDE 13

σ σʹ ⟦op⟧eff(σ)(σʹ) ⟦op⟧eff(σ) ⟦op⟧val Effector: ⟦op⟧eff ∈ State ➞ (State ➞ State) Replica states: σ ∈ State

  • p

Return value: ⟦op⟧val ∈ State ➞ Value

Operation semantics

Effector

slide-14
SLIDE 14

σ σʹ ⟦op⟧eff(σ)(σʹ) ⟦op⟧eff(σ) ⟦op⟧val Effector: ⟦op⟧eff ∈ State ➞ (State ➞ State) Replica states: σ ∈ State

  • p

Return value: ⟦op⟧val ∈ State ➞ Value

Operation semantics

Effector

slide-15
SLIDE 15

σ σʹ ⟦op⟧eff(σ)(σʹ) ⟦op⟧eff(σ) ⟦op⟧val ⟦balance()⟧eff(σ) = λσ. σ State = Z

  • p

⟦balance()⟧val(σ) = σ

Operation semantics

slide-16
SLIDE 16

σ σʹ ⟦op⟧eff(σ)(σʹ) ⟦op⟧eff(σ) ⟦op⟧val ⟦deposit(100)⟧eff(σ) = λσʹ. (σʹ + 100)

  • p

Operation semantics

slide-17
SLIDE 17

σ 50 ⟦op⟧eff(σ)(σʹ) ⟦op⟧eff(σ) ⟦op⟧val ⟦deposit(100)⟧eff(σ) = λσʹ. (σʹ + 100)

  • p

Operation semantics

slide-18
SLIDE 18

σ 50 150 ⟦op⟧eff(σ) ⟦op⟧val ⟦deposit(100)⟧eff(σ) = λσʹ. (σʹ + 100)

  • p

Operation semantics

slide-19
SLIDE 19
  • Effectors have to commute
  • Eventual consistency: replicas receiving the

same messages in different orders end up in the same state

  • Replicated data types [Shapiro+ 2011]:

ready-made commutative implementations

Ensuring eventual consistency

⟦deposit(100)⟧eff(σ) = λσʹ. (σʹ + 100)

slide-20
SLIDE 20

σ ⟦op⟧eff(σ) ⟦op⟧val

  • p

Operation semantics

if σ ≥ 100 then (λσʹ. σʹ - 100) else (λσʹ. σʹ) ⟦withdraw(100)⟧eff(σ) = σʹ ⟦op⟧eff(σ)(σʹ)

slide-21
SLIDE 21

σ ⟦op⟧eff(σ) ⟦op⟧val

  • p

Operation semantics

if σ ≥ 100 then (λσʹ. σʹ - 100) else (λσʹ. σʹ) ⟦withdraw(100)⟧eff(σ) = σʹ ⟦op⟧eff(σ)(σʹ)

slide-22
SLIDE 22

σ ⟦op⟧eff(σ) ⟦op⟧val

  • p

Operation semantics

if σ ≥ 100 then (λσʹ. σʹ - 100) else (λσʹ. σʹ) ⟦withdraw(100)⟧eff(σ) = σʹ ⟦op⟧eff(σ)(σʹ)

slide-23
SLIDE 23

σ ⟦op⟧eff(σ) ⟦op⟧val

  • p

Operation semantics

if σ ≥ 100 then (λσʹ. σʹ - 100) else (λσʹ. σʹ) ⟦withdraw(100)⟧eff(σ) = σʹ ⟦op⟧eff(σ)(σʹ)

slide-24
SLIDE 24

balance = 100 withdraw(100) : ✔ balance = 100 withdraw(100) : ✔ balance = 0 balance = 0

λσʹ. σʹ - 100

if σ ≥ 100 then (λσʹ. σʹ - 100) else (λσʹ. σʹ) ⟦withdraw(100)⟧eff(σ) =

slide-25
SLIDE 25

balance = 100 withdraw(100) : ✔ balance = 100 withdraw(100) : ✔ balance = 0 balance = -100 balance = 0

λσʹ. σʹ - 100

if σ ≥ 100 then (λσʹ. σʹ - 100) else (λσʹ. σʹ) ⟦withdraw(100)⟧eff(σ) =

slide-26
SLIDE 26

Strengthening consistency

  • Token = {τ1, τ2, ...}
  • Symmetric conflict relation ⋈ ⊆ Token × Token

Token system ≈ locks on steroids:

slide-27
SLIDE 27

Strengthening consistency

  • Token = {τ1, τ2, ...}
  • Symmetric conflict relation ⋈ ⊆ Token × Token

Token system ≈ locks on steroids: Example - mutual exclusion lock: Token = {τ}; τ ⋈ τ

slide-28
SLIDE 28

Strengthening consistency

  • Token = {τ1, τ2, ...}
  • Symmetric conflict relation ⋈ ⊆ Token × Token

Token system ≈ locks on steroids: Example - mutual exclusion lock: Token = {τ}; τ ⋈ τ

Each operation associated with a set of tokens: ⟦op⟧tok ∈ State ➞ P(Token)

slide-29
SLIDE 29

balance = 100 withdraw(100) : ✔ balance = 100

{τ} τ ⋈ τ

Operations associated with conflicting tokens cannot be unaware of each other

slide-30
SLIDE 30

balance = 100 withdraw(100) : ✔ balance = 100 Anything I don’t know about? withdraw(100) : ?

{τ} τ ⋈ τ {τ}

Operations associated with conflicting tokens cannot be unaware of each other

slide-31
SLIDE 31

balance = 100 balance = 100 withdraw(100) : ✔ balance = 0 withdraw(100) : ?

τ ⋈ τ {τ} {τ}

Operations associated with conflicting tokens cannot be unaware of each other

slide-32
SLIDE 32

balance = 100 withdraw(100) : ✘ balance = 0 balance = 100 withdraw(100) : ✔

τ ⋈ τ {τ} {τ}

Operations associated with conflicting tokens cannot be unaware of each other

slide-33
SLIDE 33

balance = 100 withdraw(100) : ✘ balance = 0 deposit(100)

∅

balance = 100 withdraw(100) : ✔

τ ⋈ τ

No synchronisation

{τ} {τ}

Operations associated with conflicting tokens cannot be unaware of each other

slide-34
SLIDE 34

balance = 100 withdraw(100) : ✘ balance = 0 deposit(100)

∅

balance = 100 withdraw(100) : ✔

τ ⋈ τ

No synchronisation

{τ} {τ}

Operations associated with conflicting tokens cannot be unaware of each other

Do we always have I = (balance ≥ 0)?

slide-35
SLIDE 35

σʹ ⟦op⟧eff(σ)(σʹ) ∈ I ? ⟦

  • p

⟧

e f f

( σ ) σ ∈ I

  • p

Effect applied in a different state!

slide-36
SLIDE 36

σʹ ⟦op⟧eff(σ)(σʹ) ∈ I ? ⟦

  • p

⟧

e f f

( σ ) σ ∈ I

  • p

if σ ≥ 100 then (λσʹ. σʹ - 100) else (λσʹ. σʹ) ⟦withdraw(100)⟧eff(σ) = ⟦op⟧eff(σ) = if P(σ) then f(σ) else if...

slide-37
SLIDE 37

σʹ ⟦op⟧eff(σ)(σʹ) ∈ I ? ⟦

  • p

⟧

e f f

( σ ) σ ∈ I

  • p

⟦op⟧eff(σ) = if P(σ) then f(σ) else if...

  • 1. Effector safety: f preserves I when executed in

any state satisfying P

slide-38
SLIDE 38

σʹ ⟦

  • p

⟧

e f f

( σ ) σ ∈ I

  • p

⟦op⟧eff(σ) = if P(σ) then f(σ) else if... ⟦op⟧eff(σ)(σʹ) ∈ I ✔

  • 1. Effector safety: f preserves I when executed in

any state satisfying P

slide-39
SLIDE 39

σʹ ⟦

  • p

⟧

e f f

( σ ) σ ∈ I

  • p

⟦op⟧eff(σ) = if P(σ) then f(σ) else if... ⟦op⟧eff(σ)(σʹ) ∈ I ✔ P(σʹ)?

  • 1. Effector safety: f preserves I when executed in

any state satisfying P

slide-40
SLIDE 40

σʹ ⟦

  • p

⟧

e f f

( σ ) σ ∈ I

  • p

⟦op⟧eff(σ) = if P(σ) then f(σ) else if... ⟦op⟧eff(σ)(σʹ) ∈ I ✔ P(σʹ)?

  • 2. Precondition stability: P will hold when f is

applied at any replica

  • 1. Effector safety: f preserves I when executed in

any state satisfying P

slide-41
SLIDE 41

⟦op⟧eff(σ) = if P(σ) then f(σ) else if...

  • 2. Precondition stability: P will hold when f is

applied at any replica

  • 1. Effector safety: f preserves I when executed in

any state satisfying P

CISE tool: ‘Cause I’m Strong Enough

By Mahsa Najafzadeh (UPMC & INRIA) Discharges proof obligations using Z3 SMT solver

slide-42
SLIDE 42
slide-43
SLIDE 43

31

slide-44
SLIDE 44

32

  • 1. Effector safety: f preserves I when executed in

any state satisfying P

slide-45
SLIDE 45

32

  • 1. Effector safety: f preserves I when executed in

any state satisfying P ✔

slide-46
SLIDE 46

33

  • 2. Precondition stability: P is preserved by

concurrent operations

slide-47
SLIDE 47

33

  • 2. Precondition stability: P is preserved by

concurrent operations

slide-48
SLIDE 48

33

  • 2. Precondition stability: P is preserved by

concurrent operations

Bug: concurrent withdrawals may violate the invariant

slide-49
SLIDE 49

34

  • 2. Precondition stability: P is preserved by

concurrent operations

Add a token restricting concurrency

slide-50
SLIDE 50

34

  • 2. Precondition stability: P is preserved by

concurrent operations ✔

Add a token restricting concurrency

slide-51
SLIDE 51

Conclusion

  • First proof rule and tool for proving

invariants of weakly consistent applications

  • Case studies: fragments of web applications,

replicated file system in progress

  • Future work: other consistency models,

automatic inference of consistency levels