The Push/Pull Model of Transactions Matthew Parkinson Eric - - PowerPoint PPT Presentation

▶
the push pull model of transactions
SMART_READER_LITE
LIVE PREVIEW

The Push/Pull Model of Transactions Matthew Parkinson Eric - - PowerPoint PPT Presentation

Oct 28, 2022 486 likes •964 views

The Push/Pull Model of Transactions Matthew Parkinson Eric Koskinen Yale University, New Haven Microsoft Research, Cambridge United States United Kingdom CertiKOS Weekly Lunch 16 October 2015 Appeared in PLDI 2015 Thread Thread Thread

slide-1
SLIDE 1

The Push/Pull Model

  • f Transactions

Eric Koskinen Yale University, New Haven United States Matthew Parkinson Microsoft Research, Cambridge United Kingdom

Appeared in PLDI 2015

CertiKOS Weekly Lunch 16 October 2015

slide-2
SLIDE 2

Concurrent HashMap Concurrent Queue Concurrent List

Thread 1 Thread 3 Thread 2 Thread 4

put get enq deq add rm

Shared Memory

rd wr

slide-3
SLIDE 3

Concurrent HashMap Concurrent Queue Concurrent List

Thread 1 Thread 3 Thread 2 Thread 4

put get enq deq add rm

Shared Memory

rd wr

Adding atomic to a language . . . Version 4.7

slide-4
SLIDE 4

Serializable. Opaque. …

Concurrent HashMap Concurrent Queue Concurrent List

Thread 1 Thread 3 Thread 2 Thread 4

put get enq deq add rm

Shared Memory

rd wr

Adding atomic to a language . . .

slide-5
SLIDE 5

Concurrent HashMap Concurrent Queue Concurrent List

Thread 1 Thread 3 Thread 2 Thread 4

put get enq deq add rm

Shared Memory

rd wr

many issues arise in the implementation.

Serializable. Opaque. …

? ? ?

Adding atomic to a language . . .

slide-6
SLIDE 6

P e s s i m i s m !

Thread

atomic { x := 3; y := 9; }

x changed? y changed?

Thread

atomic { x := 3; y := 9; }

all ok?

O p t i m i s m !

slide-7
SLIDE 7

S e r i a l i z a b i l i t y !

Interleaved execution equivalent to some serial execution.

Opacity!

Interleaved execution equivalent to some serial execution and cannot observe intermediate state.

P e s s i m i s m ! O p t i m i s m ! D e p e n d e n t T x n s

slide-8
SLIDE 8

ADT Ops (Boosting)

Thread

atomic { x := 3; y := 9; }

Thread

atomic { stk.push(4); ht.get('a'); }

S e r i a l i z a b i l i t y ! Opacity! P e s s i m i s m ! O p t i m i s m ! Memory Ops

slide-9
SLIDE 9

ADT Ops (Boosting) S e r i a l i z a b i l i t y ! Opacity! P e s s i m i s m ! O p t i m i s m ! Memory Ops Hardware TM Software TM Hybrid TM D e p e n d e n t T x n s

slide-10
SLIDE 10

ADT Ops (Boosting) S e r i a l i z a b i l i t y ! Opacity! P e s s i m i s m ! O p t i m i s m ! Memory Ops Hardware TM Software TM Hybrid TM Open Nested D e p e n d e n t T x n s Irrevocability Consistency Eager Writes L a z y R e a d s

slide-11
SLIDE 11

ADT Ops (Boosting) S e r i a l i z a b i l i t y ! Opacity! P e s s i m i s m ! O p t i m i s m ! Memory Ops Hardware TM Software TM Hybrid TM Open Nested D e p e n d e n t T x n s Irrevocability Consistency Eager Writes L a z y R e a d s

Serializable. Opaque. …

? ? ?

slide-12
SLIDE 12

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-13
SLIDE 13

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

Serializable. … Opaque.

slide-14
SLIDE 14

Serializable. … Opaque. Linearizable Commute

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-15
SLIDE 15

Thread

atomic { if(b) {

  • pA(…);

} else … }

Thread

atomic { while(…){ if(x>MAX)

  • pB(…);

… }

slide-16
SLIDE 16

Thread

atomic { if(b) {

  • pA(…);

} else … }

Thread

atomic { while(…){ if(x>MAX)

  • pB(…);

… }

Linearizable Operation Linearizable Operation

slide-17
SLIDE 17

Thread Thread

{tx c1, σ1} {tx c2, σ2}

  • 1. Abstract away the language semantics
slide-18
SLIDE 18

Thread Thread

{tx c1, σ1} {tx c2, σ2}

  • 1. Abstract away the language semantics

example language:

slide-19
SLIDE 19
  • 1. Abstract away the language semantics

Thread Thread

  • 2. Abstract away the state (with logs)

{tx c1, σ1, L1} {tx c2, σ2, L2} G ⟨op1,_⟩ ⟨op2,_⟩

slide-20
SLIDE 20
  • 1. Abstract away the language semantics

Thread Thread

  • 2. Abstract away the state (with logs)

{tx c1, σ1, L1} {tx c2, σ2, L2} G

Closed under log prefix

Parameterized by predicate allowed

⟨op1,_⟩ ⟨op2,_⟩

slide-21
SLIDE 21

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} G

Closed under log prefix

Parameterized by predicate allowed Log Equality

⟨op1,_⟩ ⟨op2,_⟩

slide-22
SLIDE 22

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} G

Log Equality

⟨op1,_⟩ ⟨op2,_⟩

7 Simple Rules: Apply Push Pull Unpull Unpush Unapply Commit

slide-23
SLIDE 23

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}

Two Logs

atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } G ⟨ht.map(3,x),σ,σ1,c⟩ ⟨ht.map(7,2),_,gUC⟩

slide-24
SLIDE 24

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Apply

G

1

⟨ht.map(3,x),σ,σ1,c⟩ ⟨q.enq(‘a’),σ1,σ'1,c1⟩ ⟨ht.map(7,2),_,gUC⟩

1

slide-25
SLIDE 25

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Apply

G Criterion (i): Criterion (ii): Criterion (iii): L1 allows ⟨m,σ1,σ'1,id⟩

fresh(id)

APPLY

1

⟨ht.map(3,x),σ,σ1,c⟩ ⟨q.enq(‘a’),σ1,σ'1,c1⟩ ⟨ht.map(7,2),_,gUC⟩

1

{tx c1, σ1, L1}, G ➝ {tx c'1, σ'1, L1·[op,σ1,σ'1,c1]}, G

append Possible next op Valid semantics of log

slide-26
SLIDE 26

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Unapply

G

UNAPPLY

⟨ht.map(7,2),_,gUC⟩ ⟨ht.map(3,x),σ,σ1,c⟩ ⟨q.enq(‘a’),σ1,σ'1,c1⟩ {tx c'1, σ'1, L1·[op,σ1,σ'1,c1]}, G
 ➝ {tx c1, σ1, L1}, G

slide-27
SLIDE 27

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}

Two Logs

atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ G ⟨ht.map(7,2),_,gUC⟩

slide-28
SLIDE 28

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}

Two Logs

atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.map(3,x),_⟩ Push ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ G ⟨ht.map(7,2),_,gUC⟩

slide-29
SLIDE 29

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}

Two Logs

atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Push

G

PUSH

Criterion (i): Criterion (ii): Criterion (iii):

  • p ◀ ⎣`L⎦unpushed

G allows op

Act as if op happens next No conflict w/ other uncmted

Left-mover over logs ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ {tx c, σ, `L·[op]·L'}, G ➝ {tx c, σ, `L·[op]·L'}, G·[op,gUC] ⎣G⎦gUC ∖ ⎣L1⎦pushed ◀ op

Uncommitted

Application: Out-of-order PUSHing with redo-logs Application: Optimism vs. Pessimism

slide-30
SLIDE 30

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.map(3,x),_⟩

Pull

G

PULL

L2 allows op Criterion (i): Criterion (ii):

  • p ◀ ⎣L2⎦pushed ∪ ⎣L2⎦unpushed

Criterion (iii):

  • p ∉ L2

Didn’t pull already Local log allows op Can act as if op happened earlier

⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ {tx c2, σ2, L2}, G1·[op,g]·G2 ➝ {tx c2, σ2, L2·[op]}, G1·[op,g]·G2

Application: Opacity [GK’08] and dependent transactions [RRHW’09]

⟨q.enq(‘a’),_,gUC⟩

Push

slide-31
SLIDE 31

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Apply

⟨ht.get(5),_⟩ G ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-32
SLIDE 32

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Push

⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩

Pull

G ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-33
SLIDE 33

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ G ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

Pull

⟨q.enq(‘a’),_⟩

slide-34
SLIDE 34

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ G ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.get(9),_⟩

Commit

slide-35
SLIDE 35

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ G ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.get(9),_⟩

Commit COMMIT

cmt(G, L1, G')

⎣L1⎦pulled ⊆ ⎣G⎦gC Criterion (ii): Criterion (iii): Criterion (iv): L1 ⊆ G Criterion (i):

fin(c1)

Pushed all my stuff Pulled ops are committed Swap my flags from gUC to gC

{tx c1, σ1, L1}, G ➝ {tx c1, σ1, L1}, G’

slide-36
SLIDE 36

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ ⟨

Unapply

G

slide-37
SLIDE 37

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩

Unpull

G

UNPULL

(Local log is still allowed, even with element removed.)

slide-38
SLIDE 38

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ G

Unpush UNPUSH

(Everything I subsequently pushed could have been pushed earlier.)

Application: Inverses or “compensating actions”

slide-39
SLIDE 39

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ G

Unapply

q . d e q ( )

slide-40
SLIDE 40

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ G

Unapply

q . d e q ( )

slide-41
SLIDE 41
  • Theorem. The Push/Pull model is serializable.

Simulation with uninterleaved machine Closure under log rewind Preservation invariant, universal quantification

  • ver dropping uncommitted global ops

universal quantifying over local log rewind.

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-42
SLIDE 42
  • Theorem. Fragments of The Push/Pull model are opaque.

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

example: Don’t PULL uncommitted effects.

slide-43
SLIDE 43

Other Applications.

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

  • Pessimistic models of STM.
  • Pessimistic models of STM/Boosting.
  • Optimistic models of STM.
  • Optimistic models of STM/Boosting (TRANSACT’15)
  • Closed-nested transactions: don’t share until commit.
  • Dependent transactions: view uncommitted effects but

delay commit (or cascading aborts)

slide-44
SLIDE 44

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

Mixing hardware TM with transactional boosting Models that are yet-to-come.

slide-45
SLIDE 45

Serializable. … Opaque. Linearizable Commute

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-46
SLIDE 46

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

Eric Koskinen IBM Research, New York United States Matthew Parkinson Microsoft Research, Cambridge United Kingdom

Thank you!