The Push/Pull Model of Transactions Matthew Parkinson Eric - - PowerPoint PPT Presentation

▶
the push pull model of transactions
SMART_READER_LITE
LIVE PREVIEW

The Push/Pull Model of Transactions Matthew Parkinson Eric - - PowerPoint PPT Presentation

Aug 12, 2023 162 likes •641 views

The Push/Pull Model of Transactions Matthew Parkinson Eric Koskinen IBM Research, New York Microsoft Research, Cambridge United States United Kingdom PLDI 2015 Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm

slide-1
SLIDE 1

The Push/Pull Model

  • f Transactions

Eric Koskinen IBM Research, New York United States Matthew Parkinson Microsoft Research, Cambridge United Kingdom

PLDI 2015

slide-2
SLIDE 2

Concurrent HashMap Concurrent Queue Concurrent List

Thread 1 Thread 3 Thread 2 Thread 4

put get enq deq add rm

Shared Memory

rd wr

slide-3
SLIDE 3

Concurrent HashMap Concurrent Queue Concurrent List

Thread 1 Thread 3 Thread 2 Thread 4

put get enq deq add rm

Shared Memory

rd wr

Adding atomic to a language . . . Version 4.7

slide-4
SLIDE 4

Serializable. Opaque. …

Concurrent HashMap Concurrent Queue Concurrent List

Thread 1 Thread 3 Thread 2 Thread 4

put get enq deq add rm

Shared Memory

rd wr

Adding atomic to a language . . .

slide-5
SLIDE 5

Concurrent HashMap Concurrent Queue Concurrent List

Thread 1 Thread 3 Thread 2 Thread 4

put get enq deq add rm

Shared Memory

rd wr

many issues arise in the implementation.

Serializable. Opaque. …

? ? ?

Adding atomic to a language . . .

slide-6
SLIDE 6

P e s s i m i s m !

Thread

atomic { x := 3; y := 9; }

x changed? y changed?

Thread

atomic { x := 3; y := 9; }

all ok?

O p t i m i s m !

slide-7
SLIDE 7

S e r i a l i z a b i l i t y !

Interleaved execution equivalent to some serial execution.

Opacity!

Interleaved execution equivalent to some serial execution and cannot observe intermediate state.

P e s s i m i s m ! O p t i m i s m ! D e p e n d e n t T x n s

slide-8
SLIDE 8

ADT Ops (Boosting)

Thread

atomic { x := 3; y := 9; }

Thread

atomic { stk.push(4); ht.get('a'); }

S e r i a l i z a b i l i t y ! Opacity! P e s s i m i s m ! O p t i m i s m ! Memory Ops

slide-9
SLIDE 9

ADT Ops (Boosting) S e r i a l i z a b i l i t y ! Opacity! P e s s i m i s m ! O p t i m i s m ! Memory Ops Hardware TM Software TM Hybrid TM D e p e n d e n t T x n s

slide-10
SLIDE 10

ADT Ops (Boosting) S e r i a l i z a b i l i t y ! Opacity! P e s s i m i s m ! O p t i m i s m ! Memory Ops Hardware TM Software TM Hybrid TM Open Nested D e p e n d e n t T x n s Irrevocability Consistency Eager Writes L a z y R e a d s

slide-11
SLIDE 11

ADT Ops (Boosting) S e r i a l i z a b i l i t y ! Opacity! P e s s i m i s m ! O p t i m i s m ! Memory Ops Hardware TM Software TM Hybrid TM Open Nested D e p e n d e n t T x n s Irrevocability Consistency Eager Writes L a z y R e a d s

Serializable. Opaque. …

? ? ?

slide-12
SLIDE 12

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-13
SLIDE 13

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

Serializable. … Opaque.

slide-14
SLIDE 14

Serializable. … Opaque. Linearizable Commute

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-15
SLIDE 15

Thread

atomic { if(b) {

  • pA(…);

} else … }

Thread

atomic { while(…){ if(x>MAX)

  • pB(…);

… }

slide-16
SLIDE 16

Thread

atomic { if(b) {

  • pA(…);

} else … }

Thread

atomic { while(…){ if(x>MAX)

  • pB(…);

… }

Linearizable Operation Linearizable Operation

slide-17
SLIDE 17

Thread Thread

{tx c1, σ1} {tx c2, σ2}

  • 1. Abstract away the language semantics
slide-18
SLIDE 18

Thread Thread

{tx c1, σ1} {tx c2, σ2}

  • 1. Abstract away the language semantics

example language:

slide-19
SLIDE 19
  • 1. Abstract away the language semantics

Thread Thread

  • 2. Abstract away the state (with logs)

{tx c1, σ1, L1} {tx c2, σ2, L2} G ⟨op1,_⟩ ⟨op2,_⟩

slide-20
SLIDE 20
  • 1. Abstract away the language semantics

Thread Thread

  • 2. Abstract away the state (with logs)

{tx c1, σ1, L1} {tx c2, σ2, L2} G

Closed under log prefix

Parameterized by predicate allowed

⟨op1,_⟩ ⟨op2,_⟩

slide-21
SLIDE 21

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} G

Closed under log prefix

Parameterized by predicate allowed Log Equality

⟨op1,_⟩ ⟨op2,_⟩

slide-22
SLIDE 22

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} G

Log Equality

⟨op1,_⟩ ⟨op2,_⟩

7 Simple Rules: Apply Push Pull Unpull Unpush Unapply Commit

slide-23
SLIDE 23

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}

Two Logs

atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } G ⟨ht.map(3,x),σ,σ1,c⟩ ⟨ht.map(7,2),_,gUC⟩

slide-24
SLIDE 24

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Apply

G

1

⟨ht.map(3,x),σ,σ1,c⟩ ⟨q.enq(‘a’),σ1,σ'1,c1⟩ ⟨ht.map(7,2),_,gUC⟩

1

slide-25
SLIDE 25

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Apply

G Criterion (i): Criterion (ii): Criterion (iii): L1 allows ⟨m,σ1,σ'1,id⟩

fresh(id)

APPLY

1

⟨ht.map(3,x),σ,σ1,c⟩ ⟨q.enq(‘a’),σ1,σ'1,c1⟩ ⟨ht.map(7,2),_,gUC⟩

1

{tx c1, σ1, L1}, G ➝ {tx c'1, σ'1, L1·[op,σ1,σ'1,c1]}, G

append Possible next op Valid semantics of log

slide-26
SLIDE 26

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Unapply

G

UNAPPLY

⟨ht.map(7,2),_,gUC⟩ ⟨ht.map(3,x),σ,σ1,c⟩ ⟨q.enq(‘a’),σ1,σ'1,c1⟩ {tx c'1, σ'1, L1·[op,σ1,σ'1,c1]}, G
 ➝ {tx c1, σ1, L1}, G

slide-27
SLIDE 27

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}

Two Logs

atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ G ⟨ht.map(7,2),_,gUC⟩

slide-28
SLIDE 28

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}

Two Logs

atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.map(3,x),_⟩ Push ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ G ⟨ht.map(7,2),_,gUC⟩

slide-29
SLIDE 29

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}

Two Logs

atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Push

G

PUSH

Criterion (i): Criterion (ii): Criterion (iii):

  • p ◀ ⎣`L⎦unpushed

G allows op

Act as if op happens next No conflict w/ other uncmted

Left-mover over logs ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ {tx c, σ, `L·[op]·L'}, G ➝ {tx c, σ, `L·[op]·L'}, G·[op,gUC] ⎣G⎦gUC ∖ ⎣L1⎦pushed ◀ op

Uncommitted

Application: Out-of-order PUSHing with redo-logs Application: Optimism vs. Pessimism

slide-30
SLIDE 30

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.map(3,x),_⟩

Pull

G

PULL

L2 allows op Criterion (i): Criterion (ii):

  • p ◀ ⎣L2⎦pushed ∪ ⎣L2⎦unpushed

Criterion (iii):

  • p ∉ L2

Didn’t pull already Local log allows op Can act as if op happened earlier

⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ {tx c2, σ2, L2}, G1·[op,g]·G2 ➝ {tx c2, σ2, L2·[op]}, G1·[op,g]·G2

Application: Opacity [GK’08] and dependent transactions [RRHW’09]

⟨q.enq(‘a’),_,gUC⟩

Push

slide-31
SLIDE 31

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Apply

⟨ht.get(5),_⟩ G ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-32
SLIDE 32

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… }

Push

⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩

Pull

G ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-33
SLIDE 33

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ G ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

Pull

⟨q.enq(‘a’),_⟩

slide-34
SLIDE 34

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ G ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.get(9),_⟩

Commit

slide-35
SLIDE 35

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ G ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.get(9),_⟩

Commit COMMIT

cmt(G, L1, G')

⎣L1⎦pulled ⊆ ⎣G⎦gC Criterion (ii): Criterion (iii): Criterion (iv): L1 ⊆ G Criterion (i):

fin(c1)

Pushed all my stuff Pulled ops are committed Swap my flags from gUC to gC

{tx c1, σ1, L1}, G ➝ {tx c1, σ1, L1}, G’

slide-36
SLIDE 36

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ ⟨

Unapply

G

slide-37
SLIDE 37

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩

Unpull

G

UNPULL

(Local log is still allowed, even with element removed.)

slide-38
SLIDE 38

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ G

Unpush UNPUSH

(Everything I subsequently pushed could have been pushed earlier.)

Application: Inverses or “compensating actions”

slide-39
SLIDE 39

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ G

Unapply

q . d e q ( )

slide-40
SLIDE 40

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2} atomic {

  • p1;
  • p2;

… } atomic {

  • p3;
  • p4;

… } ⟨ht.get(5),_,gUC⟩ ⟨ht.get(5),_⟩ ⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩ ⟨q.enq(‘a’),_⟩ G

Unapply

q . d e q ( )

slide-41
SLIDE 41
  • Theorem. The Push/Pull model is serializable.

Simulation with uninterleaved machine Closure under log rewind Preservation invariant, universal quantification

  • ver dropping uncommitted global ops

universal quantifying over local log rewind.

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-42
SLIDE 42
  • Theorem. Fragments of The Push/Pull model are opaque.

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

example: Don’t PULL uncommitted effects.

slide-43
SLIDE 43

Other Applications.

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

  • Pessimistic models of STM.
  • Pessimistic models of STM/Boosting.
  • Optimistic models of STM.
  • Optimistic models of STM/Boosting (TRANSACT’15)
  • Closed-nested transactions: don’t share until commit.
  • Dependent transactions: view uncommitted effects but

delay commit (or cascading aborts)

slide-44
SLIDE 44

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

Mixing hardware TM with transactional boosting Models that are yet-to-come.

slide-45
SLIDE 45

Serializable. … Opaque. Linearizable Commute

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

slide-46
SLIDE 46

The PUSH/PULL Model

Push

⟨ht.get(5),_⟩

Pull

⟨ht.map(3,x),_⟩ ⟨ht.map(3,x),_⟩ ⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩ ⟨ht.map(3,x),_,gUC⟩ ⟨ht.map(7,2),_,gUC⟩

Eric Koskinen IBM Research, New York United States Matthew Parkinson Microsoft Research, Cambridge United Kingdom

Thank you!