The Push/Pull Model of Transactions Matthew Parkinson Eric - PowerPoint PPT Presentation

The Push/Pull Model of Transactions Matthew Parkinson Eric Koskinen IBM Research, New York Microsoft Research, Cambridge United States United Kingdom PLDI 2015

Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm Concurrent Concurrent Concurrent Shared HashMap Queue List Memory

Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm Concurrent Concurrent Concurrent Shared HashMap Queue List Memory Adding atomic to a language . . . Version 4.7

Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm Concurrent Concurrent Concurrent Shared HashMap Queue List Memory Adding atomic to a language . . . Serializable. Opaque. …

Thread Thread Thread Thread 1 2 3 4 rd wr put get enq deq add rm Concurrent Concurrent Concurrent Shared HashMap Queue List Memory Adding atomic to a language . . . many issues arise in the implementation. ? ? ? Serializable. Opaque. …

! m s i m i s s e P Thread atomic { x changed? x := 3; y changed? y := 9; } O p t i m i s m ! Thread atomic { x := 3; y := 9; } all ok?

! m s i m i s s e P Opacity! Interleaved execution equivalent to some serial execution and cannot observe intermediate state. ! y t i l i b a z i l a i r e S O p t i m i s m ! Interleaved execution s equivalent to some n x T serial execution. t n e d n e p e D

! m s i m i s s e P Opacity! ! y t i l i b a z i l a i r e S O p t i m i s m ! Memory Ops Thread Thread atomic { atomic { x := 3; stk.push(4); ADT Ops y := 9; ht.get('a'); } (Boosting) }

! m s i m i s s e P Opacity! Hardware TM Software TM Hybrid TM ! y t i l i b a z i l a i r e S O p t i m i s m ! Memory Ops s n x T t n e ADT Ops d n e p e (Boosting) D

Consistency ! m s i m i s s e P Opacity! Irrevocability Open Nested Hardware TM Software TM Hybrid TM ! y t i l i b a z L i l a a i z r e y S R e a O d s p t i m i s m ! Memory Ops s n x T t n e ADT Ops d Eager Writes n e p e (Boosting) D

Consistency ! m s i m i s s e P Opacity! Irrevocability Open Nested Hardware TM Software TM Hybrid TM ! y t i l i b a z L i l a a i z r e y S R e a O d s p t i m i s m ! Memory Ops s n x T ? ? ? t Serializable. Opaque. … n e ADT Ops d Eager Writes n e p e (Boosting) D

The P USH /P ULL Model ⟨ ht.map(7,2) ,_, gUC ⟩ Push Pull ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ q.enq(‘a’) ,_ ⟩

The P USH /P ULL Model ⟨ ht.map(7,2) ,_, gUC ⟩ Push Pull ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ q.enq(‘a’) ,_ ⟩ Serializable. Opaque. …

Linearizable Commute The P USH /P ULL Model ⟨ ht.map(7,2) ,_, gUC ⟩ Push Pull ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ q.enq(‘a’) ,_ ⟩ Serializable. Opaque. …

Thread Thread atomic { atomic { if(b) { while(…){ opA(…); if(x>MAX) } else … opB(…); } … }

Thread Thread atomic { atomic { if(b) { while(…){ opA(…); if(x>MAX) } else … opB(…); } … Linearizable Linearizable } Operation Operation

Thread Thread { tx c 1 , σ 1 } { tx c 2 , σ 2 } 1. Abstract away the language semantics

Thread Thread { tx c 1 , σ 1 } { tx c 2 , σ 2 } example language: 1. Abstract away the language semantics

Thread Thread G { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } ⟨ op1 , _ ⟩ ⟨ op2 , _ ⟩ 1. Abstract away the language semantics 2. Abstract away the state (with logs)

Thread Thread G { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } ⟨ op1 , _ ⟩ ⟨ op2 , _ ⟩ Closed under log prefix Parameterized by predicate allowed 1. Abstract away the language semantics 2. Abstract away the state (with logs)

Thread Thread G { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } ⟨ op1 , _ ⟩ ⟨ op2 , _ ⟩ Closed under log prefix Parameterized by predicate allowed Log Equality

Thread Thread G { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } ⟨ op1 , _ ⟩ ⟨ op2 , _ ⟩ 7 Simple Rules: Apply Push Pull Unpull Unpush Unapply Commit Log Equality

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G Two Logs ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) , σ , σ 1 , c ⟩ atomic { atomic { op1; op3; op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) , σ , σ 1 , c ⟩ atomic { atomic { ⟨ q.enq(‘a’) , σ 1 , σ ' 1 , c 1 ⟩ op1; op3; Apply op2; op4; … … } } 1 1 Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) , σ , σ 1 , c ⟩ atomic { atomic { ⟨ q.enq(‘a’) , σ 1 , σ ' 1 , c 1 ⟩ op1; op3; Apply op2; op4; … … } } A PPLY Criterion ( i ) : Possible next op 1 1 L 1 allows ⟨ m, σ 1 , σ ' 1 ,id ⟩ Criterion ( ii ) : Valid semantics of log fresh ( id ) Criterion ( iii ) : append { tx c 1 , σ 1 , L 1 }, G ➝ { tx c' 1 , σ ' 1 , L 1 ·[ op, σ 1 , σ ' 1 , c 1 ]}, G Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) , σ , σ 1 , c ⟩ atomic { atomic { Unapply ⟨ q.enq(‘a’) , σ 1 , σ ' 1 , c 1 ⟩ op1; op3; op2; op4; … … } } U NAPPLY { tx c' 1 , σ ' 1 , L 1 ·[ op, σ 1 , σ ' 1 , c 1 ]}, G 
 ➝ { tx c 1 , σ 1 , L 1 }, G Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G Two Logs ⟨ ht.map(7,2) ,_, gUC ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G Two Logs ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) ,_ ⟩ Push ⟨ ht.map(3,x) ,_, gUC ⟩ atomic { atomic { op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Application : Optimism vs. Pessimism Thread Thread Application : Out-of-order P USH ing with redo-logs { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G Two Logs ⟨ ht.map(7,2) ,_, gUC ⟩ Push ⟨ ht.map(3,x) ,_, gUC ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ Left-mover over logs op2; op4; … … } } P USH op ◀ ⎣ ` L ⎦ unpushed Criterion ( i ) : Act as if op happens next ⎣ G ⎦ gUC ∖ ⎣ L 1 ⎦ pushed ◀ op Criterion ( ii ) : No conflict w/ other uncmted Criterion ( iii ) : G allows op { tx c , σ , ` L ·[ op ]· L' }, G Uncommitted ➝ { tx c , σ , ` L ·[ op ]· L' }, G ·[ op , gUC ] Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread Application : Opacity [GK’08] and dependent transactions [RRHW’09] { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G ⟨ ht.map(7,2) ,_, gUC ⟩ Pull ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ atomic { atomic { Push ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ op2; op4; … … } } P ULL Criterion ( i ) : op ∉ L 2 Didn’t pull already Criterion ( ii ) : L 2 allows op Local log allows op op ◀ ⎣ L 2 ⎦ pushed ∪ ⎣ L 2 ⎦ unpushed Criterion ( iii ) : Can act as if op happened earlier { tx c 2 , σ 2 , L 2 }, G 1 ·[ op , g ]· G 2 ➝ { tx c 2 , σ 2 , L 2 ·[ op ]}, G 1 ·[ op , g ]· G 2 Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ Apply op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ Push ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ Pull op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ ht.get(5) , _, gUC ⟩ op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread { tx c 1 , σ 1 , L 1 } { tx c 2 , σ 2 , L 2 } G ⟨ ht.map(7,2) ,_, gUC ⟩ ⟨ ht.map(3,x) ,_, gUC ⟩ ⟨ ht.map(3,x) , _ ⟩ atomic { atomic { ⟨ ht.map(3,x) ,_ ⟩ Pull ⟨ q.enq(‘a’) ,_, gUC ⟩ ⟨ ht.get(5) , _ ⟩ op1; op3; ⟨ q.enq(‘a’) ,_ ⟩ ⟨ ht.get(5) , _ ⟩ ⟨ ht.get(5) , _, gUC ⟩ ⟨ q.enq(‘a’) , _ ⟩ op2; op4; … … } } Apply Push Pull Unpull Unpush Unapply Commit