Persistence Semantics for Weak Memory Integrating Epoch Persistency - PowerPoint PPT Presentation

Persistence Semantics for Weak Memory Integrating Epoch Persistency with the TSO Memory Model Azalea Raad Viktor Vafeiadis Max Planck Institute for Software Systems (MPI-SWS) Thursday 8 November OOPSLA 2018 Boston, USA azalea@mpi-sws.org SoundAndComplete.org @azalearaad

History Difficulty 😋 time Sequential

History Difficulty 😑 😋 time SC Sequential

History Difficulty ☹ 😑 😋 time SC WMC Sequential

History Difficulty 😤 ☹ 😑 😋 time SC WMC Persistent Sequential WMC

What is Persistent Memory? Volatile memory // x = 0 x := 1 // x = 1 // x = v : reading x yields v � 3

What is Persistent Memory? Volatile memory // x = 0 x := 1 // x = 1 // no recovery // x = 0 // x = v : reading x yields v � 3

What is Persistent Memory? Volatile memory Persistent memory // x = 0 // x = 0 x := 1 x := 1 // x = 1 // x = 1 // recovery routine // no recovery // x = 0 OR x = 1 // x = 0 // x = v : reading x yields v � 3

What is Persistent Memory? Volatile memory Persistent memory // x = 0 // x = 0 x := 1 x := 1 // x = 1 // x = 1 // recovery routine // no recovery // x = 0 OR x = 1 // x = 0 persists are asynchronous (buffered): may not persist immediately // x = v : reading x yields v � 3

(Sequential) Hardware

(Sequential) Hardware CPU (Volatile) Memory

(Sequential) Hardware CPU x:=1 : adds x:=1 to memory (Volatile) Memory

(Sequential) Hardware CPU x:=1 : adds x:=1 to memory a:=x : reads x from memory (Volatile) Memory

(Sequential) Hardware CPU x:=1 : adds x:=1 to memory a:=x : reads x from memory memory lost (Volatile) Memory

(Sequential) Hardware CPU x:=1 : adds x:=1 to memory a:=x : reads x from memory memory lost (Volatile) Memory CPU Persistence Buffer (Persistent) Memory

(Sequential) Hardware CPU x:=1 : adds x:=1 to memory a:=x : reads x from memory memory lost (Volatile) Memory CPU x:=1 : adds x:=1 to p-bu ff er Persistence Buffer (Persistent) Memory

(Sequential) Hardware CPU x:=1 : adds x:=1 to memory a:=x : reads x from memory memory lost (Volatile) Memory CPU x:=1 : adds x:=1 to p-bu ff er a:=x : if p-bu ff er contains x , reads latest entry else reads from memory Persistence Buffer (Persistent) Memory

(Sequential) Hardware CPU x:=1 : adds x:=1 to memory a:=x : reads x from memory memory lost (Volatile) Memory CPU x:=1 : adds x:=1 to p-bu ff er a:=x : if p-bu ff er contains x , reads latest entry else reads from memory Persistence Buffer p-bu ff er lost; memory retained (Persistent) Memory

(Sequential) Hardware CPU x:=1 : adds x:=1 to memory a:=x : reads x from memory memory lost (Volatile) Memory CPU x:=1 : adds x:=1 to p-bu ff er a:=x : if p-bu ff er contains x , reads latest entry else reads from memory Persistence Buffer p-bu ff er lost; memory retained unbu ff er* : p-bu ff er to memory (Persistent) Memory * at non-deterministic times

What is Memory Persistency Model? • Memory consistency model describes: 
 the order writes are made visible to other threads 
 e.g. SC, TSO, …

What is Memory Persistency Model? • Memory consistency model describes: 
 the order writes are made visible to other threads 
 e.g. SC, TSO, … • Memory persistency model describes: 
 the order writes are persisted to memory 
 e.g. Epoch Persistency

What is Memory Persistency Model? • Memory consistency model describes: 
 the order writes are made visible to other threads 
 Problem e.g. SC, TSO, … Formal Epoch Persistency Model for Mainstream Hardware ( Weak Memory Models) • Memory persistency model describes: 
 the order writes are persisted to memory 
 e.g. Epoch Persistency

What Can Go Wrong? // x=0;y=0 x := 1; y := 1; // recovery routine // x=0;y=0 OR x=1;y=1 OR x=1;y=0 OR x=0;y=1 � 6

What Can Go Wrong? // x=0;y=0 x := 1; y := 1; // recovery routine // x=0;y=0 OR x=1;y=1 OR x=1;y=0 OR x=0;y=1 !! Writes may persist out of order � 6

What Can Go Wrong? // x=0;y=0 x := 1; y := 1; // recovery routine // x=0;y=0 OR x=1;y=1 OR x=1;y=0 OR x=0;y=1 !! Writes may persist out of order ☛ persistent fence pfence � 6

Persistent Fence // x=0;y=0 x := 1; ☛ pfence; y := 1; // recovery routine // x=0;y=0 OR x=1;y=1 OR x=1;y=0 OR x=0;y=1 � 7

Persistent Fence x := 1; a y := 2; b x := 3; c pfence; z := 4; d � 8

Persistent Fence • writes on same locations persist in execution order x := 1; a persists before a c y := 2; b x := 3; c pfence; z := 4; d � 8

Persistent Fence • writes on same locations persist in execution order • writes on di ff erent locations are unordered x := 1; a persists before a c y := 2; b may persist in any order a b x := 3; c pfence; z := 4; d � 8

Persistent Fence • writes on same locations persist in execution order • writes on di ff erent locations are unordered • pfence adds a new epoch x := 1; a persists before a c y := 2; b epoch 1 may persist in any order a b x := 3; c pfence; z := 4; d epoch 2 � 8

Persistent Fence • writes on same locations persist in execution order • writes on di ff erent locations are unordered • pfence adds a new epoch • writes persist in epoch order x := 1; a persists before a c y := 2; b epoch 1 may persist in any order a b x := 3; c pfence; persist before a b c d z := 4; d epoch 2 � 8

What Can Go Wrong (Continued)? // x=0;y=0 x := 1; asynchronous pfence; (buffered) y := 1; // recovery routine // x=0;y=0 OR x=1;y=1 OR x=1;y=0 !! Execution continues ahead of persistence � 9

What Can Go Wrong (Continued)? // x=0;y=0 x := 1; asynchronous pfence; (buffered) y := 1; // recovery routine // x=0;y=0 OR x=1;y=1 OR x=1;y=0 !! Execution continues ahead of persistence ☛ persistent sync psync � 9

What Can Go Wrong (Continued)? // x=0;y=0 x := 1; asynchronous pfence; (buffered) y := 1; // recovery routine // x=0;y=0 OR x=1;y=1 OR x=1;y=0 !! Execution continues ahead of persistence ☛ persistent sync psync C1; psync; C2 • same persist-ordering as pfence • C2 executed only when all C1 writes have persisted � 9

Persistent Sync // x=0;y=0 x := 1; ☛ psync; y := 1; // recovery routine // x=0;y=0 OR x=1;y=1 OR x=1;y=0 !! Execution continues ahead of persistence ☛ persistent sync psync C1; psync; C2 • same persist-ordering as pfence • C2 executed only when all C1 writes have persisted � 10

(Sequential) Hardware x:=1 : adds x:=1 to p-bu ff er CPU a:=x : if p-bu ff er contains x , reads latest entry epoch n else reads from memory pfence . . . p-bu ff er lost; memory retained pfence epoch 2 pfence epoch 1 (Persistent) Memory � 11

(Sequential) Hardware x:=1 : adds x:=1 to p-bu ff er CPU a:=x : if p-bu ff er contains x , reads latest entry epoch n else reads from memory pfence . . . p-bu ff er lost; memory retained pfence epoch 2 pfence unbu ff er* : p-bu ff er to memory (in epoch order) epoch 1 (Persistent) Memory * at non-deterministic times � 11

(Sequential) Hardware x:=1 : adds x:=1 to p-bu ff er CPU a:=x : if p-bu ff er contains x , reads latest entry epoch n else reads from memory pfence . . . p-bu ff er lost; memory retained pfence epoch 2 pfence unbu ff er* : p-bu ff er to memory (in epoch order) epoch 1 pfence : introduces a new epoch in p-bu ff er (Persistent) Memory * at non-deterministic times � 11

(Sequential) Hardware x:=1 : adds x:=1 to p-bu ff er CPU a:=x : if p-bu ff er contains x , reads latest entry epoch n else reads from memory pfence . . . p-bu ff er lost; memory retained pfence epoch 2 pfence unbu ff er* : p-bu ff er to memory (in epoch order) epoch 1 pfence : introduces a new epoch in p-bu ff er (Persistent) Memory psync : flushes the entire p-bu ff er to memory * at non-deterministic times � 11

What about Concurrency? … TSO POWER ARMv8 � 12

What about Concurrency? … TSO POWER ARMv8 � 12

Contributions � 13

Contributions • PTSO: First formal epoch persistency semantics under 
 mainstream hardware ‣ Operational model ‣ Declarative model ‣ Equivalence of the two models � 13

Contributions • PTSO: First formal epoch persistency semantics under 
 mainstream hardware ‣ Operational model ‣ Declarative model ‣ Equivalence of the two models � 13

Contributions • PTSO: First formal epoch persistency semantics under 
 mainstream hardware ‣ Operational model ‣ Declarative model ‣ Equivalence of the two models • Verifying programs under PTSO ‣ PTSO programming pattern ‣ Correctness condition: persistent linearisability ‣ Verified several examples under PTSO � 13

Total Store Ordering (TSO) � 14

Total Store Ordering (TSO) Thread1 Thread2 Buffer Buffer (Volatile) Memory � 14

Total Store Ordering (TSO) Thread1 Thread2 x = 0; y = 0; Thread1 Thread2 ☛ ☛ x := 1; y := 1; a := y; c := x; Store Bu ff ering (SB) � 14

Total Store Ordering (TSO) Thread1 Thread2 x = 1 x = 0; y = 0; Thread1 Thread2 ☛ x := 1; y := 1; ☛ a := y; c := x; Store Bu ff ering (SB) � 14