Automatically Comparing Memory Consistency Models
1
John Wickerson Imperial Mark Batty U Kent Tyler Sorensen Imperial George A. Constantinides Imperial
S-REPLS @ Imperial Tuesday 27 September 2016
C++
x86
Automatically Comparing Memory Consistency Models John Wickerson - - PowerPoint PPT Presentation
C++ x86 Automatically Comparing Memory Consistency Models John Wickerson Mark Batty Tyler Sorensen George A. Constantinides Imperial U Kent Imperial Imperial S-REPLS @ Imperial Tuesday 27 September 2016 1 Contents Context: memory
1
John Wickerson Imperial Mark Batty U Kent Tyler Sorensen Imperial George A. Constantinides Imperial
S-REPLS @ Imperial Tuesday 27 September 2016
C++
x86
2
3
4
x = 1; y = 1; x = y = 0; r0 = y; r1 = x;
5
x = 1; x = y = 0; r0 = y; y = 1; r1 = x; r0 = 0, r1 = 1
6
x = 1; x = y = 0; r0 = y; y = 1; r1 = x; r0 = 0, r1 = 1 r0 = 1, r1 = 1
7
x = 1; x = y = 0; r0 = y; y = 1; r1 = x; r0 = 0, r1 = 1 r0 = 1, r1 = 1 r0 = 1, r1 = 0
8
x = 1; y = 1; x = y = 0; r0 = y; r1 = x; r0 = 0, r1 = 1 r0 = 1, r1 = 1 r0 = 1, r1 = 0 r0 = 0, r1 = 0
Subtleties related to relaxed memory have led to bugs in...
[Batty+ POPL'11, Batty+ ESOP'13],
[Alglave+ ASPLOS'15].
9
10
x = 1; y = 1; r0 = y; r1 = x;
11
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=0 sb sb rf
12
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb
13
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb
14
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
15
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
SC ✓
16
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
SC ✓ SC ✓
17
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
SC ✓ SC ✓ SC ✓
18
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
SC ✓ SC ✓ SC ✓ SC ×
19
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
x86 ✓ SC ✓ SC ✓ SC ✓ SC ×
20
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
x86 ✓ SC ✓ SC ✓ x86 ✓ SC ✓ SC ×
21
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
x86 ✓ SC ✓ x86 ✓ SC ✓ x86 ✓ SC ✓ SC ×
22
x = 1; y = 1; r0 = y; r1 = x;
W x=1 R y=1 W y=1 R x=1 rf sb sb rf W x=1 R y=1 W y=1 R x=0 sb sb rf W x=1 R y=0 W y=1 R x=1 rf sb sb W x=1 R y=0 W y=1 R x=0 sb sb
x86 ✓ SC ✓ x86 ✓ SC ✓ x86 ✓ SC ✓ x86 ✓ SC ×
23
24
Question Existing work
25
Question Existing work Can a given litmus test pass under a given MCM?
26
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ...
27
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM?
28
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10];
29
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another?
30
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd;
31
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation?
32
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation? manual c'examples [Vafeiaidis+ POPL'15]; manual proof [Sevcik PLDI'11];
33
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation? manual c'examples [Vafeiaidis+ POPL'15]; manual proof [Sevcik PLDI'11]; Does my MCM allow a given compiler mapping?
34
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation? manual c'examples [Vafeiaidis+ POPL'15]; manual proof [Sevcik PLDI'11]; Does my MCM allow a given compiler mapping? manual c'examples [Wickerson+ OOPSLA'15]; manual proof [Batty+ POPL'11, Batty+ POPL'12];
35
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation? manual c'examples [Vafeiaidis+ POPL'15]; manual proof [Sevcik PLDI'11]; Does my MCM allow a given compiler mapping? manual c'examples [Wickerson+ OOPSLA'15]; manual proof [Batty+ POPL'11, Batty+ POPL'12];
36
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation? manual c'examples [Vafeiaidis+ POPL'15]; manual proof [Sevcik PLDI'11]; Does my MCM allow a given compiler mapping? manual c'examples [Wickerson+ OOPSLA'15]; manual proof [Batty+ POPL'11, Batty+ POPL'12]; automatic generation
37
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation? manual c'examples [Vafeiaidis+ POPL'15]; manual proof [Sevcik PLDI'11]; Does my MCM allow a given compiler mapping? manual c'examples [Wickerson+ OOPSLA'15]; manual proof [Batty+ POPL'11, Batty+ POPL'12]; automatic generation automatic checking
38
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation? manual c'examples [Vafeiaidis+ POPL'15]; manual proof [Sevcik PLDI'11]; Does my MCM allow a given compiler mapping? manual c'examples [Wickerson+ OOPSLA'15]; manual proof [Batty+ POPL'11, Batty+ POPL'12]; automatic generation automatic checking automatic checking
39
Question Existing work Can a given litmus test pass under a given MCM? CppMem, Herd, MemSAT, Nemos, ... Which litmus tests can be run to check whether a machine conforms to a given MCM? semi-automatic generation with DIY [Alglave+ CAV'10]; Is one MCM more permissive than another? manual proof; manual examples [Batty+ POPL'16]; semi-automatic checking with DIY+Herd; Does my MCM allow a given compiler optimisation? manual c'examples [Vafeiaidis+ POPL'15]; manual proof [Sevcik PLDI'11]; Does my MCM allow a given compiler mapping? manual c'examples [Wickerson+ OOPSLA'15]; manual proof [Batty+ POPL'11, Batty+ POPL'12]; automatic generation automatic checking automatic checking automatic checking
40
Find (P,σ) where σ ∉ obsM(P) and σ ∈ obs0(P).
Find (P,σ) where σ ∉ obsM(P) and σ ∈ obs0(P).
No if ∃(P,σ) where σ ∉ obsN(P) and σ ∈ obsM(P).
Find (P,σ) where σ ∉ obsM(P) and σ ∈ obs0(P).
No if ∃(P,σ) where σ ∉ obsN(P) and σ ∈ obsM(P).
No if ∃(P,Q,σ) where σ ∉ obsM(P), σ ∈ obsM(Q) and P optimises to Q.
Find (P,σ) where σ ∉ obsM(P) and σ ∈ obs0(P).
No if ∃(P,σ) where σ ∉ obsN(P) and σ ∈ obsM(P).
No if ∃(P,Q,σ) where σ ∉ obsM(P), σ ∈ obsM(Q) and P optimises to Q.
No if ∃(P,Q,σ) where σ ∉ obsM(P), σ ∈ obsN(Q) and P compiles to Q.
Find (P,σ) where σ ∉ obsM(P) and σ ∈ obs0(P).
No if ∃(P,σ) where σ ∉ obsN(P) and σ ∈ obsM(P).
No if ∃(P,Q,σ) where σ ∉ obsM(P), σ ∈ obsM(Q) and P optimises to Q.
No if ∃(P,Q,σ) where σ ∉ obsM(P), σ ∈ obsN(Q) and P compiles to Q.
{(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
{(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
P {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
P Q {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
P Q ▸ {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
P Q Y ▸ Y'
some (consistent) execution must reach σ
{(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
P Q X Y ▸ X' Y'
every (consistent) execution must not reach σ some (consistent) execution must reach σ
{(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
X Y {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
X Y ∈ consistentN {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
∉ consistentM
X Y ∈ consistentN {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
∉ consistentM
P Q X Y ∈ consistentN {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
∉ consistentM
P Q X Y ▹ ∈ consistentN {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
∉ consistentM
P Q X Y ▸ ▹ ∈ consistentN {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
∉ consistentM
P Q X Y ▸ Y' ▹ ∈ consistentN {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
∉ consistentM
P Q X Y ▸ X' Y' ▹ ∈ consistentN {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
∉ consistentM
P Q X Y ▸ X' Y' ▹ ∈ consistentN ∈ deadM {(P,Q,σ) | σ ∉ obsM(P) ∧ σ ∈ obsN(Q) ∧ P ▸ Q}
63
WREL x 1 WREL y 2 RACQ y 1 WREL y 1 WREL x 2 RACQ x 1 co co rf rf sb sb sb sb
atomic_int x=0,y=0; x.store(1,REL); y.store(2,REL); r0=y.load(ACQ); y.store(1,REL); x.store(2,REL); r1=x.load(ACQ); r0==1 && r1==1
which we confirmed with Lahav et al. to be sufficient.
solution requiring 12 events and 6 threads
CRLX y 4/5 FREL WRLX x 1 CREL y 2/3 WRLX y 4 CACQ x 1/2 CREL x 2/3 WSC x 4 CSC y 1/2 CREL x 4/5 FAR WRLX y 1 sb sb sb sb sb sb co co co rf co rf co rf co rf co rf co rf
follows: WRLX x 1 CSC x 1/2 RSC y 0 WSC y 1 RSC x 1 S S sb sb S co rf rf
atomic_int x=0,y=0; x.store(1,RLX); r0=x.cas(1,2,SC,RLX); r1=y.load(SC); y.store(1,SC); r2=x.load(SC); r0==true && r1==0 && r2==1
RRLX x 1 Rna a 1 WSC y 1 RRLX y 1 WRLX x 1 Wna a 1 RRLX x 1 Rna a 1 WSC y 1 RRLX y 1 WRLX x 1 Wna a 1 sb cd sb cd sb cd sb cd sb cd sb cd sb rf rf rf rf rf rf π π π π π π
a: CAR,WG x 0/1 b: WREL,DV,REM x 2 co ⇣x 7!vd 2
x 7!L 0
⌘ InvA ⇣x 7!vd 2
x 7!L 0
⌘ ⇣
x 7!L 0
⌘ W x 2 ⇣x 7!vd 2
x 7!L 0
⌘ ⇣
x 7!L 0
⌘ Flu ⇣
x 7!L 0
⌘ ⇣
x 7! 0
⌘ Lk x ⇣
x 7!L 0
⌘ ⇣x 7!vd 2
x 7!L 0
⌘ Uk x ⇣x 7!vd 2
x 7! 0
⌘ ⇣
x 7! 0
⌘ fet x ⇣x 7!vc 0
x 7! 0
⌘ ⇣x 7!vc 0
x 7! 0
⌘ C x 0/1 ⇣x 7!vd 1
x 7! 0
⌘ ⇣x 7!vd 2
x 7! 0
⌘ flu x ⇣x 7!vc 2
x 7! 2
⌘ ⇣x 7!vd 1
x 7! 2
⌘ flu x ⇣x 7!vc 1
x 7! 1
⌘ co π π
GPU hardware to validate PTX2
75
76
John Wickerson Imperial Mark Batty U Kent Tyler Sorensen Imperial George A. Constantinides Imperial
Newcastle University Wednesday 31 August 2016
C++
x86