The C Language: It Is Not What You Think It Is! Mark Batty and Peter Sewell University of Cambridge Joint work of Jade Alglave, Mark Batty, Luc Maranget, Robin Morisset, Justus Matthiesen, Kayvan Memarian, Paul McKenney, Scott Owens, Pankaj Pawan, Susmit Sarkar, Peter Sewell, Francesco Zappa Nardelli, and others. p. 1 Linux Collaboration Summit, April 2013
p. 2
C is? What the Standards say: ISO/IEC 9899:1999 ISO/IEC 9899:2011 endless DRs and internet discussions What the existing compilers provide ...and what future compilers will provide “the newer version of Clang is stricter about certain things” [Behan Webster, 2.15pm] What the corpus of C code requires What programmers believe What analysis and verification tools assume p. 3
A Simple Message-Passing Example (MP) Thread 0 Thread 1 Thread 0 Thread 1 a: W[x]=1 c: R[y]=1 data=1 while (flag � = 1) {} if this reads 1... rf po po flag=1 print data ...could this read 0? b: W[y]=1 d: R[x]=0 rf Initial state: data = 0 flag = 0 Test MP p. 4
A Simple Message-Passing Example (MP) Thread 0 Thread 1 Thread 0 Thread 1 a: W[x]=1 c: R[y]=1 data=1 while (flag � = 1) {} if this reads 1... rf po po flag=1 print data ...could this read 0? b: W[y]=1 d: R[x]=0 rf Initial state: data = 0 flag = 0 Test MP x86: ./runMP.sh p. 4
A Simple Message-Passing Example (MP) Thread 0 Thread 1 Thread 0 Thread 1 a: W[x]=1 c: R[y]=1 data=1 while (flag � = 1) {} if this reads 1... rf po po flag=1 print data ...could this read 0? b: W[y]=1 d: R[x]=0 rf Initial state: data = 0 flag = 0 Test MP x86: ./runMP.sh POWER and ARM: POWER ARM Kind PowerG5 Power6 Power7 Tegra2 Tegra3 APQ8060 A5X MP Allow 10M/4.9G 6.5M/29G 1.7G/167G 40M/3.8G 138k/16M 61k/552M 437k/185M p. 4
A Simple Message-Passing Example (MP) Thread 0 Thread 1 Thread 0 Thread 1 a: W[x]=1 c: R[y]=1 data=1 while (flag � = 1) {} if this reads 1... rf po po flag=1 print data ...could this read 0? b: W[y]=1 d: R[x]=0 rf Initial state: data = 0 flag = 0 Test MP x86: ./runMP.sh POWER and ARM: POWER ARM Kind PowerG5 Power6 Power7 Tegra2 Tegra3 APQ8060 A5X MP Allow 10M/4.9G 6.5M/29G 1.7G/167G 40M/3.8G 138k/16M 61k/552M 437k/185M GCC and HotSpot: observable (in some contexts) p. 4
A Simple Message-Passing Example (MP) Thread 0 Thread 1 Thread 0 Thread 1 a: W[x]=1 c: R[y]=1 data=1 while (flag � = 1) {} if this reads 1... rf po po flag=1 print data ...could this read 0? b: W[y]=1 d: R[x]=0 rf Initial state: data = 0 flag = 0 Test MP x86: ./runMP.sh POWER and ARM: POWER ARM Kind PowerG5 Power6 Power7 Tegra2 Tegra3 APQ8060 A5X MP Allow 10M/4.9G 6.5M/29G 1.7G/167G 40M/3.8G 138k/16M 61k/552M 437k/185M GCC and HotSpot: observable (in some contexts) C/C++11 standards: undefined program p. 4
Explaining MP Have to deal with h/w relaxations (for multiple architectures) and compiler optimisations Thread 0 Thread 1 Thread 0 Thread 1 a: W[x]=1 c: R[y]=1 data=1 while (flag � = 1) ; if this reads 1... rf po po flag=1 print data ...could this read 0? b: W[y]=1 d: R[x]=0 rf Initial state: data = 0 flag = 0 Test MP p. 5
Explaining MP Have to deal with h/w relaxations (for multiple architectures) and compiler optimisations Thread 0 Thread 1 r = data Thread 0 Thread 1 a: W[x]=1 c: R[y]=1 data=1 while (flag � = 1) ; if this reads 1... rf po po flag=1 print data ...could this read 0? b: W[y]=1 d: R[x]=0 rf Initial state: data = 0 flag = 0 Test MP p. 5
A Less Simple Example POWER ARM Kind PowerG5 Power6 Power7 Tegra2 Tegra3 APQ8060 A5X PPOCA Allow 1.1k/3.4G 0/49G 175k/157G 0/24G 0/39G 233/743M 0/2.2G PPOAA Forbid 0/3.4G 0/46G 0/209G 0/24G 0/39G 0/26G 0/2.2G Thread 0 Thread 1 a: W[z]=1 c: R[y]=1 dmb/sync ctrl rf b: W[y]=1 d: W[x]=1 rf e: R[x]=1 addr rf f: R[z]=0 Test PPOCA p. 6
Options 1. program w.r.t. particular h/w architecture and compiler optimisation knowledge 2. define macros that try to abstract (de facto Linux std) 3. incorporate into language semantics (C/C++11) p. 7
A data-race-free model H/W memory models must define the behaviour of all programs For C/C++ can deem racy programs to be bad, saying: programs that are race-free in sequentially consistant (SC) semantics have SC behaviour programs that have a race in some execution in SC semantics can behave in any way at all ...and then add several tiers of low-level atomics , each with more performance (and relaxed behaviour) than the last: SC, release/acquire, release/consume, relaxed. p. 8
What we’ve done clarify multicore h/w behaviour: x86, ARM, IBM Power clarify C/C++11 concurrency semantics and standards Central artifacts: mathematical reference models describes the set of all allowed behaviours of a program mathematically precise (might be wrong, but not ambiguous) as clear as possible executable as test oracle (for small programs) sound (w.r.t. implementations or standards, variously) p. 9
Using the models cppmem tool C/C++11 concurrency compilation scheme (research) compiler verification (production) compiler testing p. 10
� � Testing Compilers vs Semantics [Robin Morisset, Pankaj Pawan, Francesco Zappa Nardelli] Reduce problem to testing compilers on sequential programs: random sequential program � � � �������������������� � GCC -O2 � � � � � � � � reference semantics ( GCC -O0 ) � opt executable execute sound-transformation ? � opt trace reference trace A transformation is sound if it does not introduce any behavior in any non-racy context. [PLDI 2013] p. 11
Compiler Concurrency Bug int g1 = 1; int g2 = 0; int func1(void) { int l; for (l = 0; (l != 4); l++) { if (g1) return l; for (g2 = 0; (g2 >= 26); ++g2) ; } } int main (int argc, char* argv[]) { func1(); } miscompiled by GCC 4.7.0 with -O2 p. 12
Csem: Scaling up to more of C? Remember, we want a usable reference model. Formalise the Standard? p. 13
Csem: Scaling up to more of C? Remember, we want a usable reference model. Formalise the Standard? Maybe as a starting point — but immediately hit cases where ISO standard and de facto standard differ. p. 13
Simple Questions PC.1 In practice, can one compare (with <, >, <=, or >=) two pointers to separately allocated objects (of compatible object types)? p. 14
Simple Questions PC.1 In practice, can one compare (with <, >, <=, or >=) two pointers to separately allocated objects (of compatible object types)? asked 11 experts: 4 used and supported in practice 2 not useful, but compilers do support it 4 should not be used; compilers might well not support it 1 don’t know PC.1 (standard) Is it allowed by the C standard? 1 allowed 6 not allowed 3 don’t know p. 14
Simple Questions PC.1 In practice, can one compare (with <, >, <=, or >=) two pointers to separately allocated objects (of compatible object types)? “Again, memory allocator implementations do this a lot. Also, this is used in practice to avoid deadlock – if a pair of locks to be acquired have the same address, only acquire one of them. (In fact, I am currently working on a patch that does just this.)” p. 14
Simple Questions PC.1 In practice, can one compare (with <, >, <=, or >=) two pointers to separately allocated objects (of compatible object types)? “May produce inconsistent results in practice if p and q straddle the exact middle of the address space. We’ve run into practical problems with this.” p. 14
Simple Basic Questions PR.1.a Given two separately allocated objects of the same type, can a pointer to the first be turned into a usable pointer to the second, by pointer arithmetic together with a check that the result has an identical representation to that of another usable pointer to the second? int y = 2, x=1; int main() { intptr t offset = &y - &x; // ? int *p = &x + offset, *q = &y; if (memcmp(&p, &q, sizeof(p)) == 0) { *p = 11; // ? printf("x=%d y=%d *p=%d *q=%d",x,y,*p,*q); } } p. 15
Simple Basic Questions PR.1.a Given two separately allocated objects of the same type, can a pointer to the first be turned into a usable pointer to the second, by pointer arithmetic together with a check that the result has an identical representation to that of another usable pointer to the second? int y = 2, x=1; int main() { intptr t offset = &y - &x; // ? int *p = &x + offset, *q = &y; if (memcmp(&p, &q, sizeof(p)) == 0) { *p = 11; // ? printf("x=%d y=%d *p=%d *q=%d",x,y,*p,*q); } } $gcc-4.7 -std=c11 -pedantic -Wall -O1: x=1 y=11 *p=11 *q=11 $gcc-4.7 -std=c11 -pedantic -Wall -O2: x=1 y=2 *p=11 *q=2 p. 15
moral: what matters is not just what happens to the bits at runtime, but what the compiler analysis assumes p. 16
Basic Questions ET.1 If one uses a pointer to a struct to initialise a single member of the struct in a malloc’d region, does 1. the footprint of the whole struct take on the struct type as its effective type, or 2. does the footprint of the member take on an effective type of that struct member, or 3. does the footprint of the member take on an effective type as the type of that struct member? p. 17
Recommend
More recommend
