Race Why is parallelism hard? Non-determinism!! Practice Theory - - PowerPoint PPT Presentation

Parallel Algorithms: Theory and Practice

Race

CS26 S260 – Lecture cture 9* Yan n Gu

Why is parallelism “hard”?

2

Theory Practice

Non-determinism!!

Why is parallelism “hard”?

3

• Sc

Schedu duli ling ng is is unkno nown wn

• Rela

lativ ive e orderin ing g for operatio tions ns is is un unknown nown

• Hard to d

debug ug

• Bugs can be non-deterministic!
• Bugs can be different if you rerun the code
• Referred to as race hazard / condition

Non-determinism!!

Race hazard can cause severe consequences

4

• Therac-25 radia

iation tion therapy py mach chin ine e — kil ille led 3 3 people le and d serio iously usly in injur ured ed many y more (betwee ween n 1985 a and d 1987).

https:// //en.wiki wikiped pedia. a.org/ rg/wiki wiki/Th /Therac rac-25 25

• North America

ican n Bla lackout ut of 2003 2003 — le left 50 m mil illi lion n peopl ple e wit ithout ut power for up to a a w week. .

https ps:// //en. en.wiki kiped pedia.org

• rg/w

/wiki ki/N /North

• rthea

east st_bl blacko ckout_ t_of_

• f_2

003 003

• Race

ce bugs s are notorio

• riousl

usly y difficult icult to discover cover by y co conventional nventional testing! ting!

Determinacy Races

• Definiti

inition: n: a deter ermina minacy cy race occurs when n two lo logi gicall lly para ralle llel l in instru ructi ctions

• ns acc

ccess ss the same me memo mory ry lo loca catio ion n and at le least t one of the in instru tructio tions ns perform

• rms

s a w writ ite.

5

direct_reduce(A, n) { parallel_for (i=0;i<n;i++) sum = sum + 1; return sum; }

Determinacy Races

• Definiti

inition: n: a deter ermina minacy cy race occurs when n two lo logi gicall lly para ralle llel l in instru ructi ctions

• ns acc

ccess ss the same me memo mory ry lo loca catio ion n and at le least t one of the in instru tructio tions ns perform

• rms

s a w writ ite.

6

sum = 0 r0 = sum r0 += 1 sum = r0 r1 = sum r1 += 1 sum = r1 Return sum direct_reduce(A, n) { parallel_for (i=0;i<2;i++) sum = sum + 1; return sum; }

Determinacy Races

• Definiti

inition: n: a deter ermina minacy cy race occurs when n two lo logi gicall lly para ralle llel l in instru ructi ctions

• ns acc

ccess ss the same me memo mory ry lo loca catio ion n and at le least t one of the in instru tructio tions ns perform

• rms

s a w writ ite.

7

sum = 0 r0 = sum r0 += 1 sum = r0 r1 = sum r1 += 1 sum = r1 return sum 1 2 3 4 5 6 direct_reduce(A, n) { parallel_for (i=0;i<2;i++) sum = sum + 1; return sum; }

Determinacy Races

• Definiti

inition: n: a deter ermina minacy cy race occurs when n two lo logi gicall lly para ralle llel l in instru ructi ctions

• ns acc

ccess ss the same me memo mory ry lo loca catio ion n and at le least t one of the in instru tructio tions ns perform

• rms

s a w writ ite.

8

direct_reduce(A, n) { parallel_for (i=0;i<2;i++) sum = sum + 1; return sum; } sum = 0 r0 = sum r0 += 1 sum = r0 r1 = sum r1 += 1 sum = r1 return sum 1 2 5 3 4 6

Types of Races

• Su

Suppose

• se that in

instruct uction ion A and in instruct uction ion B both access s a lo loca catio ion n x, and suppose ppose that t A∥B (A is parallel to B).

• Two se

section ions s of code are in indepe epend ndent ent if if they y have e no determi rminac nacy y races s between een them. m.

9

A B Race Type Read Read No race Read Write Read race Write Read Read race Write Write Write race

Avoiding races

• Itera

rations ions of a para rall llel_for el_for lo loop shoul uld d be in indepen ependent dent

• Between

een two in in_pa paral alle lel tasks, s, the code of the spaw awned ned chil ild shoul uld d be in indepe epend ndent ent of th the code of the parent nt, , in inclu ludi ding ng code execut cuted ed by addi dition ional al spawne wned d or ca call lled chil ildr dren en

10

Benefit of being race-free

• Sc

Schedu duli ling ng is is stil ill l unkno nown wn

• Re

Rela lativ ive e ord rderi ring ng for r opera ratio tions ns is is st stil ill l unkno nown wn

• However

er, , the comput uted ed valu lue e of ea each in instruc uction tion is is determi rminist nistic! ic! This is is is ea easy y to d debug. ug.

• Check the correctness of the sequential execution
• Check if the parallel execution is the same as the sequential one
• Race detection

tion: : gi given a D DAG, G, show w all ll th the races

• Fals

lse e sharing ring: : nast sty y rela lated ed effec fect

• E.g., updating x.a and x.b in parallel is safe

but can be inefficient

11

Struct { char a, b; } x;