Separating Functional and Parallel Correctness using - PowerPoint PPT Presentation

Separating Functional and Parallel Correctness using Nondeterministic Sequential Specifications Jacob Burnim, George Necula, Koushik Sen University of California, Berkeley 1 HotPar ' 10, Berkeley, CA June 14, 2010

Parallel Programming is Hard � Key Culprit: Nondeterministic interleaving of parallel threads. � Painful to reason simultaneously about parallelism and functional correctness. � Goal: Decompose efforts in addressing parallelism and functional correctness. � Allow programmers to reason about functional correctness sequentially . � Independently show correctness of parallelism. 2

Our Approach � Goal: Decompose efforts in addressing parallelism and functional correctness. Parallel Functional ≤ program specification 3

Our Approach � Goal: Decompose efforts in addressing parallelism and functional correctness. Program / specification Parallel Functional program specification 4

Our Approach � Goal: Decompose efforts in addressing parallelism and functional correctness. Parallelism Correctness. Prove independently of complex & sequential function correctness. Program / specification Parallel Functional program specification 5

Our Approach � Goal: Decompose efforts in addressing parallelism and functional correctness. Parallelism Correctness. Want to be able to Prove independently of reason about functional complex & sequential correctness without function correctness. parallel interleavings . Sequential program / Parallel Functional specification program specification 6

Our Approach � Use sequential but nondeterministic specification for a program’s parallelism. � User annotates intended nondeterminism . Nondeterministic sequential Parallel Functional program/spec program specification 7

Our Approach � Use sequential but nondeterministic specification for a program’s parallelism. � User annotates intended nondeterminism . Parallelism correct if Can address functional adds no unintended correctness without nondeterminism . parallel interleavings . Nondeterministic sequential Parallel Functional program/spec program specification 8

Outline � Overview � Motivating Example � Nondeterministic Sequential (NDSEQ) Specifications for Parallel Correctness � Proving Parallel Correctness � Future Work � Conclusions 9

Motivating Example � Goal: Find minimum-cost solution. � Simplified branch-and-bound benchmark. for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) if cost < best: best = cost best_soln = w 10

Motivating Example � Goal: Find minimum-cost solution. � Simplified branch-and-bound benchmark. Input: List of possible solutions. for (w in queue ): if (lower_bnd(w) >= best): continue cost = compute_cost(w) Output: Solution from if cost < best: input queue with best = cost minimum cost. best_soln = w 11

Motivating Example � Goal: Find minimum-cost solution. � Simplified branch-and-bound benchmark. for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost (w) if cost < best: best = cost Computes cost of best_soln = w solution w. Expensive . 12

Motivating Example � Goal: Find minimum-cost solution. � Simplified branch-and-bound benchmark. Computes cheap lower bound on cost of w. for (w in queue): if ( lower_bnd (w) >= best): continue cost = compute_cost (w) if cost < best: best = cost Computes cost of best_soln = w solution w. Expensive . 13

Motivating Example � Goal: Find minimum-cost solution. � Simplified branch-and-bound benchmark. Computes cheap lower bound on cost of w. for (w in queue): if ( lower_bnd (w) >= best): continue Prune when w cannot cost = compute_cost (w) have minimum-cost. if cost < best: best = cost Computes cost of best_soln = w solution w. Expensive . 14

Motivating Example bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 3 cost: 9 best: ∞� best_soln: � � for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) if cost < best: best = cost best_soln = w 15

Motivating Example bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 3 cost: 9 best: ∞� best_soln: � � prune?( a ) for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) if cost < best: best = cost best_soln = w 16

Motivating Example bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 3 cost: 9 best: 2 best_soln: � � prune?( a ) for (w in queue): if (lower_bnd(w) >= best): update( a ) continue cost = compute_cost(w) if cost < best: best = cost best_soln = w 17

Motivating Example bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 3 cost: 9 best: 2 best_soln: � � prune?( a ) for (w in queue): if (lower_bnd(w) >= best): update( a ) continue cost = compute_cost(w) prune?( b ) if cost < best: best = cost best_soln = w 18

Motivating Example bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 3 cost: 9 best: 2 best_soln: � � prune?( a ) for (w in queue): if (lower_bnd(w) >= best): update( a ) continue cost = compute_cost(w) prune?( b ) if cost < best: update( b ) best = cost best_soln = w 19

Motivating Example bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 3 cost: 9 best: 2 best_soln: � � prune?( a ) for (w in queue): if (lower_bnd(w) >= best): update( a ) continue cost = compute_cost(w) prune?( b ) if cost < best: update( b ) best = cost best_soln = w prune?( c ) 20

Motivating Example bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 3 cost: 9 best: 2 best_soln: � � prune?( a ) for (w in queue): if (lower_bnd(w) >= best): update( a ) continue cost = compute_cost(w) prune?( b ) if cost < best: update( b ) best = cost best_soln = w prune?( c ) 21

Motivating Example � Goal: Find minimum-cost solution. � Simplified branch-and-bound benchmark. for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) if cost < best: best = cost best_soln = w 22

Motivating Example � Goal: Find minimum-cost solution. How do we � Simplified branch-and-bound benchmark. parallelize this code? for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) if cost < best: best = cost best_soln = w 23

Parallelizing our Example � Goal: Find min-cost solution in parallel . � Simplified branch-and-bound benchmark. parallel-for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) atomic : if cost < best: best = cost best_soln = w 24

Parallelizing our Example � Goal: Find min-cost solution in parallel . � Simplified branch-and-bound benchmark. Loop iterations can be run in parallel. parallel-for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) atomic : if cost < best: Updates to best best = cost are atomic. best_soln = w 25

Prove Parallelism Correct? � Claim: Parallelization is correct. � If there are any bugs, they are sequential . � Want to prove parallelization correct. parallel-for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) atomic : if cost < best: best = cost best_soln = w 26

Prove Parallelism Correct? � Claim: Parallelization is correct. � If there are any bugs, they are sequential . � Want to prove parallelization correct. Idea: Specify that parallel version parallel-for (w in queue): if (lower_bnd(w) >= best): gives same result as sequential. continue cost = compute_cost(w) atomic : if cost < best: best = cost best_soln = w 27

Parallel-Sequential Equivalence? bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 2 cost: 9 best: ∞� best_soln: � � parallel-for (w in queue): if (lower_bnd(w) >= best): continue cost = compute_cost(w) atomic : if cost < best: best = cost best_soln = w 28

Parallel-Sequential Equivalence? bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 2 cost: 9 best: 2 best_soln: � � parallel-for (w in queue): prune?( a ) if (lower_bnd(w) >= best): update( a ) continue cost = compute_cost(w) atomic : if cost < best: best = cost best_soln = w 29

Parallel-Sequential Equivalence? bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 2 cost: 9 best: 2 best_soln: � � parallel-for (w in queue): prune?( a ) if (lower_bnd(w) >= best): update( a ) continue cost = compute_cost(w) prune?( b ) atomic : if cost < best: update( b ) best = cost best_soln = w 30

Parallel-Sequential Equivalence? bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 2 cost: 9 best: 2 best_soln: � � parallel-for (w in queue): prune?( a ) if (lower_bnd(w) >= best): update( a ) continue cost = compute_cost(w) prune?( b ) atomic : if cost < best: update( b ) best = cost prune?( c ) best_soln = w 31

Parallel-Sequential Equivalence? bound: 1 bound: 0 bound: 5 queue: ( a ) ( b ) ( c ) cost: 2 cost: 2 cost: 9 best: 2 best_soln: � � parallel-for (w in queue): prune?( a ) if (lower_bnd(w) >= best): Sequential program update( a ) continue cost = compute_cost(w) always finds prune?( b ) atomic : best_soln = (a) . if cost < best: update( b ) best = cost prune?( c ) best_soln = w 32