Parallel Search Ciaran McCreesh and Patrick Prosser This Weeks - - PowerPoint PPT Presentation

▶

parallel search

Parallel Search Ciaran McCreesh and Patrick Prosser This Weeks - - PowerPoint PPT Presentation

Sep 05, 2022 278 likes •704 views

Parallel Search Ciaran McCreesh and Patrick Prosser This Weeks Lectures Search and Discrepancies Parallel Constraint Programming Parallel Search Embarrassingly Parallel Search Work stealing Confidence Based Work Stealing Ciaran McCreesh

slide-1

SLIDE 1

Parallel Search

Ciaran McCreesh and Patrick Prosser

slide-2

SLIDE 2

This Week’s Lectures

Search and Discrepancies Parallel Constraint Programming Parallel Search

Embarrassingly Parallel Search Work stealing Confidence Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 1 / 12

slide-3

SLIDE 3

Fixed Parallel Tree Search, Again

Ciaran McCreesh and Patrick Prosser Parallel Search 2 / 12

slide-4

SLIDE 4

Fixed Parallel Tree Search, Again

Ciaran McCreesh and Patrick Prosser Parallel Search 2 / 12

slide-5

SLIDE 5

Fixed Parallel Tree Search, Again

Ciaran McCreesh and Patrick Prosser Parallel Search 2 / 12

slide-6

SLIDE 6

What We Need

We need a large parallelisable portion of the algorithm, and good work balance, or we don’t get much of an improvement.

Ciaran McCreesh and Patrick Prosser Parallel Search 3 / 12

slide-7

SLIDE 7

Embarrassingly Parallel Search

If we create n subproblems, chances are we’ll get poor balance. We can’t tell beforehand where the really hard subproblems will be. What if we create lots of subproblems, and distribute them dynamically?

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-8

SLIDE 8

Embarrassingly Parallel Search

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-9

SLIDE 9

Embarrassingly Parallel Search

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-10

SLIDE 10

Embarrassingly Parallel Search

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-11

SLIDE 11

Embarrassingly Parallel Search

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-12

SLIDE 12

Embarrassingly Parallel Search

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-13

SLIDE 13

Embarrassingly Parallel Search

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-14

SLIDE 14

Embarrassingly Parallel Search

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-15

SLIDE 15

Embarrassingly Parallel Search

Ciaran McCreesh and Patrick Prosser Parallel Search 4 / 12

slide-16

SLIDE 16

Randomised Work Stealing

What if we split work entirely dynamically? Whenever a worker is idle, have it steal a subproblem from another randomly selected worker.

Ciaran McCreesh and Patrick Prosser Parallel Search 5 / 12

slide-17

SLIDE 17

Randomised Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 5 / 12

slide-18

SLIDE 18

Randomised Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 5 / 12

slide-19

SLIDE 19

Randomised Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 5 / 12

slide-20

SLIDE 20

Randomised Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 5 / 12

slide-21

SLIDE 21

Randomised Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 5 / 12

slide-22

SLIDE 22

Randomised Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 5 / 12

slide-23

SLIDE 23

Speedups from Parallel Tree Search

If a decision problem is satisfiable, or for an optimisation problem, our speedups could be arbitrary: one worker might find a feasible or strong solution very quickly. In particular, superlinear speedups are possible, as are no speedups at all. For an unsatisfiable decision problem, or for an enumeration problem, our speedups can be at best linear (assuming we do not change the search tree): we are dividing up a fixed amount

f work.

If we do not communicate bounds, or if we do not preserve sequential ordering, we could get an absolute slowdown.

Ciaran McCreesh and Patrick Prosser Parallel Search 6 / 12

slide-24

SLIDE 24

Work Splitting Affects Search

Ciaran McCreesh and Patrick Prosser Parallel Search 7 / 12

slide-25

SLIDE 25

Work Splitting Affects Search

⋆

Ciaran McCreesh and Patrick Prosser Parallel Search 7 / 12

slide-26

SLIDE 26

Work Splitting Affects Search

⋆

Ciaran McCreesh and Patrick Prosser Parallel Search 7 / 12

slide-27

SLIDE 27

Work Splitting Affects Search

For satisfiable instances and optimisation problems, where you split the work doesn’t just affect balance. It also affects the amount of work to do. We just saw an example where better work balance gave worse performance, because it took longer to find a solution. Remember Harvey and Ginsberg’s Limited Discrepancy Search?

2 Value-ordering heuristics are most likely to wrong higher up in

the tree (there is least information available when no or few choices have been made).

Stealing early or splitting high introduces diversity against early heuristic choices. Stealing late gives a close-to-sequential

rdering.

Ciaran McCreesh and Patrick Prosser Parallel Search 7 / 12

slide-28

SLIDE 28

Confidence-Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 8 / 12

slide-29

SLIDE 29

Confidence-Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 8 / 12

slide-30

SLIDE 30

Confidence-Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 8 / 12

slide-31

SLIDE 31

Confidence-Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 8 / 12

slide-32

SLIDE 32

Confidence-Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 8 / 12

slide-33

SLIDE 33

Using Confidence-Based Work Stealing

I would like to be able to tell you that all you need to do to get excellent speedups from parallelism, and all of the benefits of discrepancy searches with none of the costs, is to annotate your model and your heuristics a little bit and then use Confidence-Based Work Stealing. But the implementations aren’t quite at that stage yet.

Ciaran McCreesh and Patrick Prosser Parallel Search 9 / 12

slide-34

SLIDE 34

Using Confidence-Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 9 / 12

slide-35

SLIDE 35

Using Confidence-Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 9 / 12

slide-36

SLIDE 36

Using Confidence-Based Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 9 / 12

slide-37

SLIDE 37

Early Diversity Work Stealing

Confidence seems hard. What if we just steal as early as possible, and subject to that, as far left as possible? We’ll get reproducible runtimes, can guarantee we’ll never get a substantial slowdown, and adding more processors will never make things worse. Added bonus: we can parallelise conflict-directed backjumping this way too (although we no longer expect a linear speedup for unsatisfiable instances).

Ciaran McCreesh and Patrick Prosser Parallel Search 10 / 12

slide-38

SLIDE 38

Early Diversity Work Stealing

Ciaran McCreesh and Patrick Prosser Parallel Search 10 / 12

slide-39

SLIDE 39

Early Diversity Work Stealing

100 101 102 103 104 105 106 107 108 100 101 102 103 104 105 106 107 108 Parallel Runtime (ms) Sequential Runtime (ms) LV (sat) LV (unsat) BVG / BVGm M4D / M4Dr SF (sat) SF (unsat) r football images (sat) images (unsat) meshes (sat) meshes (unsat)

slide-40

SLIDE 40

Early Diversity Work Stealing

500 1000 1500 2000 2500 100 101 102 103 104 105 106 107 108 Instances solved Sequential Runtime (ms) Sequential Threaded

slide-41

SLIDE 41

This is Not The Exam Question

What is balance, and why is it a problem if we try to parallelise a tree-search by creating n sub-trees for n processors? Suggest two potential remedies. Why does Amdahl’s law not apply to parallel tree-search? Why are super-linear speedups possible? Suppose we are solving a decision problem which has a sequential part taking one second to run, and a parallelisable part which takes twenty seconds to run on one processor. What is the best possible runtime we might see when using ten processors to solve this problem with a parallel tree-search, if the instance is satisfiable? What if it is unsatisfiable?

Ciaran McCreesh and Patrick Prosser Parallel Search 12 / 12

slide-42

SLIDE 42