The Challenges of Irregular Parallelism Chris Seaton - - PowerPoint PPT Presentation

▶
the challenges of irregular parallelism
SMART_READER_LITE
LIVE PREVIEW

The Challenges of Irregular Parallelism Chris Seaton - - PowerPoint PPT Presentation

Nov 16, 2022 188 likes •741 views

The Challenges of Irregular Parallelism Chris Seaton seatonc@cs.man.ac.uk chrisseaton.com Advanced Processor Technologies Group Supervisors: Ian Watson and Mikel Lujn Why should I write parallel programs? CPU CPU CPU H. Stutter. The

slide-1
SLIDE 1

The Challenges of Irregular Parallelism

Chris Seaton seatonc@cs.man.ac.uk chrisseaton.com

Supervisors: Ian Watson and Mikel Luján Advanced Processor Technologies Group

slide-2
SLIDE 2

Why should I write parallel programs?

slide-3
SLIDE 3

CPU

slide-4
SLIDE 4

CPU CPU

slide-5
SLIDE 5
  • H. Stutter. The free lunch is over – a fundamental turn toward concurrency in software. Dr. Dobb’s, 30(3), 2005.
slide-6
SLIDE 6

Core Core

slide-7
SLIDE 7

Core Core Core Core

slide-8
SLIDE 8

Core Core Core Core Core Core Core Core Core Core Core Core Core Core Core Core

slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11

We already know how to make parallel programs

slide-12
SLIDE 12

Structure your program so that there are tasks that can run at the same time

slide-13
SLIDE 13

Time

slide-14
SLIDE 14

Time

slide-15
SLIDE 15

A B C a b c

Matrix multiplication aA + bB + cC

slide-16
SLIDE 16

D E F a b c

Matrix multiplication aD + bE + cF

slide-17
SLIDE 17

Matrix multiplication

slide-18
SLIDE 18

Matrix multiplication

Time

slide-19
SLIDE 19

However, this only works if those tasks are entirely independent of each other

slide-20
SLIDE 20

Time

slide-21
SLIDE 21

Time Some kind of shared object - choke point

slide-22
SLIDE 22

Ideally don’t write software with shared objects that cause choke points

slide-23
SLIDE 23

Photo courtesy of Argonne National Laboratory.

slide-24
SLIDE 24

Let’s look at a tricky problem

slide-25
SLIDE 25
slide-26
SLIDE 26
slide-27
SLIDE 27
slide-28
SLIDE 28
slide-29
SLIDE 29
slide-30
SLIDE 30
slide-31
SLIDE 31
slide-32
SLIDE 32

Time The entire board is one big shared

  • bject
slide-33
SLIDE 33

We’re calling this an irregular problem - we can’t divide up the shared resource before the tasks start

slide-34
SLIDE 34

“It's easier to ask forgiveness than it is to get permission.”

This photograph is a work of a sailor or employee of the U.S. Navy, taken or made during the course of the person's official duties. As a work of the U.S. federal government, the image is in the public domain.

slide-35
SLIDE 35

We’ll assume that tasks will not get in each other’s way If they do, we’ll sort it out when it happens

slide-36
SLIDE 36

Time

slide-37
SLIDE 37

Time

slide-38
SLIDE 38

Time

slide-39
SLIDE 39

A little more time

slide-40
SLIDE 40

Two questions: How can you tell when one task gets in the way of another? How can you cancel a task that has already been running?

slide-41
SLIDE 41

Transactional memory

Instead of writing to memory, write to a log You can tell if two tasks are getting in each

  • ther’s way by comparing their logs

You can cancel a task by throwing the log away

slide-42
SLIDE 42
slide-43
SLIDE 43

0x80d2ef52 0x4ee47f35 0xd6b4eba9 0x2c86d524 0xe617f31d 0x40578fff 0x9bb7febc 0x4ddc0e5f 0xbd660807 0xecb10c62 0xbae0a866 0xe4c2615d 0x61639ad5 0xe617f31d 0x6a8bdcf3 0xe9e88989 0x8fbcf724 0x095b76c0

slide-44
SLIDE 44

0x80d2ef52 0x4ee47f35 0xd6b4eba9 0x2c86d524 0xe617f31d 0x40578fff 0x9bb7febc 0x4ddc0e5f 0xbd660807 0xecb10c62 0xbae0a866 0xe4c2615d 0x61639ad5 0xe617f31d 0x6a8bdcf3 0xe9e88989 0x8fbcf724 0x095b76c0

slide-45
SLIDE 45

0x80d2ef52 0x4ee47f35 0xd6b4eba9 0x2c86d524 0xe617f31d 0x40578fff 0x9bb7febc 0x4ddc0e5f 0xbd660807 0xecb10c62 0xbae0a866 0xe4c2615d 0x61639ad5 0xe617f31d 0x6a8bdcf3 0xe9e88989 0x8fbcf724 0x095b76c0

Actually write to memory

slide-46
SLIDE 46
slide-47
SLIDE 47

0x80d2ef52 0x4ee47f35 0xd6b4eba9 0x2c86d524 0xe617f31d 0x40578fff 0x9bb7febc 0x4ddc0e5f 0xbd660807 0x124683b3 0x60f005b2 0x831327fa 0xf69d8cf9 0x9ea7c8df 0x61f43a4a 0x170c4b44 0x7778a6aa 0x73068a29

slide-48
SLIDE 48

0x80d2ef52 0x4ee47f35 0xd6b4eba9 0x2c86d524 0xe617f31d 0x40578fff 0x9bb7febc 0x4ddc0e5f 0xbd660807 0x124683b3 0x60f005b2 0x831327fa 0xf69d8cf9 0x9ea7c8df 0x61f43a4a 0x170c4b44 0x7778a6aa 0x73068a29

Actually write to memory Actually write to memory

slide-49
SLIDE 49

Transactional memory is moving from research to production

C, C++, Java, Scala, Clojure, Haskell

slide-50
SLIDE 50

There are other techniques for reversing computation

If you know you added x to a value, subtract

  • x. If you know you added a node to a graph,

remove it. The Jefferson Time Warp System from the mid-80s - send anti-messages over networks.

slide-51
SLIDE 51

Do we have a solution?

slide-52
SLIDE 52

Transactional memory can be slow The hardware is probably limited Not the magic bullet some hoped Optimistic execution in general can be wasteful

slide-53
SLIDE 53

Irregular problems are the billion dollar problems

Physical meshes Web and social graphs Machine learning networks Data mining

[1]

[1] J. Tournois, C. Wormser, P . Alliez, and M. Desbrun. Interleaving Delaunay refinement and optimization for practical isotropic tetrahedron mesh generation. Technical Report 6826, INRIA, 2009.

slide-54
SLIDE 54

Chris Seaton seatonc@cs.man.ac.uk chrisseaton.com