Session 5 - Introductory Course in Parallel and Distributed - - PowerPoint PPT Presentation

▶

Mar 02, 2023 178 likes •330 views

Scope Sample Course Topics Session 5 - Introductory Course in Parallel and Distributed Computing Yves Robert & Behrooz Shirazi TCPP Workshop February 2010 yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 1/ 12

SLIDE 1

Scope Sample Course Topics

Session 5 - Introductory Course in Parallel and Distributed Computing

Yves Robert & Behrooz Shirazi TCPP Workshop February 2010

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 1/ 12

SLIDE 2

Scope Sample Course Topics

Outline

1

Scope

2

Sample Course Topics

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 2/ 12

SLIDE 3

Scope Sample Course Topics

Outline

1

Scope

2

Sample Course Topics

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 3/ 12

SLIDE 4

Scope Sample Course Topics

Approaches/High-level Topics

Global overview vs in-depth coverage of limited material Practical approach vs theory-oriented approach (applies to everything: algos/prog/distributed) Algorithms vs programming Parallel vs distributed computing Fine-grain (threads) vs coarse-grain (MPI) parallelism Parallel architectures (shared-memory, distributed memory, multicores) and their impact Applications and tools (CS vs non-CS major)

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 4/ 12

SLIDE 5

Scope Sample Course Topics

Outline

1

Scope

2

Sample Course Topics

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 5/ 12

SLIDE 6

Scope Sample Course Topics

Overview oriented course

Introduction to parallel computing Multi-core processors POSIX and Pthreads programming Cache memory and Cache Coherence in MP Interconnect Networks for MP Multi-computers MPI and MPI programming Current supercomputers Algorithms and programs

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 6/ 12

SLIDE 7

Scope Sample Course Topics

Practice-oriented course

Introduction to parallel processing Cray XMT architecture Cray XMT programming model Cray XMT programming tools Reasoning about performance on the Cray XMT Data parallelism Task parallelism Using and optimizing recursive algorithms Dataflow Graph algorithms Other cool algorithms

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 7/ 12

SLIDE 8

Scope Sample Course Topics

Theory-oriented course

Sorting networks P-RAM: EREW/CREW/CRCW, simulation, Brent’s theorem Cole’s sorting machine Pointer jumping, recursive doubling, list ranking, prefix sums Matrix operations Connected components, sorted paths Graph algorithms

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 8/ 12

SLIDE 9

Scope Sample Course Topics

Distributed computing

Models, broadcast Leader election in a ring, in a general graph The consensus problem Mutual exclusion, Atomic snapshots of shared memory Peer-to-peer computing DHT and join/search algorithms

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 9/ 12

SLIDE 10

Scope Sample Course Topics

Parallel architectures

Shared-memory computers SIMD, SPMD and BSP algorithms Cache and memory hierarchy, Level3 BLAS Multicores and GPUs Distributed-memory computers Macro-communications (broadcast, scatter, all-to-all, etc) Static interconnection networks (rings, grids, hypercubes, fat trees, etc) Dynamic interconnection networks (switches, Benes, Clos, etc) Routing, store-and-forward, wormhole, bandwidth sharing

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 10/ 12

SLIDE 11

Scope Sample Course Topics

Parallel algorithms

Models :Sorting networks Models: P-RAMS Interconnection networks Case study: ring and hypercube Peer-to-peer computing, distributed hash tables Algorithms on rings (matrices, stencils) Algorithms on 2D grids Load balancing for heterogeneous platforms Task graph scheduling Loop parallelization

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 11/ 12

SLIDE 12

Scope Sample Course Topics

Scheduling and automatic parallelization

Task graphs Makespan: polynomial/NP-complete instances List scheduling, Graham’s bound With communications and heterogeneous processors: HEFT Realistic models (one-port, bounded multi-port) Divisible load scheduling Steady-state scheduling Multi-criteria (throughput, latency, reliability, energy, cost) Dependence levels and Kennedy-Allen algorithm Dependence vectors and Lamport hyperplane method Unimodular matrices and space-time transformations

yves.robert@ens-lyon.fr, shirazi@wsu.edu February 2010 Introductory Course 12/ 12