SLIDE 1
MPI Background
MPI : Message Passing Interface Began in Supercomputing ’92
Vendors
IBM, Intel, Cray
Library writers
PVM
Application specialists
National Laboratories, Universities
Introduction to Parallel Computing Irene Moulitsas Programming - - PowerPoint PPT Presentation
Introduction to Parallel Computing Irene Moulitsas Programming - - PowerPoint PPT Presentation
Introduction to Parallel Computing Irene Moulitsas Programming using the Message-Passing Paradigm MPI Background MPI : Message Passing Interface Began in Supercomputing 92 Vendors IBM, Intel, Cray Library writers PVM
SLIDE 2
SLIDE 3
Why MPI ?
One of the oldest libraries Wide-spread adoption. Portable. Minimal requirements on the underlying
hardware
Explicit parallelization
Intellectually demanding Achieves high performance Scales to large number of processors
SLIDE 4
MPI Programming Structure
Asynchronous
Hard to reason Non-deterministic behavior
Loosely synchronous
Synchronize to perform interactions Easier to reason
SPMD
Single Program Multiple Data
SLIDE 5
MPI Features
Communicator Information Point to Point communication Collective Communication Topology Support Error Handling
SLIDE 6
Six Golden MPI Functions
MPI is 125 functions MPI has 6 most used functions
SLIDE 7
MPI Functions: Initialization
Must be called by all processes MPI_SUCCESS
“mpi.h”
SLIDE 8
MPI Functions: Communicator
MPI_Comm
MPI_COMM_WORLD
SLIDE 9
Hello World !
SLIDE 10
Hello World ! (correct)
SLIDE 11
MPI Functions: Send, Recv
source
MPI_ANY_SOURCE
MPI_Status
MPI_SOURCE MPI_TAG MPI_ERROR
SLIDE 12
MPI Functions: Datatypes
SLIDE 13
Send/Receive Examples
SLIDE 14
Blocking Non-Buffered Communication
SLIDE 15
Send/Receive Examples
SLIDE 16
Blocking Buffered Communication
SLIDE 17
Send/Receive Examples
SLIDE 18
MPI Functions: SendRecv
SLIDE 19
MPI Functions: ISend, IRecv
Non-blocking MPI_Request
SLIDE 20
MPI Functions: Test, Wait
MPI_Test tests if operation finished. MPI_Wait blocks until operation is finished.
SLIDE 21
Non-Blocking Non-Buffered Communication
SLIDE 22
Example
SLIDE 23
Example
SLIDE 24
Example
SLIDE 25
MPI Functions: Synchronization
SLIDE 26
Collective Communications
One-to-All Broadcast All-to-One Reduction All-to-All Broadcast & Reduction All-Reduce & Prefix-Sum Scatter and Gather All-to-All Personalized
SLIDE 27
MPI Functions: Broadcast
SLIDE 28
MPI Functions: Scatter & Gather
SLIDE 29
MPI Functions: All Gather
SLIDE 30
MPI Functions: All-to-All Personalized
SLIDE 31
MPI Functions: Reduction
SLIDE 32
MPI Functions: Operations
SLIDE 33
MPI Functions: All-reduce
Same as MPI_Reduce, but all processes receive
the result of MPI_Op operation.
SLIDE 34
MPI Functions: Prefix Scan
SLIDE 35
MPI Names
SLIDE 36
MPI Functions: Topology
SLIDE 37
Performance Evaluation
Elapsed (wall-clock) time
SLIDE 38