A quick introduction to MPI (Message Passing Interface) M1IF - APPD - - PowerPoint PPT Presentation

▶
a quick introduction to mpi message passing interface
SMART_READER_LITE
LIVE PREVIEW

A quick introduction to MPI (Message Passing Interface) M1IF - APPD - - PowerPoint PPT Presentation

Oct 06, 2023 482 likes •826 views

Introduction Point-to-point communication Collective communications Custom communicators A quick introduction to MPI (Message Passing Interface) M1IF - APPD Oguz Kaya Pierre Pradic cole Normale Suprieure de Lyon, France 1 / 34 Oguz

slide-1
SLIDE 1

Introduction Point-to-point communication Collective communications Custom communicators

A quick introduction to MPI (Message Passing Interface)

M1IF - APPD Oguz Kaya Pierre Pradic

École Normale Supérieure de Lyon, France

1 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-2
SLIDE 2

Introduction Point-to-point communication Collective communications Custom communicators

Introduction

2 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-3
SLIDE 3

Introduction Point-to-point communication Collective communications Custom communicators What is MPI ?

Standardized and portable message-passing system. Started in the 90’s, still used today in research and industry. Good theoretical model. Good performances on HPC networks (InfiniBand ...).

3 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-4
SLIDE 4

Introduction Point-to-point communication Collective communications Custom communicators What is MPI ?

De facto standard for communications in HPC applications.

4 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-5
SLIDE 5

Introduction Point-to-point communication Collective communications Custom communicators Programming model

APIs:

C and Fortran APIs. C++ API deprecated by MPI-3 (2008).

Environment:

Many implementations of the standard (mainly OpenMPI and MPICH) Compiler (wrappers around gcc) Runtime (mpirun)

5 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-6
SLIDE 6

Introduction Point-to-point communication Collective communications Custom communicators Programming model

Compiling:

gcc → mpicc g++ → mpic++ / mpicxx gfortran → mpifort

Executing:

mpirun -n <nb procs> <executable> <args> ex : mpirun -n 10 ./a.out

note: mpiexec and orterun are synonyms of mpirun see man mpirun for more details

6 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-7
SLIDE 7

Introduction Point-to-point communication Collective communications Custom communicators MPI context

Context limits

All MPI call must be nested in the MPI context delimited by MPI_Init and MPI_Finalize.

1 #i n c l u d e <mpi . h> 2 3 i n t main ( i n t argc , char ∗ argv [ ] ) 4 { 5 MPI_Init(&argc , &argv ) ; 6 7 // . . . 8 9 MPI_Finalize ( ) ; 10 11 return 0; 12 }

7 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-8
SLIDE 8

Introduction Point-to-point communication Collective communications Custom communicators MPI context

Hello World

1 #i n c l u d e <s t d i o . h> 2 #i n c l u d e <mpi . h> 3 4 i n t main ( i n t argc , char ∗ argv [ ] ) 5 { 6 i n t rank , s i z e ; 7 8 MPI_Init(&argc , &argv ) ; 9 10 MPI_Comm_rank(MPI_COMM_WORLD, &rank ) ; 11 MPI_Comm_size(MPI_COMM_WORLD, &s i z e ) ; 12 13 p r i n t f ( " Hell o ␣from␣ proc ␣%d␣/␣%d\n" , rank , s i z e ) ; 14 15 MPI_Finalize ( ) ; 16 17 return 0; 18 }

8 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-9
SLIDE 9

Introduction Point-to-point communication Collective communications Custom communicators Synchronization

Code:

p r i n t f ( "[%d ] ␣ step ␣1\n" , rank ) ; MPI_Barrier (MPI_COMM_WORLD) ; p r i n t f ( "[%d ] ␣ step ␣2\n" , rank ) ;

Output:

[ 0 ] step 1 [ 1 ] step 1 [ 2 ] step 1 [ 3 ] step 1 [ 3 ] step 2 [ 0 ] step 2 [ 2 ] step 2 [ 1 ] step 2

9 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-10
SLIDE 10

Introduction Point-to-point communication Collective communications Custom communicators

Point-to-point communication

10 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-11
SLIDE 11

Introduction Point-to-point communication Collective communications Custom communicators

Send and Receive Sending data:

i n t MPI_Send( const void ∗ data , i n t count , MPI_Datatype datatype , i n t d e s t i n a t i o n , i n t tag , MPI_Comm communicator ) ;

Receiving data:

i n t MPI_Recv( void ∗ data , i n t count , MPI_Datatype datatype , i n t source , i n t tag , MPI_Comm communicator , MPI_Status∗ s t a t u s ) ;

11 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-12
SLIDE 12

Introduction Point-to-point communication Collective communications Custom communicators

Example

1 i n t rank , s i z e ; 2 MPI_Comm_rank(MPI_COMM_WORLD, &rank ) ; 3 MPI_Comm_size(MPI_COMM_WORLD, &s i z e ) ; 4 5 i n t number ; 6 switch ( rank ) 7 { 8 case 0: 9 number = −1; 10 MPI_Send(&number , 1 , MPI_INT , 1 , 0 , MPI_COMM_WORLD) ; 11 break ; 12 case 1: 13 MPI_Recv(&number , 1 , MPI_INT , 0 , 0 , MPI_COMM_WORLD, 14 MPI_STATUS_IGNORE) ; 15 p r i n t f ( " r e c e i v e d ␣number : ␣%d\n" , number ) ; 16 break ; 17 }

12 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-13
SLIDE 13

Introduction Point-to-point communication Collective communications Custom communicators

Asynchronious communications Sending data:

i n t MPI_Isend ( const void ∗ data , i n t count , MPI_Datatype datatype , i n t d e s t i n a t i o n , i n t tag , MPI_Comm communicator , MPI_Request∗ r e q u e s t ) ;

Receiving data:

i n t MPI_Irecv ( void ∗ data , i n t count , MPI_Datatype datatype , i n t source , i n t tag , MPI_Comm communicator , MPI_Request∗ r e q u e s t ) ;

13 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-14
SLIDE 14

Introduction Point-to-point communication Collective communications Custom communicators

Other functions

MPI_Probe, MPI_Iprobe MPI_Test, MPI_Testany, MPI_Testall MPI_Cancel MPI_Wtime, MPI_Wtick

14 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-15
SLIDE 15

Introduction Point-to-point communication Collective communications Custom communicators

Simple datatypes

MPI_SHORT short int MPI_INT int MPI_LONG long int MPI_LONG_LONG long long int MPI_UNSIGNED_CHAR unsigned char MPI_UNSIGNED_SHORT unsigned short int MPI_UNSIGNED unsigned int MPI_UNSIGNED_LONG unsigned long int MPI_UNSIGNED_LONG_LONG unsigned long long int MPI_FLOAT float MPI_DOUBLE double MPI_LONG_DOUBLE long double MPI_BYTE char

15 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-16
SLIDE 16

Introduction Point-to-point communication Collective communications Custom communicators

Composed datatypes

Composed structure:

Structures; Array.

Possibilities are almost limitless ... ... but sometimes difficult to setup.

16 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-17
SLIDE 17

Introduction Point-to-point communication Collective communications Custom communicators

Collective communications

17 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-18
SLIDE 18

Introduction Point-to-point communication Collective communications Custom communicators One-to-all communication

Broadcast

18 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-19
SLIDE 19

Introduction Point-to-point communication Collective communications Custom communicators One-to-all communication

Broadcast

i n t MPI_Bcast ( void ∗ data , i n t count , MPI_Datatype datatype , i n t root , MPI_Comm communicator ) ;

19 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-20
SLIDE 20

Introduction Point-to-point communication Collective communications Custom communicators One-to-all communication

Reduce

20 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-21
SLIDE 21

Introduction Point-to-point communication Collective communications Custom communicators One-to-all communication

Reduce

i n t MPI_Reduce( const void ∗ sendbuf , void ∗ recvbuf , i n t count , MPI_Datatype datatype , MPI_Op

  • perator ,

i n t root , MPI_Comm communicator ) ;

21 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-22
SLIDE 22

Introduction Point-to-point communication Collective communications Custom communicators One-to-all communication

Scatter

22 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-23
SLIDE 23

Introduction Point-to-point communication Collective communications Custom communicators One-to-all communication

Scatter

i n t MPI_Scatter ( const void ∗ sendbuf , i n t sendcount , MPI_Datatype sendtype , void ∗ recvbuf , i n t recvcount , MPI_Datatype recvtype , i n t root , MPI_Comm communicator ) ;

23 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-24
SLIDE 24

Introduction Point-to-point communication Collective communications Custom communicators One-to-all communication

Gather

24 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-25
SLIDE 25

Introduction Point-to-point communication Collective communications Custom communicators One-to-all communication

Gather

i n t MPI_Gather ( const void ∗ sendbuf , i n t sendcount , MPI_Datatype sendtype , void ∗ recvbuf , i n t recvcount , MPI_Datatype recvtype , i n t root , MPI_Comm communicator ) ;

25 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-26
SLIDE 26

Introduction Point-to-point communication Collective communications Custom communicators All-to-all communication

Allreduce

26 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-27
SLIDE 27

Introduction Point-to-point communication Collective communications Custom communicators All-to-all communication

AllReduce

i n t MPI_Allreduce ( const void ∗ sendbuf , void ∗ recvbuf , i n t count , MPI_Datatype datatype , MPI_Op

  • perator ,

MPI_Comm communicator ) ;

27 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-28
SLIDE 28

Introduction Point-to-point communication Collective communications Custom communicators All-to-all communication

Allgather

28 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-29
SLIDE 29

Introduction Point-to-point communication Collective communications Custom communicators All-to-all communication

AllGather

i n t MPI_Allgather ( const void ∗ sendbuf , i n t sendcount , MPI_Datatype sendtype , void ∗ recvbuf , i n t recvcount , MPI_Datatype recvtype , MPI_Comm communicator ) ;

29 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-30
SLIDE 30

Introduction Point-to-point communication Collective communications Custom communicators All-to-all communication

Alltoall

30 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-31
SLIDE 31

Introduction Point-to-point communication Collective communications Custom communicators All-to-all communication

Alltoall

i n t MPI_Alltoall ( const void ∗ sendbuf , i n t sendcount , MPI_Datatype sendtype , void ∗ recvbuf , i n t recvcount , MPI_Datatype recvtype , MPI_Comm communicator ) ;

31 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-32
SLIDE 32

Introduction Point-to-point communication Collective communications Custom communicators

Custom communicators

32 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-33
SLIDE 33

Introduction Point-to-point communication Collective communications Custom communicators

MPI_COMM_WORLD can be split into smaller, more appropriate communicators.

i n t MPI_Comm_split(MPI_Comm communicator , i n t color , i n t key , MPI_Comm∗ newcommunicator ) ;

33 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI

slide-34
SLIDE 34

Introduction Point-to-point communication Collective communications Custom communicators

Example

1 . . . 2 i n t rank , s i z e ; 3 MPI_Init(&argc , &argv ) ; 4 MPI_Comm_rank(MPI_COMM_WORLD, &rank ) ; 5 MPI_Comm_size(MPI_COMM_WORLD, &s i z e ) ; 6 7 i n t hrank , vrank ; 8 i n t hsize , v s i z e ; 9 MPI_Comm hcomm , vcomm ; 10 MPI_Comm_split(MPI_COMM_WORLD, rank%p , rank , &vcomm ) ; 11 MPI_Comm_split(MPI_COMM_WORLD, rank /p , rank , &hcomm ) ; 12 MPI_Comm_rank(hcomm , &hrank ) ; 13 MPI_Comm_size(hcomm , &h s i z e ) ; 14 MPI_Comm_rank(vcomm , &vrank ) ; 15 MPI_Comm_size(vcomm , &v s i z e ) ; 16 . . .

34 / 34 Oguz Kaya, Pierre Pradic M1IF - Presentation MPI