程序代写代做代考 flex Microsoft PowerPoint - mpi-2

Microsoft PowerPoint – mpi-2

High Performance Computing
Course Notes

Message Passing Programming II

Dr Ligang He

2Computer Science, University of Warwick

MPI functions

MPI is a complex system comprising of numerous
functions with various parameters and variants

Six of them are indispensable, but can write a large number
of useful programs already

Other functions add flexibility (datatype),

robustness (non-blocking send/receive), efficiency (ready-

mode communication), modularity (communicators, groups)

or convenience (collective operations, topology).

In the lectures, we are going to cover most commonly

encountered functions

3Computer Science, University of Warwick

Modularity

 MPI supports modular programming via communicators

 Provides information hiding by encapsulating local
communications and having local namespaces for
processes

 All MPI communication operations specify a communicator
(process group that is engaged in the communication)

4Computer Science, University of Warwick

Creating new communicators – Approach 1

MPI_Comm world, workers;

MPI_Group world_group, worker_group;

int ranks[1];

MPI_Init(&argc, &argv);

world=MPI_COMM_WORLD;

MPI_Comm_size(world, &numprocs);

MPI_Comm_rank(world, &myid);

server=numprocs-1;

MPI_Comm_group(world, &world_group);

ranks[0]=server;

MPI_Group_excl(world_group, 1, ranks, &worker_group);
MPI_Comm_create(world, worker_group, &workers);
MPI_Group_free(&world_group);

MPI_Comm_free(&workers);

int MPI_Group_excl (MPI_Group group, int n, int *ranks,
MPI_Group *newgroup)

5Computer Science, University of Warwick

Creating new communicators – functions

int MPI_Comm_group(MPI_Comm comm, MPI_Group
*group)

int MPI_Group_excl(MPI_Group group, int n, int *ranks,
MPI_Group *newgroup)

int MPI_Group_incl(MPI_Group group, int n, int *ranks,
MPI_Group *newgroup)

int MPI_Comm_create(MPI_Comm comm, MPI_Group
group, MPI_Comm *newcomm)

int MPI_Group_free(MPI_Group *group)

int MPI_Comm_free(MPI_Comm *comm)

6Computer Science, University of Warwick

Creating new communicators – Approach 2

MPI_Comm_split (comm, colour, key, newcomm)

Creates one or more new communicators from the original
comm

comm communicator (handle)

colour control of subset assignment (processes
with same colour are in same new
communicator)

key control of rank assignment

newcomm new communicator

Is a collective communication operation (must be executed by all
processes in the comm)

Is used to (re-) allocate processes to communicator (groups)

7Computer Science, University of Warwick

Creating new communicators – Approach 2

MPI_Comm_split (comm, colour, key, newcomm)

The number of communicators created is the same as the
number of different values of colour

 The processes with the same value of colour will be put in
the same communicator

These processes are assigned new ID (starting at zero)
with the order determined by the value of key

8Computer Science, University of Warwick

Creating new communicators – Approach 2

MPI_Comm_split (comm, colour, key, newcomm)

MPI_Comm comm, newcomm; int myid, color;
MPI_Comm_rank(comm, &myid); // id of current process
color = myid%3;

MPI_Comm_split(comm, colour, myid, *newcomm);

9Computer Science, University of Warwick

Communications

Point-to-point communications: involving exact two
processes, one sender and one receiver

For example, MPI_Send() and MPI_Recv()

Collective communications: involving a group of
processes

10Computer Science, University of Warwick

Collective operations

• Coordinated communication operations involving multiple
processes

• Programmer could do this by hand (tedious), MPI

• provides a specialized collective communications

barrier – synchronize all processes

reduction operations – sums, multiplies etc. distributed data

broadcast – sends data from one to all processes

gather – gathers data from all processes to one process

scatter – scatters data from one process to all processes

All executed collectively (by all processes in the group, at the
same time, with the same parameters)

11Computer Science, University of Warwick

Collective operations

MPI_Barrier (comm)

Global synchronization

comm is the communicator handle

No processes return from function until all processes have
called it

Good way of separating one phase from another

12Computer Science, University of Warwick

Barrier synchronizations

13Computer Science, University of Warwick

Implementation of Barrier

– Depending on the versions of MPI implementations
– A possible implementation is to pass a token message

around processes from process 0 to n-1,

– e.g., When a process calls MPI_Barrier
– If it is Process 0, it sends a token to process 1, and then waits

for the token to be received from process n-1
– For another process i, it waits to receive a token from process i-

1 and then send the token to process i+1

– No process can continue until the token has been passed
back to Process 0.

14Computer Science, University of Warwick

Collective operations

MPI_Bcast (buf, count, type, root, comm)

Broadcast data from root to all processes

buf address of receiver’s buffer or sender’s buffer (root)
count no. of entries in buffer (>=0)
type datatype of buffer elements
root process id of root process
comm communicator

15Computer Science, University of Warwick

Broadcast 100 ints from process 0 to every process
in the group

Example of MPI_Bcast

MPI_Comm comm;
int array[100];
int root = 0;

…
MPI_Bcast (array, 100, MPI_INT, root, comm);

16Computer Science, University of Warwick

Implementation of MPI_Bcast

Naïve Implementation:

Smarter Implementation:

The number of messages being transported is reduced.

17Computer Science, University of Warwick

Collective operations