无 阻塞通信
Non-Blocking Communications
Reusing this material
This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License.
http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_US
This means you are free to copy and redistribute the material and adapt and build on the
material under the following terms: You must give appropriate credit, provide a link to the license and indicate if changes were made. If you adapt or build on the material you must distribute your work under the same license as the original.
Acknowledge EPCC as follows: EPCC, The University of Edinburgh, www.epcc.ed.ac.uk
Note that this presentation contains images owned by others. Please seek their permission before reusing these images.
3
Deadlock
1
5
3
2
一
4 0
Communicator
4
– i.e. synchronous / asynchronous -5
Completion
• The mode of a communication determines when its constituent operations complete.
通信模式确定其组成操作何时完成。 -即同步/异步
• The form of an operation determines when the procedure implementing that operation will return
– i.e. when control is returned to the user program
操作的形式确定实现该操作的过程何时返回 -即控制权返回给用户程序时
5
Blocking Operations
• Relate to when the operation has completed.
• Only return from the subroutine call when the operation
has completed.
• These are the routines you used thus far – MPI_Ssend
– MPI_Recv
封锁作业
•与操作何时完成有关。 •仅当操作时从子例程返回
已经完成。 •这些是您到目前为止使用的例程-MPI_Ssend -MPI_Recv
6
Non-Blocking Operations
• Return straight away and allow the sub-program to continue to perform other work. At some later time the sub-program can test or wait for the completion of the non-blocking operation.
Beep! 嘟
7
非阻塞操作 •立即返回并允许子程序继续执行其他工
作。 稍后，子程序可以测试或等待非阻塞 操作的完成。
Non-Blocking Operations
• All non-blocking operations should have matching wait operations. Some systems cannot free resources until wait has been called. 。
• A non-blocking operation immediately followed by a matching wait is equivalent to a blocking operation.
• Non-blocking operations are not the same as sequential subroutine calls as the operation continues after the call has returned.
所有非阻塞操作都应具有匹配的等待操作。 某些系统在调用等待之前无法释放资源。 •立即执行匹配等待的非阻塞操作等同于阻塞操作。 •非阻塞操作与顺序子例程调用不同，因为调用返回后该操作将继续。
8
Non-Blocking Communications
noapgmade
8.
• Separate communication into three phases:
• Initiate non-blocking communication.
• Do some work (perhaps involving other communications?)
• Wait for non-blocking communication to complete. _
9
Non-Blocking Send
1
5
2
4
3
0
Communicator
10
Non-Blocking Receive
1
5
2
4
Communicator
3
0
11
Handles used for Non-blocking Comms
• datatype same as for blocking (MPI_Datatype or INTEGER).
• communicator same as for blocking (MPI_Comm or INTEGER).
• A request handle is allocated when a communication is
• request MPI_Request or INTEGER. .int?r.hkatragm
initiated.
-.-
12
Non-blocking Synchronous Send
MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request *request)
int MPI_Wait(MPI_Request *request, MPI_Status *status)
• Fortran:
MPI_ISSEND(buf, count, datatype, dest,
tag, comm, request, ierror)
MPI_WAIT(request, status, ierror)
Inmhate ,
• C:
int MPI_Issend(void* buf, int count,
13
Non-blocking Receive
• C:
int MPI_Irecv(void* buf, int count,
MPI_Datatype datatype, int src,
s t
int tag, MPI_Comm comm,
MPI_Request *request)
-_-
int MPI_Wait(MPI_Request *request,
MPI_Status *status)
• Fortran:
MPI_IRECV(buf, count, datatype, src,
tag, comm, request, ierror) MPI_WAIT(request, status, ierror)
14
B-locking and Non-Blocking
• Send and receive can be blocking or non-blocking.
• A blocking send can be used with a non-blocking receive, and vice-versa.
• Non-blocking sends can use any mode – synchronous, nnrnntrn
errnnnnarnn
buffered or standard
• Synchronous mode affects completion, not initiation. rnr-t-n-nblockiysendtnon-blockgreceive.no
n-blockigsendtblockingrec.ee . 15
Communication Modes
NON-BLOCKING OPERATION
MPI CALL
Standard send
MPI_ISEND
Synchronous send
二MPI_ISSEND MPI_IBSEND ㄨ MPI_IRECV
u
比
Buffered send
Receive
16
Completion
• Waiting versus Testing. • C:
int MPI_Wait(MPI_Request *request, 5MPI_Status *status)
int MPI_Test(MPI_Request *request,
int *flag,
MPI_Status *status)
01
Miùwnei
• Fortran:
MPI_WAIT(handle, status, ierror)
MPI_TEST(handle, flag, status, ierror)
17
Example(C) o0 1_
23
MPI_Request request; e s e s MPI_Status status;
if (rank == 0)
MPI_Issend(sendarray, 10, MPI_INT, 1, tag,
Do_something_else_while Issend_happens();
7 ({
_
UoU
,
…r a k
, s .ie :4
2
_Ftksender {
}
else if (rank == 1)
MPI_COMM_WORLD, &request);
// now wait for send to complete ‘ MPI_Wait(&request, &status);
MPI_Irecv(recvarray, 10, MPI_INT, 0, tag, MPI_COMM_WORLD, &request);
Do_something_else_while Irecv_happens(); // now wait for receive to complete;
}
Ttodereaiwr
MPI_Wait(&request, &status);
18
Example (Fortran)
integer request
integer, dimension(MPI_STATUS_SIZE) :: status
if (rank == 0) then
CALL MPI_ISSEND(sendarray, 10, MPI_INTEGER, 1, tag, MPI_COMM_WORLD, request, ierror)
CALL Do_something_else_while Issend_happens() ! now wait for send to complete
CALL MPI_Wait(request, status, ierror)
else if (rank == 1) then
CALL MPI_IRECV(recvarray, 10, MPI_INTEGER, 0, tag, MPI_COMM_WORLD, request, ierror)
CALL Do_something_else_while Irecv_happens()
! now wait for receive to complete CALL MPI_Wait(request, status, ierror)
endif
19
Multiple Communications
• Test or wait for completion of one message.
• Test or wait for completion of all messages.
• Test or wait for completion of as many messages as possible.
20
Testing Multiple Non-Blocking Comms
Process
in
in
in
21
Combined Send and Receive
• Specify all send / receive arguments in one call
– MPI implementation avoids deadlock
一次调用即可指定所有发送/接收 参数
-MPI实施避免死锁
-在简单的成对通信模式中很有 用，但不适用于非阻塞通信
– useful in simple pairwise communications patterns, but not as generally
applicable as non-blocking
int source, int recvtag,
MPI_Comm comm, MPI_Status *status);
int MPI_Sendrecv(void *sendbuf, int sendcount, MPI_Datatype sendtype, int dest, int sendtag,
mtreommǖǖ –
void *recvbuf, int recvcount, MPI_Datatype recvtype,
MPI_SENDRECV(sendbuf, sendcount, sendtype, dest, sendtag, recvbuf, recvcount, recvtype, source, recvtag,
comm, status, ierror)
22
Exercise
Rotating information around a ring
• See Exercise 4 on the sheet
• Arrange processes to communicate round a ring.
• Each process stores a copy of its rank in an integer
variable. – _ –
• Each process communicates this value to its right
neighbour, and receives a value from its left neighbour.
• Each process received.
compu
tes the
sum of all the values
。
• Repeat for the number of processes involved and print out the sum stored at each process.
23
Possible solutions
• Non-blocking send to forward neighbour – blocking receive from backward neighbour
– wait for forward send to complete
• Non-blocking receive from backward neighbour 0 竺 – blocking send to forward neighbour 然是 疑 – wait for backward receive to complete
• Non-blocking send to forward neighbour
• Non-blocking receive from backward neighbour
Irew
← wàt
– wait for forward send to complete
– wait for backward receive to complete
Isend . 0
← wait
ˋ
24
Notes
• Your neighbours do not change
– send to left, receive from right, send to left, receive from right,
• You do not alter the data you receive – receive it
– a一dd it to you running total
– pass the data unchanged along the ring
• You must not access send or receive buffers until communications are complete
– cannot read from a receive buffer until after a wait on irecv – cannot overwrite a send buffer until after a wait on issend
25