CUDA RNDV protocols

The following are default rndv protocols with default UCX_RNDV_SCHEME=auto

Inter-node:

GPUDIrectRDMA enabled

• D-D

  Sender stages cuda fragments and PUTs to destination Device buffer

• D-H

  Sender stages cuda fragments and PUTs to destination Host buffer

• H-D

  put_zcopy protocol

• H-H

  get-zcopy protocol

GPUDIrectRDMA disabled

• D-D

  Receiver requests fragments from Sender (frag rtr) Sender stages fragments and PUTs to the receiver staging buffer and notifies (frag ATP) Receiver PUTs fragment to Device buffer.

• D-H

  Sender pipelines cuda fragments and PUTs to destination Host buffer

• H-D

  Receiver GETs host fragments and PUTs to destination Device buffer

• H-H

  get-zcopy protocol

Intra-node

CUDA-IPC

• D-D

  put_zcopy protocol

• D-H

  Sender stages cuda fragments and PUTs to destination Host buffer

• H-D

  put_zcopy protocol

• H-H

  get-zcopy protocol

No CUDA_IPC

• D-D

  Sender stages cuda fragments and PUTs(IB)to destination Device buffer

• D-H

  Sender pipelines cuda fragments and PUTs to destination Host buffer

• H-D

  put_zcopy protocol (IB).

• H-H

  get-zcopy protocol