megatron.p2p_communication.send_forward_recv_forward#

megatron.p2p_communication.send_forward_recv_forward(output_tensor, recv_prev, tensor_shape=None, timers=None)#

Batched recv from previous rank and send to next rank in pipeline.