megatron.core.tensor_parallel.data

Description

Functions

broadcast_data(keys, data, datatype)

Broadcast data from rank zero of each model parallel group to the members of the same model parallel group.