megatron.utils.print_all_nodes

megatron.utils.print_all_nodes(*args, **kwargs)

If distributed is initialized, print on the last rank in all nodes.