megatron.initialize

Description

Megatron initialization.

Functions

initialize_megatron([extra_args_provider, ...])	Set global variables, initialize distributed, and set autoresume and random seeds.
set_jit_fusion_options(args)	Set PyTorch JIT layer fusion options.
write_args_to_tensorboard(args)	Write arguments to tensorboard.