megatron.training

Description

Pretrain utilities.

Functions

build_train_valid_test_data_iterators(...[, ...])
cyclic_iter(iter)
evaluate(forward_step_func, data_iterator, ...)	Evaluation.
evaluate_and_print_results(prefix, ...[, ...])	Helper function to evaluate and dump results on screen.
get_model(model_provider_func[, model_type, ...])	Build the model.
pretrain(args, ...[, ...])	Main training program.
print_datetime(string)	Note that this call will sync across all ranks.
save_checkpoint_and_time(iteration, model, ...)
train_step(forward_step_func, data_iterator, ...)	Single training step.
training_log(loss_dict, total_loss_dict, ...)	Log training information such as losses, timing, ....