megatron.optimizer.optimizer

Description

Megatron optimizer.

Classes

FP32Optimizer(optimizer, clip_grad, ...)
Float16OptimizerWithFloat16Params(optimizer, ...)	Float16 optimizer for fp16 and bf16 data types.
MegatronOptimizer(optimizer, clip_grad, ...)
MixedPrecisionOptimizer(optimizer, ...)	Base class for both the float-16 and the distributed optimizer.