megatron.optimizer.optimizer#
Description
Megatron optimizer.
Classes
|
|
|
Float16 optimizer for fp16 and bf16 data types. |
|
|
|
Base class for both the float-16 and the distributed optimizer. |
Description
Megatron optimizer.
Classes
|
|
|
Float16 optimizer for fp16 and bf16 data types. |
|
|
|
Base class for both the float-16 and the distributed optimizer. |