megatron.optimizer.distrib\_optimizer
=====================================

.. rubric:: Description

.. automodule:: megatron.optimizer.distrib_optimizer

.. currentmodule:: megatron.optimizer.distrib_optimizer


.. rubric:: Classes

.. autosummary::
    :toctree: .
    
    DistributedOptimizer
    
    Range