megatron.microbatches

Description

Megatron number of micro-batches calculators.

Classes

ConstantNumMicroBatches(global_batch_size, ...)
NumMicroBatchesCalculator()
RampupBatchsizeNumMicroBatches(...)

Functions

build_num_microbatches_calculator(args)