megatron.core.tensor_parallel.utils.VocabUtility#

class megatron.core.tensor_parallel.utils.VocabUtility#

Bases: object

Split the vocabulary into world_size chunks and return the first and last index of the vocabulary belonging to the rank partition: Note that indices in [fist, last)