megatron.text_generation.forward_step

Description

Forward step utilities.

Classes

ForwardStep(model, max_batch_size, ...)	Forward step function with all the communications.
InferenceParams(max_batch_size, max_sequence_len)	Inference parameters that are passed to the main model in order to efficienly calculate and store the context during inference.