megatron.text_generation.api.generate_and_post_process

megatron.text_generation.api.generate_and_post_process(model, prompts=None, tokens_to_generate=0, return_output_log_probs=False, top_k_sampling=0, top_p_sampling=0.0, top_p_decay=0.0, top_p_bound=0.0, temperature=1.0, add_BOS=False, use_eod_token_for_early_termination=True, stop_on_double_eol=False, stop_on_eol=False, prevent_newline_after_colon=False, random_seed=-1)

Run inference and post-process outputs, i.e., detokenize, move to cpu and convert to list.