megatron.data.dataset_utils.compile_helper#

megatron.data.dataset_utils.compile_helper()#

Compile helper function ar runtime. Make sure this is invoked on a single process.